Language modelActive

Grok 4.20 Reasoning

xAI

Released: March 10, 2026
Data date: September 8, 2026

Benchmark-Meta-Score: Unavailable
Results from one source only
Speed: 105.9 tokens/s
Output throughput
Price: $1.25 · $2.5
Input · Output / 1M tokens
Input: Text, Image
Output: Text

Grok 4.20 Reasoning is aimed at one controllable, multi-step reasoning and tool-calling run. The model accepts text and images and provides one million tokens of context for substantial working material.

This variant uses the API identifier grok-4.20-0309-reasoning. It separates the single reasoning and tool-calling run from Grok 4.20 Multi-Agent, which is aimed at parallel research. xAI publishes neither weights nor a parameter count.

Specifications and access

Specification	Value and source
Weights	ProprietarySource
Access	APISource
Context window	1,000,000 tokensSource
Notes	1M context, reasoning mode and agentic tool calling, parameter count not disclosedSource
API model ID	grok-4.20-0309-reasoningSource
Source status	activeSource
Release	2026-03-10Source
Maximum output	not statedSource
Input	Text, ImageSource
Output	TextSource
Tool calling	YesSource
Structured output	YesSource
Model aliases	grok-4.20-reasoningSource
Reasoning mode	low, medium, highSource

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-10-15 - 2026-10-15

Overall index

Not enough independent results for a score yet

An overall score needs results from at least two sources.

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Test series: 22
Test families: 20 / 65
Sources: 1
Share of the fixed test panel (informative): 30 %

Coding

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.4
95% interval: 0.0 to 5.4
Test series: 6
Test families: 5 / 15
Sources: 1
Share of the fixed test panel (informative): 26 %

Reasoning

Index 1.9 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.4
95% interval: 0.4 to 6.8
Test series: 1
Test families: 1 / 7
Sources: 1
Share of the fixed test panel (informative): 7 %

Knowledge

Index 1.3 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.9
95% interval: 0.3 to 4.3
Test series: 1
Test families: 1 / 3
Sources: 1
Share of the fixed test panel (informative): 33 %

Finance

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.1
95% interval: 0.0 to 1.8
Test series: 6
Test families: 6 / 7
Sources: 1
Share of the fixed test panel (informative): 86 %

Legal

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.4
95% interval: 0.0 to 2.5
Test series: 3
Test families: 3 / 4
Sources: 1
Share of the fixed test panel (informative): 75 %

Health

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.1
95% interval: 0.0 to 0.6
Test series: 3
Test families: 3 / 3
Sources: 1
Share of the fixed test panel (informative): 100 %

Multimodal

Index 1.5 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.1
95% interval: 0.3 to 5.4
Test series: 1
Test families: 1 / 4
Sources: 1
Share of the fixed test panel (informative): 13 %

Education

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.1
95% interval: 0.0 to 0.5
Test series: 1
Test families: 1 / 2
Sources: 1
Share of the fixed test panel (informative): 50 %

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v3

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. Unknown or ambiguous tiers exclude the model from that series. Within each series, percentages and rates are logit-transformed, other units stay unchanged, and all values are then standardized. A one-factor model then estimates one shared strength per model and, per series, how sharply it separates the models. Area scores move away from that strength only as far as the model's own results in the area support. The overall score weights the core areas at 75% and finance, law and health at 25%. It requires at least eight test series from two sources and three areas. The 0 to 100 scale refers to a fixed reference group of 29 language models whose average sits at 50.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured series are not entered as zero. Where a model has no own results in a core area, its overall strength stands in, and the note “estimated without own results in …” names the area. Fewer test series lead to a larger standard error.

The 95% interval follows from the standard error of 200 recalculations with randomly redrawn test families. It describes how precisely the recorded test series determine the value. It is no prediction for future tasks or for later recalculations with other participants.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 21 test families and 12 of 22 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: GPQA Diamond
Reasoning · Mean rank percentile of the family: 65.8 / 100
Family weight in the fixed test panel (informative): 1.53 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-gpqa
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.77 %
  Raw value:
  88.636 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  77
  Rank percentile within the cohort:
  65.8 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-gpqa:1:overall
  Documented comparison setup
  Publication cohort:
  vals-gpqa:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 59.421. Standard error: 1.594. Cost per task in USD: 0.034461. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-10-15-951eb5224097. Source payload hash: 951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MMMU-Pro
Multimodal · Mean rank percentile of the family: 48.9 / 100
Family weight in the fixed test panel (informative): 2.68 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-mmmu
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  1.34 %
  Raw value:
  83.468 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  48
  Rank percentile within the cohort:
  48.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-mmmu:1:overall
  Documented comparison setup
  Publication cohort:
  vals-mmmu:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 44.461. Standard error: 0.893. Cost per task in USD: 0.026669. Source ID: vals-ai. Evidence capture ID: vals-mmmu-2026-10-15-29395186eddd. Source payload hash: 29395186eddd2c70601228c584a14b3c1d8f9dba4b68256561b46b0dd03c635a. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: SWE-bench Verified
Coding · Mean rank percentile of the family: 33.9 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-swebench
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.36 %
  Raw value:
  72.2 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  60
  Rank percentile within the cohort:
  33.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-swebench:1:overall
  Documented comparison setup
  Publication cohort:
  vals-swebench:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 134.759. Standard error: 2.006. Cost per task in USD: 0.515946. Source ID: vals-ai. Evidence capture ID: vals-swebench-2026-10-15-7a4e2e848528. Source payload hash: 7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Terminal-Bench
Coding · Mean rank percentile of the family: 30.7 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 33 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-terminal-bench-2
  Series weight within the test family:
  16.67 %
  Nominal contribution to the overall panel:
  0.12 %
  Raw value:
  40.449 %
  Metric:
  accuracy
  Version:
  2.0
  Participating models:
  43
  Rank percentile within the cohort:
  48.8 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-terminal-bench-2:2.0:overall
  Documented comparison setup
  Publication cohort:
  vals-terminal-bench-2:2.0:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-06-04. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 161.408. Standard error: 5.232. Cost per task in USD: 0.154109. Source ID: vals-ai. Evidence capture ID: vals-terminal-bench-2-2026-10-15-38e640d99525. Source payload hash: 38e640d995259778e531f048b2f2c7b3b281cb05b56be643ddc9dc53eb85adf7. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-terminal-bench-2-1
  Series weight within the test family:
  16.67 %
  Nominal contribution to the overall panel:
  0.12 %
  Raw value:
  44.195 %
  Metric:
  accuracy
  Version:
  2.1
  Participating models:
  57
  Rank percentile within the cohort:
  12.5 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-terminal-bench-2-1:2.1:overall
  Documented comparison setup
  Publication cohort:
  vals-terminal-bench-2-1:2.1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-28. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 305.726. Standard error: 0.991. Cost per task in USD: 0.407362. Source ID: vals-ai. Evidence capture ID: vals-terminal-bench-2-1-2026-10-15-a5306805cdf8. Source payload hash: a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Code Migration
Coding · Mean rank percentile of the family: 1.6 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-code-migration
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.71 %
  Raw value:
  0.314 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  63
  Rank percentile within the cohort:
  1.6 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-code-migration:1:overall
  Documented comparison setup
  Publication cohort:
  vals-code-migration:1:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 373.673. Standard error: 0.11. Cost per task in USD: 0.289486. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: CorpFin v2
Finance · Mean rank percentile of the family: 62.2 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-corp-fin-v2
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  63.675 %
  Metric:
  accuracy
  Version:
  2
  Participating models:
  75
  Rank percentile within the cohort:
  62.2 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-corp-fin-v2:2:overall
  Documented comparison setup
  Publication cohort:
  vals-corp-fin-v2:2:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 9.67. Standard error: 0.947. Cost per task in USD: 0.163386. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: EMB
Finance · Mean rank percentile of the family: 5.1 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-emb
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  11.739 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  60
  Rank percentile within the cohort:
  5.1 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-emb:1:overall
  Documented comparison setup
  Publication cohort:
  vals-emb:1:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 367.615. Standard error: 1.553. Cost per task in USD: 0.469926. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-10-15-e1170aaade72. Source payload hash: e1170aaade72d6b81079f9cb90244c41f028a12e29d982c3ec1d6e9913a9872d. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Finance Agent (v2)
Finance · Mean rank percentile of the family: 5.0 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-finance-agent-v2
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  28.492 %
  Metric:
  accuracy
  Version:
  2
  Participating models:
  61
  Rank percentile within the cohort:
  5.0 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-finance-agent-v2:2:overall
  Documented comparison setup
  Publication cohort:
  vals-finance-agent-v2:2:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 351.413. Standard error: 0.315. Cost per task in USD: 0.791809. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-10-15-3780be1a23c9. Source payload hash: 3780be1a23c9133a3c34a6095024af162b35a8ef50963a62fc67a6049880426c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: LiveCodeBench v6
Coding · Mean rank percentile of the family: 64.9 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-lcb
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.71 %
  Raw value:
  84.265 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  78
  Rank percentile within the cohort:
  64.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-lcb:1:overall
  Documented comparison setup
  Publication cohort:
  vals-lcb:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 76.09. Standard error: 1.028. Cost per task in USD: 0.05249. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-10-15-1a7c7dabc9d9. Source payload hash: 1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Harvey's Legal Agent Benchmark
Legal · Mean rank percentile of the family: 10.7 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-legal-agent-benchmark
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  0 %
  Metric:
  task resolution rate
  Version:
  1
  Participating models:
  62
  Rank percentile within the cohort:
  10.7 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-legal-agent-benchmark:1:overall
  Documented comparison setup
  Publication cohort:
  vals-legal-agent-benchmark:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 502.903. Standard error: 0. Cost per task in USD: 0.986814. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-10-15-7587eca48586. Source payload hash: 7587eca48586be017c9db8c1a11e6f6797b1eb62b7627cbf198f35f592f4fc69. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: LegalBench
Legal · Mean rank percentile of the family: 20.5 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-legal-bench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  77.738 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  79
  Rank percentile within the cohort:
  20.5 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-legal-bench:1:overall
  Documented comparison setup
  Publication cohort:
  vals-legal-bench:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000
  Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-13. Temperature: 0.7. Top P: 0.95. Maximum output tokens: 2000000. Latency in seconds: 7.574. Standard error: 0.48. Cost per task in USD: 0.006065. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-10-15-c8486ba756ba. Source payload hash: c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Legal Research Bench
Legal · Mean rank percentile of the family: 15.8 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
Test family: MedCode
Health · Mean rank percentile of the family: 9.0 / 100
Family weight in the fixed test panel (informative): 2.78 % · Coverage of this test family: 100 %
Test family: MedQA
Health · Mean rank percentile of the family: 76.9 / 100
Family weight in the fixed test panel (informative): 2.78 % · Coverage of this test family: 100 %
Test family: MedScribe
Health · Mean rank percentile of the family: 4.4 / 100
Family weight in the fixed test panel (informative): 2.78 % · Coverage of this test family: 100 %
Test family: MMLU-Pro
Knowledge · Mean rank percentile of the family: 61.0 / 100
Family weight in the fixed test panel (informative): 3.57 % · Coverage of this test family: 100 %
Test family: MortgageTax
Finance · Mean rank percentile of the family: 1.9 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
Test family: SAGE
Education · Mean rank percentile of the family: 22.4 / 100
Family weight in the fixed test panel (informative): 0 % · Coverage of this test family: 100 %
Test family: Tax Agent Bench
Finance · Mean rank percentile of the family: 12.7 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Mean rank percentile of the family: 61.8 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Mean rank percentile of the family: 11.1 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	0	0	6
Vals AI	24	1	186

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

6 measurements from these sources still lack a verified test-family identity and do not increase the test-series count.

Published benchmarks

Taste-Bench

Overall

502 technical decision points from engineering and research workflows. Each choice is presented in both answer orders. The overall score averages engineering and research and requires both answer orders to be correct. Unparseable responses count as wrong. Reasoning settings and output budget are part of the setup.

Note (Grok 4.20 Reasoning): 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

1.GPT-5.6 Sol59.7%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

2.GPT-5.559.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

3.Claude Opus 555.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

4.Grok 4.554.6%

Note: 10 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

5.GPT-5.6 Terra54%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

6.GLM-5.253.9%

Note: 17 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

7.Claude Sonnet 551.6%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

8.GPT-5.6 Luna49%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

9.MiniMax M345.3%

Note: 3 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

10.DeepSeek-V4-Flash43.3%

Note: 2 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

11.GPT-5.4 mini40.1%

Note: 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

12.Mistral Medium 3.537.7%

Note: 24 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

13.GPT-5.4 nano36.6%

Note: 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

14.Grok 4.20 Reasoning15.7%

higher is betterAs of September 26, 2026Scale: 0% - 100%

Source: Taste-Bench paper

|

gradually.ai

Methodology and context

Comparison cohort: taste-bench-paper-v2-502
Metric: both-orders-correct, engineering/research macro average
Scale: 0-100
Participants: 14
Methodology and context: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/grok-4.20-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: xAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-b780156e6976. Source payload hash: b780156e6976a567e249d97bf51e2088015b07b0c2aa65663f57ed1fba7c45b0. Parser version: curated-v1. Imported at: 2026-10-15

Source: Taste-Bench paper

Health

MedCode

Overall

1.MiniMax M346.29%

2.GLM-5.141.6%

3.Nemotron 3 Ultra38.62%

4.Grok 4.338.07%

5.MiniMax M2.734.44%

6.MiniMax M2.134.08%

7.MiMo-V2.5-Pro32.48%

8.Ling 3.0 Flash32.28%

9.Grok 4.20 Reasoning32.16%

10.MiMo-V2.531.9%

higher is betterAs of October 15, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-medcode:1:overall
Metric: accuracy
Scale: 0-100
Participants: 10
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-13. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 16.546. Standard error: 2.124. Cost per task in USD: 0.036189. Source ID: vals-ai. Evidence capture ID: vals-medcode-2026-10-15-988a6ae1e07e. Source payload hash: 988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

MedScribe

Overall

1.MiniMax M387.25%

2.MiMo-V2.5-Pro83.73%

3.Ling 3.0 Flash80.9%

4.MiniMax M2.180.78%

5.MiniMax M2.779.87%

6.Grok 4.374.4%

7.GLM-5.172.27%

8.MiMo-V2.572.15%

9.Grok 4.20 Reasoning63.41%

10.Nemotron 3.5 Lightning4.27%

higher is betterAs of October 15, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-medscribe:1:overall
Metric: accuracy
Scale: 0-100
Participants: 10
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-13. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 18.596. Standard error: 2.095. Cost per task in USD: 0.031303. Source ID: vals-ai. Evidence capture ID: vals-medscribe-2026-10-15-271fc122ddc2. Source payload hash: 271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

Education

SAGE

Sub-task	Grok 4.20 Reasoning (This model)	MiniMax M3	Qwen 3.6 27B	Grok 4.3	MiMo-V2.5
Automata	40.48% (best value in this row)	37.3%	35.71%	35.32%	29.76%
Linear Algebra	39.33%	66.8% (best value in this row)	59.75%	18.33%	58.76%
Overall	38.24%	50.57% (best value in this row)	45.58%	19.74%	43.27%
Partial Differential Equations	34.92%	47.62% (best value in this row)	41.27%	5.56%	41.27%

higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

SAGE

Calculus

1.Grok 4.348.15%

2.Grok 4.20 Reasoning40.12%

higher is betterAs of October 15, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-sage:1:calculus
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 119.528. Standard error: 6.472. Cost per task in USD: 0.130055. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

Vals Multimodal Index

SWE-bench

1.GPT-5.6 Sol99.02%

2.Claude Fable 597.06%

3.Claude Opus 595.1%

3.Kimi K395.1%

5.GPT-5.6 Luna93.14%

6.Claude Opus 4.889.22%

7.Qwen 3.8 Max 090287.26%

8.Grok 4.586.28%

9.GPT-5.582.6%

10.Claude Opus 4.782.35%

11.Claude Sonnet 4.677.45%

11.Gemini 3.6 Flash77.45%

13.Claude Sonnet 575.49%

13.Gemini 3.5 Flash75.49%

13.Inkling75.49%

13.Inkling Small75.49%

17.Kimi K2.674.51%

18.GPT-5.6 Terra73.53%

19.Gemini 3.1 Pro Preview72.55%

20.Grok 4.370.59%

21.MiniMax M369.61%

22.Gemini 3 Flash Preview68.63%

23.GPT-5.4 mini67.65%

23.GPT-5.4 nano67.65%

25.Grok 4.20 Reasoning66.67%

25.Qwen 3.7 Plus66.67%

27.MiMo-V2.564.71%

27.Mistral Medium 3.564.71%

29.Gemini 3.1 Flash-Lite57.84%

higher is betterAs of October 15, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-vals-multimodal-index:1.2:swebench
Metric: benchmark score
Scale: 0-100
Participants: 29
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-11. Latency in seconds: 151.343. Standard error: 4.668. Cost per task in USD: 0.586871. Source ID: vals-ai. Evidence capture ID: vals-vals_multimodal_index-2026-10-15-077b22575f6e. Source payload hash: 077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

AIME: AIME 2024 (Show measurement, test conditions, and source)

Source value: 98.75
Score: 98.75
Metric: accuracy
Unit: %
Benchmark version: 1
Category: math
Task: aime_2024
Task label: AIME 2024
Direction: Higher is better
Comparison cohort: vals-aime:1:aime_2024
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Temperature: 0.7. Top P: 0.95. Latency in seconds: 97.839. Standard error: 0.61. Cost per task in USD: 0.051914. Source ID: vals-ai. Evidence capture ID: vals-aime-2026-10-15-7d016210b39d. Source payload hash: 7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 0.7. Top P: 0.95
Context: The source dataset marks this benchmark as archived. The result remains visible as historical evidence and is not interchangeable with an active leaderboard.

AIME: AIME 2025 (Show measurement, test conditions, and source)

Source value: 94.167
Score: 94.167
Metric: accuracy
Unit: %
Benchmark version: 1
Category: math
Task: aime_2025
Task label: AIME 2025
Direction: Higher is better
Comparison cohort: vals-aime:1:aime_2025
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Temperature: 0.7. Top P: 0.95. Latency in seconds: 115.759. Standard error: 0.833. Cost per task in USD: 0.06688. Source ID: vals-ai. Evidence capture ID: vals-aime-2026-10-15-7d016210b39d. Source payload hash: 7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 0.7. Top P: 0.95
Context: The source dataset marks this benchmark as archived. The result remains visible as historical evidence and is not interchangeable with an active leaderboard.

AIME: Overall (Show measurement, test conditions, and source)

Source value: 96.458
Score: 96.458
Metric: accuracy
Unit: %
Benchmark version: 1
Category: math
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-aime:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Temperature: 0.7. Top P: 0.95. Latency in seconds: 106.799. Standard error: 0.516. Cost per task in USD: 0.059397. Source ID: vals-ai. Evidence capture ID: vals-aime-2026-10-15-7d016210b39d. Source payload hash: 7d016210b39d24351bbb5d12be09a973decac461bb2486eb1f23724fed2b7e6c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 0.7. Top P: 0.95
Context: The source dataset marks this benchmark as archived. The result remains visible as historical evidence and is not interchangeable with an active leaderboard.

Recorded measurement details

Taste-Bench: Overall (Show measurement, test conditions, and source)

Source value: 15.7
Score: 15.7
Metric: both-orders-correct, engineering/research macro average
Unit: %
Benchmark version: arxiv:2609.25804v2
Category: agentic
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: taste-bench-paper-v2-502
Source type: independent-evaluator
Evaluator: Taste-Bench paper
Status: active
Retrieved at: 2026-09-26
Methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/grok-4.20-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: xAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-b780156e6976. Source payload hash: b780156e6976a567e249d97bf51e2088015b07b0c2aa65663f57ed1fba7c45b0. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.
Context: 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

Source Taste-Bench paper

MedCode: Overall (Show measurement, test conditions, and source)

Source value: 32.156
Score: 32.156
Metric: accuracy
Unit: %
Benchmark version: 1
Category: health
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-medcode:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-13. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 16.546. Standard error: 2.124. Cost per task in USD: 0.036189. Source ID: vals-ai. Evidence capture ID: vals-medcode-2026-10-15-988a6ae1e07e. Source payload hash: 988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

MedScribe: Overall (Show measurement, test conditions, and source)

Source value: 63.412
Score: 63.412
Metric: accuracy
Unit: %
Benchmark version: 1
Category: health
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-medscribe:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-13. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 18.596. Standard error: 2.095. Cost per task in USD: 0.031303. Source ID: vals-ai. Evidence capture ID: vals-medscribe-2026-10-15-271fc122ddc2. Source payload hash: 271fc122ddc2b0602bd6c7947f65a9a7f0d8cab94f6dc27d62a9380a9317f9c0. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

SAGE: Automata (Show measurement, test conditions, and source)

Source value: 40.476
Score: 40.476
Metric: accuracy
Unit: %
Benchmark version: 1
Category: education
Task: automata
Task label: Automata
Direction: Higher is better
Comparison cohort: vals-sage:1:automata
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 84.742. Standard error: 6.345. Cost per task in USD: 0.044921. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

SAGE: Linear Algebra (Show measurement, test conditions, and source)

Source value: 39.329
Score: 39.329
Metric: accuracy
Unit: %
Benchmark version: 1
Category: education
Task: linear_algebra
Task label: Linear Algebra
Direction: Higher is better
Comparison cohort: vals-sage:1:linear_algebra
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 49.23. Standard error: 5.459. Cost per task in USD: 0.083369. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

SAGE: Overall (Show measurement, test conditions, and source)

Source value: 38.242
Score: 38.242
Metric: accuracy
Unit: %
Benchmark version: 1
Category: education
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-sage:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 66.358. Standard error: 3.425. Cost per task in USD: 0.063231. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

SAGE: Partial Differential Equations (Show measurement, test conditions, and source)

Source value: 34.921
Score: 34.921
Metric: accuracy
Unit: %
Benchmark version: 1
Category: education
Task: PDEs
Task label: Partial Differential Equations
Direction: Higher is better
Comparison cohort: vals-sage:1:PDEs
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 65.101. Standard error: 5.962. Cost per task in USD: 0.061403. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

SAGE: Calculus (Show measurement, test conditions, and source)

Source value: 40.123
Score: 40.123
Metric: accuracy
Unit: %
Benchmark version: 1
Category: education
Task: calculus
Task label: Calculus
Direction: Higher is better
Comparison cohort: vals-sage:1:calculus
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: SpaceXAI. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-08. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 119.528. Standard error: 6.472. Cost per task in USD: 0.130055. Source ID: vals-ai. Evidence capture ID: vals-sage-2026-10-15-cc5fb3fecdb0. Source payload hash: cc5fb3fecdb0382d492ff167a42ad84b9c5c3883373c78ed87baaa972b8b6f7c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Temperature: 1. Top P: 0.95. Maximum output tokens: 30000

Vals Multimodal Index: SWE-bench (Show measurement, test conditions, and source)

Source value: 66.667
Score: 66.667
Metric: benchmark score
Unit: %
Benchmark version: 1.2
Category: aggregate
Task: swebench
Task label: SWE-bench
Direction: Higher is better
Comparison cohort: vals-vals-multimodal-index:1.2:swebench
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-11. Latency in seconds: 151.343. Standard error: 4.668. Cost per task in USD: 0.586871. Source ID: vals-ai. Evidence capture ID: vals-vals_multimodal_index-2026-10-15-077b22575f6e. Source payload hash: 077b22575f6e075b7b78157d000ec8f94d7ead409c7dcbcf04887630ed572a2d. Parser version: vals-astro-v3. Imported at: 2026-10-15

Page 1 of 17

Sources and data date

Type	Evidence and data date
Speed	SpaceXAI. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 3 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording xAI does not publish parameter counts on the model page.
Not found in the inspected sources	active-parameters-billions Original wording xAI does not publish active parameter counts on the model page.
Not found in the inspected sources	knowledge-cutoff Original wording The official xAI model page did not state a knowledge cutoff.
Not found in the inspected sources	profile-benchmarks Original wording The exact model documentation page exposed capabilities and pricing but no benchmark table.
Additional source	Taste-Bench paper (retrieved September 26, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0