Grok 4.20 Reasoning: Specifications & Sources

502 technical decision points from engineering and research workflows. Each choice is presented in both answer orders. The overall score averages engineering and research and requires both answer orders to be correct. Unparseable responses count as wrong. Reasoning settings and output budget are part of the setup.

Note (Grok 4.20 Reasoning): 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

1.GPT-5.6 Sol59.7%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

2.GPT-5.559.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

3.Claude Opus 555.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

4.Grok 4.554.6%

Note: 10 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

5.GPT-5.6 Terra54%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

6.GLM-5.253.9%

Note: 17 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

7.Claude Sonnet 551.6%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

8.GPT-5.6 Luna49%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

9.MiniMax M345.3%

Note: 3 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

10.DeepSeek-V4-Flash43.3%

Note: 2 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

11.GPT-5.4 mini40.1%

Note: 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

12.Mistral Medium 3.537.7%

Note: 24 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

13.GPT-5.4 nano36.6%

Note: 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

14.Grok 4.20 Reasoning15.7%

higher is betterAs of September 26, 2026Scale: 0% - 100%

Source: Taste-Bench paper

|

gradually.ai

Methodology and context

Comparison cohort: taste-bench-paper-v2-502
Metric: both-orders-correct, engineering/research macro average
Scale: 0-100
Participants: 14
Methodology and context: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/grok-4.20-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: xAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-b780156e6976. Source payload hash: b780156e6976a567e249d97bf51e2088015b07b0c2aa65663f57ed1fba7c45b0. Parser version: curated-v1. Imported at: 2026-09-29

Source: Taste-Bench paper

Coding

Code Migration

Overall

1.Claude Sonnet 5.569.83%

2.GPT-6 Astra67.74%

3.Claude Opus 5.566.65%

4.Claude Opus 557.47%

5.GPT-6 Sol57.2%

6.Claude Fable 555.07%

7.Claude Fable 5.154.61%

8.GPT-5.6 Sol52.92%

9.GPT-5.6 Terra47.8%

10.Claude Opus 4.847.25%

11.DeepSeek-V4.1-Flash45.62%

12.GPT-5.545.16%

13.Grok 4.744.82%

14.Grok 4.644.57%

15.GPT-5.6 Luna44.55%

16.Claude Sonnet 544.39%

17.GLM-5.344.22%

18.Claude Opus 4.743.88%

19.GPT-6 Luna42.55%

20.Claude Sonnet 4.639.89%

21.GLM-5.237.87%

22.Grok 4.536.6%

23.Gemini 3.8 Flash36.55%

24.GPT-5.434.98%

25.Gemini 3.7 Flash34.8%

26.Gemini 3.6 Flash30.93%

27.Kimi K2.627.77%

28.Gemini 3.5 Flash26.75%

29.DeepSeek-V4-Pro26.2%

30.GLM-5.125.77%

31.Kimi K2.7 Code25.39%

32.Qwen 3.8 Max 090223.96%

33.MiMo-V2.5-Pro21.56%

34.GLM-5.3-Flash20.52%

35.MiniMax M319.93%

36.Gemini 3.1 Pro Preview17.31%

37.Kimi K316.1%

38.GPT-5.4 nano14.47%

39.MiMo-V2.514.23%

40.Qwen 3.8 27B14.16%

41.Inkling Small13.68%

42.Qwen 3.7 Max13.07%

43.GPT-5.4 mini12.94%

44.Qwen 3.7 Plus12.86%

45.Inkling11.79%

46.MiniMax M2.78.69%

47.Kimi K2.56.96%

48.Grok 4.36.8%

49.Gemini 3 Flash Preview6.39%

50.Gemini 3.5 Flash-Lite6.15%

51.Mistral Medium 3.55.13%

52.Nemotron 3 Ultra4.91%

53.Gemini 3.1 Flash-Lite4.61%

54.Mercury 2.54.49%

55.Nemotron 3.5 Lightning2.95%

56.Grok 4.20 Reasoning0.31%

57.Ling 3.0 Flash0%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-code-migration:1:overall
Metric: accuracy
Scale: 0-100
Participants: 57
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 373.673. Cost per task in USD: 0.289486. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-09-29-f0b317d5bed3. Source payload hash: f0b317d5bed3a2654a9ea39762d5e57f4649306adce3e6103e845ee1b9b94add. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

LiveCodeBench v6

Overall

1.Claude Fable 5.190.52%

2.Claude Fable 589.78%

3.Gemini 3.8 Flash89.48%

4.Claude Opus 589.03%

5.Gemini 3.7 Flash88.65%

6.Gemini 3.1 Pro Preview88.49%

7.Grok 4.688.22%

8.Gemini 3.6 Flash88.08%

9.Qwen 3.8 Max 090287.85%

10.Claude Opus 4.887.82%

11.Gemini 3.5 Flash87.6%

12.DeepSeek-V4-Pro87.48%

13.Grok 4.587.35%

14.GPT-5.3-Codex87.31%

15.Kimi K387.19%

16.Qwen 3.7 Max87.06%

17.Kimi K2.686.77%

18.GPT-5 mini86.61%

19.Nemotron 3 Ultra85.98%

20.Inkling Small85.93%

21.GPT-5.6 Terra85.93%

22.GPT-585.91%

23.Gemini 3 Flash Preview85.59%

24.Inkling85.52%

25.GPT-5.585.3%

26.Claude Opus 4.785.07%

27.Grok 4.384.49%

28.Grok 4.20 Reasoning84.27%

29.GPT-5.484.14%

30.GPT-5.4 nano84.01%

31.Qwen 3.8 27B84%

32.Ling 3.0 Flash83.99%

33.o383.91%

34.Kimi K2.583.87%

35.GPT OSS 120B83.23%

36.GPT-5.6 Sol82.6%

37.Claude Sonnet 582.43%

38.GLM-4.782.23%

39.o4-mini82.21%

40.MiniMax M382.15%

41.Claude Sonnet 4.682.09%

42.Kimi K2.7 Code82.05%

43.GLM-581.87%

44.MiniMax M2.181.76%

45.MiMo-V2.581.51%

46.GPT-5.4 mini81.47%

47.GLM-5.181.38%

48.MiMo-V2.5-Pro81.35%

49.GLM-4.681.04%

50.GLM-5.380.54%

51.GLM-5.3-Flash80.51%

52.GPT OSS 20B80.39%

53.Gemini 3.1 Flash-Lite80.12%

54.MiniMax M2.779.93%

55.MiniMax M2.579.21%

56.Gemini 3.5 Flash-Lite79.01%

57.Claude Opus 4.575.03%

58.o3-mini71.48%

59.Kimi K2 071170.45%

60.DeepSeek-R170.22%

61.GPT-5 nano70.22%

62.DeepSeek-V3.269.86%

63.GLM-5.269.5%

64.GLM-4.567.45%

65.DeepSeek-V3 032465.48%

66.Kimi K2 Thinking63.15%

67.GPT-4.1 mini58.16%

68.Mistral Large 355.34%

69.GPT-4.154.67%

70.o150.26%

71.GPT-4o43.44%

72.GPT-4.1 nano42.72%

73.Llama 4 Scout38.54%

74.Command A35.07%

75.Mistral Small 3.131.82%

76.GPT-4o mini26.42%

77.Command R+18.24%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-lcb:1:overall
Metric: accuracy
Scale: 0-100
Participants: 77
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 76.09. Cost per task in USD: 0.05249. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-09-29-b6ee1006a41a. Source payload hash: b6ee1006a41aab0c68eef9d3e42a2151634fffaaeb6bdc2230a6d02994198a59. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Finance

CorpFin v2

Overall

1.Claude Opus 573.19%

2.Claude Fable 571.83%

3.Kimi K371.56%

4.Inkling Small69.62%

5.Inkling68.57%

6.Grok 4.368.53%

7.GPT-5.568.42%

8.Kimi K2.568.26%

9.MiniMax M368.1%

10.Grok 4.567.41%

11.Claude Sonnet 566.98%

12.Kimi K2.666.74%

13.Claude Opus 4.866.71%

14.Gemini 3 Flash Preview66.43%

15.Grok 4.666.16%

16.GLM-5.266.12%

17.Claude Opus 4.766.08%

18.Qwen 3.8 Max 090265.85%

19.Nemotron 3 Ultra65.46%

20.Claude Sonnet 4.665.31%

20.GPT-5.6 Terra65.31%

22.GPT-5.465.27%

23.Gemini 3.5 Flash64.69%

24.Gemini 3.1 Pro Preview64.49%

25.GLM-5.164.45%

26.GPT-5.6 Sol64.38%

27.GPT-5.6 Luna64.22%

28.Qwen 3.7 Max63.71%

29.Grok 4.20 Reasoning63.68%

30.Gemini 3.6 Flash63.33%

31.GPT-4.163.05%

32.GLM-562.9%

33.Qwen 3.6 27B62.32%

34.Ling 3.0 Flash61.93%

35.MiMo-V2.5-Pro61.42%

36.DeepSeek-V4-Pro61.38%

37.Claude Opus 4.561.31%

38.GPT-5.4 nano61.19%

38.MiniMax M2.761.19%

40.GPT-561.07%

41.Mistral Large 361.03%

42.GLM-4.560.96%

43.GPT-5.4 mini60.92%

44.Claude Sonnet 4.560.8%

44.Gemini 2.5 Pro60.8%

46.Gemini 3.5 Flash-Lite60.68%

47.Kimi K2 Thinking60.57%

48.Claude Haiku 4.560.3%

49.GPT-5 mini60.18%

50.MiMo-V2.559.91%

51.o359.71%

52.MiniMax M2.559.6%

53.Gemini 3.1 Flash-Lite59.36%

54.o4-mini58.97%

55.MiniMax M2.158.9%

56.Mistral Medium 3.558.78%

57.GPT OSS 120B58.24%

58.GPT-4.1 mini57.93%

59.GLM-4.656.84%

60.DeepSeek-V3 032454.74%

61.Trinity Large Thinking54.66%

62.Nemotron 3.5 Lightning54.24%

63.DeepSeek-R154.12%

64.GPT OSS 20B53.15%

65.DeepSeek-V352.49%

66.DeepSeek-V3.151.48%

67.Kimi K2 071150.39%

68.DeepSeek-V3.247.94%

69.Llama 4 Scout46.78%

70.GLM-4.746.39%

71.Command A45.96%

72.GPT-4o mini45.46%

73.o3-mini45.3%

74.Mistral Small 3.144.17%

75.GPT-4.1 nano42.08%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-corp-fin-v2:2:overall
Metric: accuracy
Scale: 0-100
Participants: 75
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Latency in seconds: 9.67. Cost per task in USD: 0.163386. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-09-29-83186cfafe52. Source payload hash: 83186cfafe526f0b8fe2ae86af05e8209fe799c8bfa48292ef1fef5db66650ee. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

EMB

Overall

1.Claude Fable 5.176.67%

2.Claude Opus 5.575.94%

3.Claude Sonnet 5.575.71%

4.Claude Fable 573.67%

5.Claude Opus 573.56%

6.GPT-5.6 Sol72.34%

7.Gemini 3.8 Flash72.2%

8.GPT-6 Astra71.7%

9.GPT-6 Sol71.53%

10.Gemini 3.7 Flash71.33%

11.Claude Opus 4.869.37%

12.GPT-6 Luna68.52%

13.GPT-5.6 Luna67.12%

14.Grok 4.766.99%

15.Kimi K366.4%

16.Claude Sonnet 566.32%

17.GPT-5.6 Terra66.21%

18.Gemini 3.6 Flash65.41%

19.GPT-5.564.54%

20.Claude Opus 4.763.73%

21.Gemini 3.5 Flash63.55%

22.Grok 4.662.73%

23.GLM-5.261.53%

24.Claude Sonnet 4.660.15%

25.Qwen 3.8 Max 090260.07%

26.Qwen 3.8 27B59.66%

27.Kimi K2.657.86%

28.DeepSeek-V4.1-Flash57.21%

29.Qwen 3.7 Max57.04%

30.GLM-5.356.34%

31.GLM-5.3-Flash55.93%

32.MiMo-V2.5-Pro55.23%

33.MiMo-V2.555.09%

34.Grok 4.552.91%

35.Gemini 3.1 Pro Preview52.62%

36.DeepSeek-V4-Pro51.62%

37.Qwen 3.7 Plus49.35%

38.MiniMax M347.76%

39.GPT-5.4 mini45.43%

40.GPT-5.4 nano44.75%

41.Gemini 3.5 Flash-Lite43.23%

42.Inkling38.36%

43.Inkling Small31.76%

44.Nemotron 3 Ultra31.74%

45.Kimi K2.528.47%

46.MiniMax M2.728.45%

47.Gemini 3 Flash Preview26.02%

48.Ling 3.0 Flash25.01%

49.Grok 4.318.39%

50.Nemotron 3.5 Lightning16.99%

51.Grok 4.20 Reasoning11.74%

52.Mistral Medium 3.511.19%

53.Mercury 2.58.82%

54.Gemini 3.1 Flash-Lite8.63%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-emb:1:overall
Metric: accuracy
Scale: 0-100
Participants: 54
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 367.615. Cost per task in USD: 0.469926. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-09-29-90535b2fd4dc. Source payload hash: 90535b2fd4dc213540eac0aff8ae769e34b6b1ba54e1ed6d0a51dcfe307d5598. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Finance Agent (v2)

Overall

1.Gemini 3.8 Flash61.44%

2.Gemini 3.7 Flash59.04%

3.Claude Fable 5.158.88%

4.Claude Opus 558.63%

5.Claude Opus 5.558.59%

6.Claude Sonnet 5.558.1%

7.Gemini 3.5 Flash57.86%

8.GLM-5.3-Flash57.85%

9.Claude Fable 556.31%

10.Gemini 3.6 Flash56.3%

11.GLM-5.355.84%

12.GPT-5.6 Luna55.04%

13.GPT-5.6 Terra54.44%

14.Kimi K354.36%

15.Claude Opus 4.853.92%

16.Claude Sonnet 553.91%

17.GPT-5.6 Sol53.76%

18.Grok 4.653.68%

19.GPT-6 Astra53.54%

20.DeepSeek-V4.1-Flash53.48%

21.Grok 4.752.25%

22.GPT-5.551.76%

23.Claude Opus 4.751.51%

24.Claude Sonnet 4.651.04%

25.Qwen 3.8 Max 090250.59%

26.GPT-6 Luna49.87%

27.GLM-5.249.7%

28.GPT-6 Sol49.05%

29.Qwen 3.8 27B48.55%

30.Grok 4.548.35%

31.MiniMax M348.27%

32.Qwen 3.7 Max47.78%

33.Gemini 3.5 Flash-Lite47.44%

34.Inkling46.6%

35.GPT-5.4 mini45.36%

36.Kimi K2.644.9%

37.GLM-5.144.79%

38.DeepSeek-V4-Pro44.08%

39.Gemini 3.1 Pro Preview42.98%

40.Gemini 3 Flash Preview42.55%

41.MiMo-V2.5-Pro41.5%

42.Inkling Small41.26%

43.Qwen 3.7 Plus38.22%

44.GPT-5.4 nano38.22%

45.Grok 4.337.73%

46.Nemotron 3 Ultra37.67%

47.MiMo-V2.536.73%

48.Kimi K2.535.79%

49.Mistral Medium 3.532.1%

50.Ling 3.0 Flash30.31%

51.Gemini 3.1 Flash-Lite29.99%

52.Grok 4.20 Reasoning28.49%

53.MiniMax M2.727.89%

54.Mercury 2.518.56%

55.Nemotron 3.5 Lightning18.5%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-finance-agent-v2:2:overall
Metric: accuracy
Scale: 0-100
Participants: 55
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 351.413. Cost per task in USD: 0.791809. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-09-29-addcd124c670. Source payload hash: addcd124c6706893659c1ae26698bf6ffa29c91018ee95b98ab15b9d4b6edcdb. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

GPQA Diamond

Overall

1.Gemini 3.1 Pro Preview95.45%

2.GPT-5.6 Sol95.2%

3.Grok 4.694.7%

4.Gemini 3.8 Flash94.44%

5.Gemini 3.7 Flash93.94%

6.Qwen 3.8 Max 090293.69%

7.Claude Fable 5.193.43%

7.Claude Opus 593.43%

7.Gemini 3.6 Flash93.43%

10.Claude Fable 593.18%

10.GPT-5.593.18%

12.Grok 4.592.93%

12.Kimi K392.93%

14.Gemini 3.5 Flash92.68%

14.MiniMax M392.68%

16.Claude Opus 4.892.42%

17.GPT-5.491.67%

17.GPT-5.6 Luna91.67%

19.Grok 4.391.41%

20.GPT-5.6 Terra90.91%

21.Claude Opus 4.790.15%

21.Qwen 3.7 Max90.15%

23.DeepSeek-V4-Pro89.39%

24.Kimi K2.689.14%

25.Claude Sonnet 588.89%

25.Qwen 3.8 27B88.89%

27.Grok 4.20 Reasoning88.64%

28.GLM-5.388.13%

29.Gemini 3 Flash Preview87.88%

30.Inkling87.12%

31.MiniMax M2.786.62%

32.GLM-5.3-Flash86.36%

33.Nemotron 3 Ultra86.11%

34.Claude Sonnet 4.685.61%

34.GLM-5.285.61%

34.GPT-585.61%

37.Ling 3.0 Flash84.85%

38.GLM-5.184.52%

39.Kimi K2.584.09%

39.o384.09%

41.Gemini 3.5 Flash-Lite83.84%

42.Inkling Small83.59%

43.GLM-583.33%

44.GPT-5.4 mini83.08%

45.MiMo-V2.5-Pro82.58%

46.MiniMax M2.582.07%

47.MiMo-V2.581.57%

48.Gemini 3.1 Flash-Lite81.06%

49.GPT-5 mini80.3%

50.GLM-4.780.05%

51.Claude Opus 4.579.55%

52.GPT OSS 120B78.54%

52.Kimi K2 Thinking78.54%

52.MiniMax M2.178.54%

55.GPT-5.4 nano77.53%

56.DeepSeek-V3.276.26%

57.o3-mini75.51%

58.GLM-4.674.5%

58.o4-mini74.5%

60.o173.23%

61.GLM-4.572.22%

62.Kimi K2 071171.46%

63.GPT OSS 20B68.94%

64.Mistral Large 368.43%

65.GPT-4.1 mini67.93%

66.GPT-4.165.4%

67.GPT-5 nano63.38%

68.DeepSeek-V3 032461.62%

69.DeepSeek-V354.55%

70.GPT-4.1 nano50.76%

71.Command A48.49%

72.Llama 4 Scout46.97%

73.GPT-4o mini44.19%

73.Mistral Small 3.144.19%

75.Mistral Medium 3.534.85%

76.Command R+31.06%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-gpqa:1:overall
Metric: accuracy
Scale: 0-100
Participants: 76
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 59.421. Cost per task in USD: 0.034461. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-09-29-8caf7b4807e8. Source payload hash: 8caf7b4807e87a2880c30218acea7855ea3a11721d56f7e049a77fadaf35dd34. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Legal

Harvey's Legal Agent Benchmark

Overall · Task fully resolved

1.Grok 4.615.83%

2.Grok 4.512.92%

3.Grok 4.712.5%

4.Claude Fable 511.25%

4.Qwen 3.8 27B11.25%

6.Kimi K310.83%

7.Qwen 3.8 Max 090210.42%

8.Gemini 3.8 Flash10%

9.Claude Opus 4.89.58%

10.Gemini 3.7 Flash8.75%

11.GLM-5.38.33%

12.GLM-5.27.08%

13.Claude Fable 5.16.67%

13.Claude Opus 4.76.67%

13.Claude Opus 56.67%

13.DeepSeek-V4.1-Flash6.67%

13.GLM-5.3-Flash6.67%

18.GPT-6 Astra5.42%

19.Claude Sonnet 4.65%

19.Claude Sonnet 55%

21.MiniMax M34.17%

22.Claude Opus 5.53.75%

22.DeepSeek-V4-Pro3.75%

22.GPT-5.53.75%

25.Gemini 3.6 Flash3.33%

26.Claude Sonnet 5.52.92%

26.GPT-6 Luna2.92%

28.Gemini 3.5 Flash2.5%

28.GPT-5.6 Sol2.5%

30.Inkling2.08%

30.MiMo-V2.5-Pro2.08%

32.GPT-6 Sol1.67%

32.Inkling Small1.67%

32.Kimi K2.61.67%

32.MiMo-V2.51.67%

32.Qwen 3.7 Max1.67%

37.GPT-5.6 Luna1.25%

37.Ling 3.0 Flash1.25%

39.GPT-5.6 Terra0.83%

40.Grok 4.30.42%

40.Mistral Medium 3.50.42%

40.Nemotron 3 Ultra0.42%

43.Gemini 3 Flash Preview0%

43.Gemini 3.1 Flash-Lite0%

43.Gemini 3.1 Pro Preview0%

43.Gemini 3.5 Flash-Lite0%

43.GLM-5.10%

43.GPT-5.40%

43.GPT-5.4 mini0%

43.GPT-5.4 nano0%

43.Grok 4.20 Reasoning0%

43.Kimi K2.50%

43.Mercury 2.50%

43.MiniMax M2.70%

43.Nemotron 3.5 Lightning0%

43.Qwen 3.7 Plus0%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-legal-agent-benchmark:1:overall
Metric: task resolution rate
Scale: 0-100
Participants: 56
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 502.909. Cost per task in USD: 0.986814. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-09-29-d6fb708e5bcc. Source payload hash: d6fb708e5bccec56c7a5e5ad97918d9cc47c5ebefb88af4a1a5771493dadba2f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

LegalBench

Overall

1.Claude Fable 588.56%

2.Claude Fable 5.188.51%

3.Gemini 3.1 Pro Preview87.4%

4.Gemini 3.7 Flash87.26%

5.Gemini 3.8 Flash86.99%

6.Claude Opus 586.97%

7.GPT-5.6 Sol86.97%

8.Gemini 3 Flash Preview86.86%

9.Gemini 3.6 Flash86.71%

10.GPT-5.586.52%

11.Grok 4.686.31%

12.GPT-5.486.04%

13.GPT-586.02%

14.Kimi K386.02%

15.Grok 4.585.97%

16.MiniMax M385.42%

17.Claude Opus 4.785.25%

18.GPT-5.6 Terra85.11%

19.Qwen 3.7 Max84.91%

20.GLM-5.384.84%

21.Kimi K2.684.74%

22.Grok 4.384.46%

23.GLM-5.184.39%

24.Grok 4.784.39%

25.Gemini 3.5 Flash-Lite84.07%

25.GLM-5.284.07%

27.GLM-584.06%

28.GPT-5.6 Luna84.03%

29.MiniMax M2.783.98%

30.GLM-5.3-Flash83.93%

31.Claude Sonnet 583.92%

32.Gemini 3.1 Flash-Lite83.76%

33.o383.76%

34.Qwen 3.8 Max 090283.61%

35.Gemini 3.5 Flash83.6%

36.Claude Opus 4.883.57%

37.GLM-4.783.36%

38.DeepSeek-V4.1-Flash83.28%

39.GPT-4.183.1%

40.Mercury 2.583.06%

41.Inkling Small82.99%

42.Inkling82.95%

43.Claude Opus 4.582.84%

44.Qwen 3.8 27B82.43%

45.Claude Sonnet 4.682.12%

46.Nemotron 3 Ultra82.07%

47.GPT-5 mini81.77%

48.Kimi K2 071181.45%

49.DeepSeek-V380.76%

50.o180.39%

51.DeepSeek-V4-Pro80.32%

52.Kimi K2 Thinking80.2%

53.MiniMax M2.579.96%

54.Command A79.7%

55.Ling 3.0 Flash79.69%

56.GLM-4.679.61%

57.o4-mini79.19%

58.Mistral Large 379.14%

59.MiMo-V2.578.89%

60.GPT-4.1 mini78.04%

61.GPT-5.4 nano77.92%

62.Grok 4.20 Reasoning77.74%

63.DeepSeek-V3 032477.73%

64.MiMo-V2.5-Pro77.13%

65.GPT OSS 120B75.94%

66.GLM-4.575.63%

67.MiniMax M2.175.45%

68.Llama 4 Scout72.04%

69.o3-mini71.54%

70.GPT OSS 20B70.85%

71.Mistral Small 3.169.16%

72.Command R+68.29%

73.Gemma 2 27B67.98%

74.DeepSeek-R167.32%

75.DeepSeek-V3.265.96%

76.GPT-4.1 nano61.06%

77.GPT-5 nano50.13%

78.Command R32.97%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-legal-bench:1:overall
Metric: accuracy
Scale: 0-100
Participants: 78
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 7.574. Cost per task in USD: 0.006065. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-09-29-c1213019b552. Source payload hash: c1213019b55234a262ae86fd4ebfe291530f315c988a4ee32fd501c13777f85f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Legal Research Bench

Overall · All-pass

1.Claude Fable 5.155.29%

1.Claude Opus 555.29%

3.Claude Opus 5.550.48%

4.Claude Fable 549.52%

5.GLM-5.349.04%

6.Claude Sonnet 5.548.08%

6.GPT-5.6 Sol48.08%

6.Grok 4.648.08%

9.Qwen 3.8 Max 090247.6%

10.Grok 4.747.12%

11.GLM-5.3-Flash45.19%

12.Kimi K344.23%

13.Claude Opus 4.843.75%

14.Claude Sonnet 541.83%

15.DeepSeek-V4.1-Flash41.35%

15.GPT-5.6 Terra41.35%

17.GPT-5.540.39%

18.GPT-6 Astra39.42%

19.Gemini 3.8 Flash38.94%

20.Claude Opus 4.738.46%

20.Claude Sonnet 4.638.46%

22.Grok 4.537.98%

23.GPT-5.6 Luna36.54%

24.Qwen 3.8 27B36.06%

25.Gemini 3.7 Flash34.62%

26.GLM-5.231.25%

27.Gemini 3.5 Flash30.77%

28.GPT-6 Luna30.29%

29.MiniMax M329.81%

30.GPT-6 Sol28.85%

31.Inkling28.37%

32.GLM-5.127.89%

33.Inkling Small25.48%

33.Qwen 3.7 Max25.48%

35.Gemini 3.6 Flash25%

36.DeepSeek-V4-Pro23.08%

37.Gemini 3.1 Pro Preview20.67%

38.Gemini 3 Flash Preview18.27%

39.Qwen 3.7 Plus16.35%

40.Kimi K2.515.87%

40.Kimi K2.615.87%

40.MiMo-V2.5-Pro15.87%

43.Grok 4.315.39%

43.Nemotron 3 Ultra15.39%

45.Gemini 3.5 Flash-Lite13.94%

45.Grok 4.20 Reasoning13.94%

47.GPT-5.4 mini12.5%

48.MiniMax M2.710.58%

49.MiMo-V2.59.14%

49.Mistral Medium 3.59.14%

51.GPT-5.4 nano6.25%

52.Mercury 2.54.33%

53.Gemini 3.1 Flash-Lite3.37%

54.Nemotron 3.5 Lightning2.89%

55.Ling 3.0 Flash0%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-legal-research:1:overall
Metric: all-pass rate
Scale: 0-100
Participants: 55
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 244.487. Cost per task in USD: 0.261499. Source ID: vals-ai. Evidence capture ID: vals-legal_research-2026-09-29-b15acd492114. Source payload hash: b15acd492114174e151eff7ff17a38d547d5a1e93d2d598c153066542bbc56b7. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Health

MedCode

Overall

1.Claude Opus 563.57%

2.Gemini 3.1 Pro Preview59.06%

3.Claude Fable 556.07%

4.Gemini 3 Flash Preview55.92%

5.Gemini 3.5 Flash55.83%

6.Claude Opus 4.754.86%

7.Claude Fable 5.153.51%

8.Gemini 3.7 Flash53.39%

9.Claude Opus 4.853.22%

10.Gemini 3.6 Flash53.15%

11.Claude Sonnet 5.552.92%

12.Gemini 2.5 Pro50.59%

13.Claude Opus 5.549.8%

14.GPT-549.63%

15.Grok 4.749.55%

16.GPT-5.549.1%

17.Kimi K348.88%

18.GPT-6 Astra48.49%

19.Claude Opus 4.648.24%

20.Gemini 3.8 Flash48.14%

21.Gemini 3.1 Flash-Lite47.6%

22.Claude Sonnet 547.54%

23.o347.29%

24.GPT-6 Sol47.07%

25.MiniMax M346.29%

26.Claude Opus 4.545.17%

27.Grok 4.644.71%

28.GPT-6 Luna44.69%

29.GPT-5.6 Sol43.97%

30.Gemini 3.5 Flash-Lite43.49%

31.GPT-5.6 Terra43.41%

32.Grok 4.543.29%

33.GPT-5 mini43.05%

34.GLM-5.342.86%

35.GPT-5.6 Luna42.39%

36.GLM-5.141.6%

37.GPT-5.441.29%

38.Inkling41.19%

39.DeepSeek-V4.1-Flash41.17%

40.GPT-5.4 nano41.03%

41.GLM-5.240.77%

42.Qwen 3.8 Max 090240.67%

43.Claude Sonnet 4.540.57%

44.DeepSeek-V4-Pro40.46%

45.Kimi K2.640.14%

46.Kimi K2.539.32%

47.Qwen 3.7 Max38.75%

48.Nemotron 3 Ultra38.62%

49.Grok 4.338.07%

50.Inkling Small37.89%

51.MiniMax M2.734.44%

52.MiniMax M2.134.08%

53.o4-mini33.79%

54.Mistral Medium 3.533.75%

55.GLM-4.732.77%

56.MiMo-V2.5-Pro32.48%

57.Ling 3.0 Flash32.28%

58.Grok 4.20 Reasoning32.16%

59.MiMo-V2.531.9%

60.Mercury 2.531.33%

61.GPT-5 nano30.44%

62.Qwen 3.8 27B28.7%

63.Gemini 2.5 Flash-Lite27.12%

64.Llama 4 Scout23.31%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-medcode:1:overall
Metric: accuracy
Scale: 0-100
Participants: 64
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-26. Latency in seconds: 16.546. Cost per task in USD: 0.036189. Source ID: vals-ai. Evidence capture ID: vals-medcode-2026-09-29-7062229b5ab4. Source payload hash: 7062229b5ab4a5e16f014a78fd550f9f34e7147be6be796b610ab1ca3de055ab. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

MedQA

Overall

1.o196.52%

2.Gemini 3.1 Pro Preview96.37%

3.GPT-596.32%

4.GPT-5.496.09%

5.GPT-5 mini96.06%

5.o396.06%

7.o4-mini96.02%

8.Gemini 3 Flash Preview95.81%

9.o3-mini94.83%

10.Grok 4.20 Reasoning94.55%

11.Kimi K2.594.37%

12.GLM-594.27%

13.GLM-4.793.74%

14.GPT-5 nano93.26%

15.Claude Opus 4.593.16%

16.Kimi K2 Thinking92.59%

17.MiniMax M2.592.53%

18.GLM-4.692.23%

19.Claude Sonnet 4.692.06%

20.GPT OSS 120B91.36%

21.GPT-4.191.18%

22.MiniMax M2.191.16%

23.DeepSeek-R190.8%

24.GLM-4.589.98%

25.DeepSeek-V3.289.45%

26.GPT-4o88.16%

27.GPT-4.1 mini84.63%

28.Kimi K2 071183.98%

29.GPT OSS 20B82.88%

30.Mistral Large 382.23%

31.DeepSeek-V3 032482%

32.DeepSeek-V380.9%

33.Command A80.55%

34.GPT-4o mini72.44%

35.Mistral Small 3.169.1%

36.GPT-4.1 nano68.23%

37.Mixtral 8x22B62.14%

38.Llama 4 Scout50.9%

39.Command R+2.65%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-medqa:1:overall
Metric: accuracy
Scale: 0-100
Participants: 39
Methodology and context: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Latency in seconds: 12.225. Cost per task in USD: 0.009106. Source ID: vals-ai. Evidence capture ID: vals-medqa-2026-09-29-2886a92806d7. Source payload hash: 2886a92806d7b2e892c96203b7c7b462a4e28ec54dae900401734c38aa443883. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Recorded measurement details

Taste-Bench: Overall (Show measurement, test conditions, and source)

Source value: 15.7
Score: 15.7
Metric: both-orders-correct, engineering/research macro average
Unit: %
Benchmark version: arxiv:2609.25804v2
Category: agentic
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: taste-bench-paper-v2-502
Source type: independent-evaluator
Evaluator: Taste-Bench paper
Status: active
Retrieved at: 2026-09-26
Methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/grok-4.20-reasoning. Source model name: Grok 4.20 Reasoning. Provider reported by source: xAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-b780156e6976. Source payload hash: b780156e6976a567e249d97bf51e2088015b07b0c2aa65663f57ed1fba7c45b0. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.
Context: 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

Source Taste-Bench paper

Code Migration: Overall (Show measurement, test conditions, and source)

Source value: 0.314
Score: 0.314
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-code-migration:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 373.673. Cost per task in USD: 0.289486. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-09-29-f0b317d5bed3. Source payload hash: f0b317d5bed3a2654a9ea39762d5e57f4649306adce3e6103e845ee1b9b94add. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

CorpFin v2: Overall (Show measurement, test conditions, and source)

Source value: 63.675
Score: 63.675
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Latency in seconds: 9.67. Cost per task in USD: 0.163386. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-09-29-83186cfafe52. Source payload hash: 83186cfafe526f0b8fe2ae86af05e8209fe799c8bfa48292ef1fef5db66650ee. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

EMB: Overall (Show measurement, test conditions, and source)

Source value: 11.739
Score: 11.739
Metric: accuracy
Unit: %
Benchmark version: 1
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-emb:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 367.615. Cost per task in USD: 0.469926. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-09-29-90535b2fd4dc. Source payload hash: 90535b2fd4dc213540eac0aff8ae769e34b6b1ba54e1ed6d0a51dcfe307d5598. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Finance Agent (v2): Overall (Show measurement, test conditions, and source)

Source value: 28.492
Score: 28.492
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-finance-agent-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 351.413. Cost per task in USD: 0.791809. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-09-29-addcd124c670. Source payload hash: addcd124c6706893659c1ae26698bf6ffa29c91018ee95b98ab15b9d4b6edcdb. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

GPQA Diamond: Overall (Show measurement, test conditions, and source)

Source value: 88.636
Score: 88.636
Metric: accuracy
Unit: %
Benchmark version: 1
Category: reasoning
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-gpqa:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 59.421. Cost per task in USD: 0.034461. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-09-29-8caf7b4807e8. Source payload hash: 8caf7b4807e87a2880c30218acea7855ea3a11721d56f7e049a77fadaf35dd34. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

LiveCodeBench v6: Overall (Show measurement, test conditions, and source)

Source value: 84.265
Score: 84.265
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-lcb:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 76.09. Cost per task in USD: 0.05249. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-09-29-b6ee1006a41a. Source payload hash: b6ee1006a41aab0c68eef9d3e42a2151634fffaaeb6bdc2230a6d02994198a59. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Harvey's Legal Agent Benchmark: Overall · Task fully resolved (Show measurement, test conditions, and source)

Source value: 0
Score: 0
Metric: task resolution rate
Unit: %
Benchmark version: 1
Category: legal
Task: overall
Task label: Overall · Task fully resolved
Direction: Higher is better
Comparison cohort: vals-legal-agent-benchmark:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 502.909. Cost per task in USD: 0.986814. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-09-29-d6fb708e5bcc. Source payload hash: d6fb708e5bccec56c7a5e5ad97918d9cc47c5ebefb88af4a1a5771493dadba2f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

LegalBench: Overall (Show measurement, test conditions, and source)

Source value: 77.738
Score: 77.738
Metric: accuracy
Unit: %
Benchmark version: 1
Category: legal
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-legal-bench:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 7.574. Cost per task in USD: 0.006065. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-09-29-c1213019b552. Source payload hash: c1213019b55234a262ae86fd4ebfe291530f315c988a4ee32fd501c13777f85f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Legal Research Bench: Overall · All-pass (Show measurement, test conditions, and source)

Source value: 13.942
Score: 13.942
Metric: all-pass rate
Unit: %
Benchmark version: 1
Category: legal
Task: overall
Task label: Overall · All-pass
Direction: Higher is better
Comparison cohort: vals-legal-research:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 244.487. Cost per task in USD: 0.261499. Source ID: vals-ai. Evidence capture ID: vals-legal_research-2026-09-29-b15acd492114. Source payload hash: b15acd492114174e151eff7ff17a38d547d5a1e93d2d598c153066542bbc56b7. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

MedCode: Overall (Show measurement, test conditions, and source)

Source value: 32.156
Score: 32.156
Metric: accuracy
Unit: %
Benchmark version: 1
Category: health
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-medcode:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-26. Latency in seconds: 16.546. Cost per task in USD: 0.036189. Source ID: vals-ai. Evidence capture ID: vals-medcode-2026-09-29-7062229b5ab4. Source payload hash: 7062229b5ab4a5e16f014a78fd550f9f34e7147be6be796b610ab1ca3de055ab. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

MedQA: Overall (Show measurement, test conditions, and source)

Source value: 94.55
Score: 94.55
Metric: accuracy
Unit: %
Benchmark version: 1
Category: health
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-medqa:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: grok/grok-4.20-0309-reasoning. Source model name: Grok 4.20 Reasoning. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Latency in seconds: 12.225. Cost per task in USD: 0.009106. Source ID: vals-ai. Evidence capture ID: vals-medqa-2026-09-29-2886a92806d7. Source payload hash: 2886a92806d7b2e892c96203b7c7b462a4e28ec54dae900401734c38aa443883. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Page 1 of 3

Our tests

SVG short test

One SVG prompt and fixed output settings. Captures are shown only with 0 reported reasoning tokens or documented lack of reasoning controls. Identical test conditions do not make the models' internal compute identical.

Grok 4.20 Reasoning

Earlier tests

Pelican on a bicycle

A community classic for free-form SVG drawing.

Grok 4.20 Reasoning

Reasoning (requested): high

Current attempt

Run date: Oct 1, 2026
API provider: openrouter
Requested API model: x-ai/grok-4.20
API model reported in response: x-ai/grok-4.20
Reasoning setting: high
Pinned endpoint: xAI | x-ai/grok-4.20-20260309
Temperature: Provider default
Token limit including reasoning: 8,192 tokens
Test protocol: community-visual-tests-v1
Technical checks: Passed, not a quality rating
Response time: 68.5 s

Visually reviewed. This individual test does not establish overall model quality.

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

Popularity

23.5 / 100

Specifications and access

Specification	Value and source
Weights	ProprietarySource
Access	APISource
Context window	1,000,000 tokensSource
Notes	1M context, reasoning mode and agentic tool calling, parameter count not disclosedSource
API model ID	grok-4.20-0309-reasoningSource
Source status	activeSource
Release	2026-03-10Source
Maximum output	not statedSource
Input	Text, ImageSource
Output	TextSource
Tool calling	YesSource
Structured output	YesSource
Model aliases	grok-4.20-reasoningSource
Reasoning mode	low, medium, highSource

Published prices

API input: $1.25 per 1M tokensSource
API input: $2.5 per 1M tokens (above 200,000 context tokens)Source
API output: $2.5 per 1M tokensSource
API output: $5 per 1M tokens (above 200,000 context tokens)Source
Cache read: $0.2 per 1M tokensSource
Cache read: $0.4 per 1M tokens (above 200,000 context tokens)Source

Sources and data date

Type	Evidence and data date
Speed	SpaceXAI. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · xAI developer docs, Grok 4.20 0309 Reasoning · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 3 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording xAI does not publish parameter counts on the model page.
Not found in the inspected sources	active-parameters-billions Original wording xAI does not publish active parameter counts on the model page.
Not found in the inspected sources	knowledge-cutoff Original wording The official xAI model page did not state a knowledge cutoff.
Not found in the inspected sources	profile-benchmarks Original wording The exact model documentation page exposed capabilities and pricing but no benchmark table.
Additional source	Taste-Bench paper (retrieved September 26, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Creative Commons Attribution 4.0