Language modelDeprecated

GPT-5.4 nano

OpenAI

Released: March 17, 2026
Data date: September 8, 2026
Predecessor: GPT-5 nano

Intelligence: 24 / 100
Overall score
Speed: 176.1 tokens/s
Output throughput
Price: $0.2 · $1.25
Input · Output / 1M tokens
Input: Text, Image
Output: Text

GPT-5.4 nano is the GPT-5.4 family’s smallest API tier. OpenAI released it alongside GPT-5.4 mini as an update to GPT-5 nano. Nano targets classification, data extraction, and ranking, while mini also handles computer use.

GPT-5.4 nano was deprecated on October 1, 2026, with shutdown announced for April 1, 2027.

Position: Rank 71 of 86
Index score: 24 / 100

1.Claude Fable 5.193.5 / 100

2.Claude Fable 593.2 / 100

3.Claude Opus 591.3 / 100

4.Claude Sonnet 5.587.9 / 100

5.Claude Opus 5.585.1 / 100

6.Kimi K385.1 / 100

7.Gemini 3.7 Flash83.1 / 100

8.GPT-5.6 Sol81.9 / 100

9.Claude Opus 4.878.8 / 100

10.Gemini 3.8 Flash77.4 / 100

11.Qwen 3.8 Max 090276.5 / 100

12.GPT-5.575.8 / 100

13.Claude Opus 4.772.6 / 100

14.Gemini 3.6 Flash70.4 / 100

15.Grok 4.669.7 / 100

16.Qwen 3.7 Max68.7 / 100

17.GPT-5.268.3 / 100

18.Gemini 3.1 Pro Preview67.5 / 100

19.GPT-5.6 Terra67.5 / 100

20.Gemini 3.5 Flash67.4 / 100

21.Gemini 3 Pro66.3 / 100

22.GLM-5.365.3 / 100

23.GPT-5.6 Luna64.4 / 100

24.Grok 4.763.7 / 100

25.DeepSeek-V4.1-Flash62.8 / 100

26.Gemini 3 Flash Preview62.7 / 100

27.GPT-5.462.7 / 100

28.GLM-5.3-Flash62.4 / 100

29.Claude Sonnet 561.6 / 100

30.Kimi K2.659.2 / 100

31.Grok 4.558.6 / 100

32.MiniMax M358.3 / 100

33.GPT-557.6 / 100

34.Claude Sonnet 4.655.9 / 100

35.Qwen 3.8 27B53.7 / 100

36.GLM-5.252.6 / 100

37.GPT-6 Luna50.7 / 100

38.Nemotron 3 Ultra49.8 / 100

39.GPT-6 Sol49.6 / 100

40.DeepSeek-V4-Pro49.6 / 100

41.Inkling Small48.5 / 100

42.Kimi K2.548.2 / 100

43.Inkling47.8 / 100

44.o346.9 / 100

45.GPT-5 mini46.1 / 100

46.GLM-546 / 100

47.GLM-5.144.2 / 100

48.Claude Opus 4.543.7 / 100

49.GPT-5.4 mini42.3 / 100

50.MiMo-V2.5-Pro42.2 / 100

51.Grok 4.20 Reasoning39.3 / 100

52.Grok 338.7 / 100

53.Gemini 3.1 Flash-Lite38.5 / 100

54.Gemini 3.5 Flash-Lite38.2 / 100

55.Grok 4.336.8 / 100

56.MiniMax M2.135.5 / 100

57.MiniMax M2.735 / 100

58.MiMo-V2.533.6 / 100

59.GLM-4.733.4 / 100

60.o4-mini33.3 / 100

61.GPT-4.132.6 / 100

62.Ling 3.0 Flash32.2 / 100

63.o131.4 / 100

64.GPT OSS 120B30.6 / 100

65.Kimi K2 Thinking28.8 / 100

66.MiniMax M2.528.2 / 100

67.DeepSeek-R127.3 / 100

68.GLM-4.625.8 / 100

69.GLM-4.525.6 / 100

70.Kimi K2 071124.4 / 100

71.GPT-5.4 nano24 / 100

72.DeepSeek-V3.222.4 / 100

73.Mistral Large 319.8 / 100

74.GPT-4.1 mini19.7 / 100

75.o3-mini19.7 / 100

76.DeepSeek-V3 032418.8 / 100

77.DeepSeek-V317.8 / 100

78.GPT OSS 20B15.1 / 100

79.GPT-5 nano12.4 / 100

80.Command A10.5 / 100

81.Mistral Medium 3.59.5 / 100

82.Llama 4 Scout6.1 / 100

83.Mistral Small 3.14.7 / 100

84.GPT-4.1 nano4 / 100

85.Command R+2.5 / 100

86.GPT-4o mini2.3 / 100

Methodology and source

The index averages rank percentiles from 10 documented comparison cohorts.

Source: Vals AI, LegalBench

Source: Vals AI, GPQA Diamond

Source: Vals AI, LiveCodeBench v6

Source: Vals AI, MMLU-Pro

Source: Vals AI, TaxEval v2

Source: Vals AI, MedScribe

Source: Vals AI, SAGE

Source: Vals AI, Terminal-Bench 2.1

Source: Vals AI, MMMU-Pro

Source: Vals AI, Public Benefits Bench v1.1

Published benchmarks

Taste-Bench

Overall

502 technical decision points from engineering and research workflows. Each choice is presented in both answer orders. The overall score averages engineering and research and requires both answer orders to be correct. Unparseable responses count as wrong. Reasoning settings and output budget are part of the setup.

Note (GPT-5.4 nano): 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

1.GPT-5.6 Sol59.7%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

2.GPT-5.559.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

3.Claude Opus 555.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

4.Grok 4.554.6%

Note: 10 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

5.GPT-5.6 Terra54%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

6.GLM-5.253.9%

Note: 17 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

7.Claude Sonnet 551.6%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

8.GPT-5.6 Luna49%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

9.MiniMax M345.3%

Note: 3 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

10.DeepSeek-V4-Flash43.3%

Note: 2 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

11.GPT-5.4 mini40.1%

Note: 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

12.Mistral Medium 3.537.7%

Note: 24 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

13.GPT-5.4 nano36.6%

14.Grok 4.20 Reasoning15.7%

Note: 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

higher is betterAs of September 26, 2026Scale: 0% - 100%

Source: Taste-Bench paper

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: taste-bench-paper-v2-502
Metric: both-orders-correct, engineering/research macro average
Scale: 0-100
Participants: 14
Methodology and context: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/gpt-5.4-nano. Source model name: GPT-5.4 nano. Provider reported by source: OpenAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: high. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-ee1a62d2083b. Source payload hash: ee1a62d2083b8b1184c46ee9fe59fc02bd5b2f0a63b87dbef0c3f163690c0542. Parser version: curated-v1. Imported at: 2026-09-29

Source: Taste-Bench paper

Coding

Code Migration

Overall

1.Claude Sonnet 5.569.83%

2.GPT-6 Astra67.74%

3.Claude Opus 5.566.65%

4.Claude Opus 557.47%

5.GPT-6 Sol57.2%

6.Claude Fable 555.07%

7.Claude Fable 5.154.61%

8.GPT-5.6 Sol52.92%

9.GPT-5.6 Terra47.8%

10.Claude Opus 4.847.25%

11.DeepSeek-V4.1-Flash45.62%

12.GPT-5.545.16%

13.Grok 4.744.82%

14.Grok 4.644.57%

15.GPT-5.6 Luna44.55%

16.Claude Sonnet 544.39%

17.GLM-5.344.22%

18.Claude Opus 4.743.88%

19.GPT-6 Luna42.55%

20.Claude Sonnet 4.639.89%

21.GLM-5.237.87%

22.Grok 4.536.6%

23.Gemini 3.8 Flash36.55%

24.GPT-5.434.98%

25.Gemini 3.7 Flash34.8%

26.Gemini 3.6 Flash30.93%

27.Kimi K2.627.77%

28.Gemini 3.5 Flash26.75%

29.DeepSeek-V4-Pro26.2%

30.GLM-5.125.77%

31.Kimi K2.7 Code25.39%

32.Qwen 3.8 Max 090223.96%

33.MiMo-V2.5-Pro21.56%

34.GLM-5.3-Flash20.52%

35.MiniMax M319.93%

36.Gemini 3.1 Pro Preview17.31%

37.Kimi K316.1%

38.GPT-5.4 nano14.47%

39.MiMo-V2.514.23%

40.Qwen 3.8 27B14.16%

41.Inkling Small13.68%

42.Qwen 3.7 Max13.07%

43.GPT-5.4 mini12.94%

44.Qwen 3.7 Plus12.86%

45.Inkling11.79%

46.MiniMax M2.78.69%

47.Kimi K2.56.96%

48.Grok 4.36.8%

49.Gemini 3 Flash Preview6.39%

50.Gemini 3.5 Flash-Lite6.15%

51.Mistral Medium 3.55.13%

52.Nemotron 3 Ultra4.91%

53.Gemini 3.1 Flash-Lite4.61%

54.Mercury 2.54.49%

55.Nemotron 3.5 Lightning2.95%

56.Grok 4.20 Reasoning0.31%

57.Ling 3.0 Flash0%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-code-migration:1:overall
Metric: accuracy
Scale: 0-100
Participants: 57
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 1594.162. Cost per task in USD: 0.416225. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-09-29-f0b317d5bed3. Source payload hash: f0b317d5bed3a2654a9ea39762d5e57f4649306adce3e6103e845ee1b9b94add. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

LiveCodeBench v6

Overall

1.Claude Fable 5.190.52%

2.Claude Fable 589.78%

3.Gemini 3.8 Flash89.48%

4.Claude Opus 589.03%

5.Gemini 3.7 Flash88.65%

6.Gemini 3.1 Pro Preview88.49%

7.Grok 4.688.22%

8.Gemini 3.6 Flash88.08%

9.Qwen 3.8 Max 090287.85%

10.Claude Opus 4.887.82%

11.Gemini 3.5 Flash87.6%

12.DeepSeek-V4-Pro87.48%

13.Grok 4.587.35%

14.GPT-5.3-Codex87.31%

15.Kimi K387.19%

16.Qwen 3.7 Max87.06%

17.Kimi K2.686.77%

18.GPT-5 mini86.61%

19.Nemotron 3 Ultra85.98%

20.Inkling Small85.93%

21.GPT-5.6 Terra85.93%

22.GPT-585.91%

23.Gemini 3 Flash Preview85.59%

24.Inkling85.52%

25.GPT-5.585.3%

26.Claude Opus 4.785.07%

27.Grok 4.384.49%

28.Grok 4.20 Reasoning84.27%

29.GPT-5.484.14%

30.GPT-5.4 nano84.01%

31.Qwen 3.8 27B84%

32.Ling 3.0 Flash83.99%

33.o383.91%

34.Kimi K2.583.87%

35.GPT OSS 120B83.23%

36.GPT-5.6 Sol82.6%

37.Claude Sonnet 582.43%

38.GLM-4.782.23%

39.o4-mini82.21%

40.MiniMax M382.15%

41.Claude Sonnet 4.682.09%

42.Kimi K2.7 Code82.05%

43.GLM-581.87%

44.MiniMax M2.181.76%

45.MiMo-V2.581.51%

46.GPT-5.4 mini81.47%

47.GLM-5.181.38%

48.MiMo-V2.5-Pro81.35%

49.GLM-4.681.04%

50.GLM-5.380.54%

51.GLM-5.3-Flash80.51%

52.GPT OSS 20B80.39%

53.Gemini 3.1 Flash-Lite80.12%

54.MiniMax M2.779.93%

55.MiniMax M2.579.21%

56.Gemini 3.5 Flash-Lite79.01%

57.Claude Opus 4.575.03%

58.o3-mini71.48%

59.Kimi K2 071170.45%

60.DeepSeek-R170.22%

61.GPT-5 nano70.22%

62.DeepSeek-V3.269.86%

63.GLM-5.269.5%

64.GLM-4.567.45%

65.DeepSeek-V3 032465.48%

66.Kimi K2 Thinking63.15%

67.GPT-4.1 mini58.16%

68.Mistral Large 355.34%

69.GPT-4.154.67%

70.o150.26%

71.GPT-4o43.44%

72.GPT-4.1 nano42.72%

73.Llama 4 Scout38.54%

74.Command A35.07%

75.Mistral Small 3.131.82%

76.GPT-4o mini26.42%

77.Command R+18.24%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-lcb:1:overall
Metric: accuracy
Scale: 0-100
Participants: 77
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 60.089. Cost per task in USD: 0.002468. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-09-29-b6ee1006a41a. Source payload hash: b6ee1006a41aab0c68eef9d3e42a2151634fffaaeb6bdc2230a6d02994198a59. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Finance

CorpFin v2

Sub-task	GPT-5.4 nano (This model)	GPT-5 mini
Exact Pages	60.37% (best value in this row)	58.86%
Max Fitting Context	62.12% (best value in this row)	60.49%
Shared Max Context	61.07%	61.19% (best value in this row)

higher is betterAs of August 11, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

CorpFin v2

Overall

1.Claude Opus 573.19%

2.Claude Fable 571.83%

3.Kimi K371.56%

4.Inkling Small69.62%

5.Inkling68.57%

6.Grok 4.368.53%

7.GPT-5.568.42%

8.Kimi K2.568.26%

9.MiniMax M368.1%

10.Grok 4.567.41%

11.Claude Sonnet 566.98%

12.Kimi K2.666.74%

13.Claude Opus 4.866.71%

14.Gemini 3 Flash Preview66.43%

15.Grok 4.666.16%

16.GLM-5.266.12%

17.Claude Opus 4.766.08%

18.Qwen 3.8 Max 090265.85%

19.Nemotron 3 Ultra65.46%

20.Claude Sonnet 4.665.31%

20.GPT-5.6 Terra65.31%

22.GPT-5.465.27%

23.Gemini 3.5 Flash64.69%

24.Gemini 3.1 Pro Preview64.49%

25.GLM-5.164.45%

26.GPT-5.6 Sol64.38%

27.GPT-5.6 Luna64.22%

28.Qwen 3.7 Max63.71%

29.Grok 4.20 Reasoning63.68%

30.Gemini 3.6 Flash63.33%

31.GPT-4.163.05%

32.GLM-562.9%

33.Qwen 3.6 27B62.32%

34.Ling 3.0 Flash61.93%

35.MiMo-V2.5-Pro61.42%

36.DeepSeek-V4-Pro61.38%

37.Claude Opus 4.561.31%

38.GPT-5.4 nano61.19%

38.MiniMax M2.761.19%

40.GPT-561.07%

41.Mistral Large 361.03%

42.GLM-4.560.96%

43.GPT-5.4 mini60.92%

44.Claude Sonnet 4.560.8%

44.Gemini 2.5 Pro60.8%

46.Gemini 3.5 Flash-Lite60.68%

47.Kimi K2 Thinking60.57%

48.Claude Haiku 4.560.3%

49.GPT-5 mini60.18%

50.MiMo-V2.559.91%

51.o359.71%

52.MiniMax M2.559.6%

53.Gemini 3.1 Flash-Lite59.36%

54.o4-mini58.97%

55.MiniMax M2.158.9%

56.Mistral Medium 3.558.78%

57.GPT OSS 120B58.24%

58.GPT-4.1 mini57.93%

59.GLM-4.656.84%

60.DeepSeek-V3 032454.74%

61.Trinity Large Thinking54.66%

62.Nemotron 3.5 Lightning54.24%

63.DeepSeek-R154.12%

64.GPT OSS 20B53.15%

65.DeepSeek-V352.49%

66.DeepSeek-V3.151.48%

67.Kimi K2 071150.39%

68.DeepSeek-V3.247.94%

69.Llama 4 Scout46.78%

70.GLM-4.746.39%

71.Command A45.96%

72.GPT-4o mini45.46%

73.o3-mini45.3%

74.Mistral Small 3.144.17%

75.GPT-4.1 nano42.08%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-corp-fin-v2:2:overall
Metric: accuracy
Scale: 0-100
Participants: 75
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Latency in seconds: 9.397. Cost per task in USD: 0.01612. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-09-29-83186cfafe52. Source payload hash: 83186cfafe526f0b8fe2ae86af05e8209fe799c8bfa48292ef1fef5db66650ee. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

EMB

Overall

1.Claude Fable 5.176.67%

2.Claude Opus 5.575.94%

3.Claude Sonnet 5.575.71%

4.Claude Fable 573.67%

5.Claude Opus 573.56%

6.GPT-5.6 Sol72.34%

7.Gemini 3.8 Flash72.2%

8.GPT-6 Astra71.7%

9.GPT-6 Sol71.53%

10.Gemini 3.7 Flash71.33%

11.Claude Opus 4.869.37%

12.GPT-6 Luna68.52%

13.GPT-5.6 Luna67.12%

14.Grok 4.766.99%

15.Kimi K366.4%

16.Claude Sonnet 566.32%

17.GPT-5.6 Terra66.21%

18.Gemini 3.6 Flash65.41%

19.GPT-5.564.54%

20.Claude Opus 4.763.73%

21.Gemini 3.5 Flash63.55%

22.Grok 4.662.73%

23.GLM-5.261.53%

24.Claude Sonnet 4.660.15%

25.Qwen 3.8 Max 090260.07%

26.Qwen 3.8 27B59.66%

27.Kimi K2.657.86%

28.DeepSeek-V4.1-Flash57.21%

29.Qwen 3.7 Max57.04%

30.GLM-5.356.34%

31.GLM-5.3-Flash55.93%

32.MiMo-V2.5-Pro55.23%

33.MiMo-V2.555.09%

34.Grok 4.552.91%

35.Gemini 3.1 Pro Preview52.62%

36.DeepSeek-V4-Pro51.62%

37.Qwen 3.7 Plus49.35%

38.MiniMax M347.76%

39.GPT-5.4 mini45.43%

40.GPT-5.4 nano44.75%

41.Gemini 3.5 Flash-Lite43.23%

42.Inkling38.36%

43.Inkling Small31.76%

44.Nemotron 3 Ultra31.74%

45.Kimi K2.528.47%

46.MiniMax M2.728.45%

47.Gemini 3 Flash Preview26.02%

48.Ling 3.0 Flash25.01%

49.Grok 4.318.39%

50.Nemotron 3.5 Lightning16.99%

51.Grok 4.20 Reasoning11.74%

52.Mistral Medium 3.511.19%

53.Mercury 2.58.82%

54.Gemini 3.1 Flash-Lite8.63%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-emb:1:overall
Metric: accuracy
Scale: 0-100
Participants: 54
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 1767.043. Cost per task in USD: 0.57312. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-09-29-90535b2fd4dc. Source payload hash: 90535b2fd4dc213540eac0aff8ae769e34b6b1ba54e1ed6d0a51dcfe307d5598. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Finance Agent (v2)

Overall

1.Gemini 3.8 Flash61.44%

2.Gemini 3.7 Flash59.04%

3.Claude Fable 5.158.88%

4.Claude Opus 558.63%

5.Claude Opus 5.558.59%

6.Claude Sonnet 5.558.1%

7.Gemini 3.5 Flash57.86%

8.GLM-5.3-Flash57.85%

9.Claude Fable 556.31%

10.Gemini 3.6 Flash56.3%

11.GLM-5.355.84%

12.GPT-5.6 Luna55.04%

13.GPT-5.6 Terra54.44%

14.Kimi K354.36%

15.Claude Opus 4.853.92%

16.Claude Sonnet 553.91%

17.GPT-5.6 Sol53.76%

18.Grok 4.653.68%

19.GPT-6 Astra53.54%

20.DeepSeek-V4.1-Flash53.48%

21.Grok 4.752.25%

22.GPT-5.551.76%

23.Claude Opus 4.751.51%

24.Claude Sonnet 4.651.04%

25.Qwen 3.8 Max 090250.59%

26.GPT-6 Luna49.87%

27.GLM-5.249.7%

28.GPT-6 Sol49.05%

29.Qwen 3.8 27B48.55%

30.Grok 4.548.35%

31.MiniMax M348.27%

32.Qwen 3.7 Max47.78%

33.Gemini 3.5 Flash-Lite47.44%

34.Inkling46.6%

35.GPT-5.4 mini45.36%

36.Kimi K2.644.9%

37.GLM-5.144.79%

38.DeepSeek-V4-Pro44.08%

39.Gemini 3.1 Pro Preview42.98%

40.Gemini 3 Flash Preview42.55%

41.MiMo-V2.5-Pro41.5%

42.Inkling Small41.26%

43.Qwen 3.7 Plus38.22%

44.GPT-5.4 nano38.22%

45.Grok 4.337.73%

46.Nemotron 3 Ultra37.67%

47.MiMo-V2.536.73%

48.Kimi K2.535.79%

49.Mistral Medium 3.532.1%

50.Ling 3.0 Flash30.31%

51.Gemini 3.1 Flash-Lite29.99%

52.Grok 4.20 Reasoning28.49%

53.MiniMax M2.727.89%

54.Mercury 2.518.56%

55.Nemotron 3.5 Lightning18.5%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-finance-agent-v2:2:overall
Metric: accuracy
Scale: 0-100
Participants: 55
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 334.538. Cost per task in USD: 0.15925. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-09-29-addcd124c670. Source payload hash: addcd124c6706893659c1ae26698bf6ffa29c91018ee95b98ab15b9d4b6edcdb. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

GPQA Diamond

Overall

1.Gemini 3.1 Pro Preview95.45%

2.GPT-5.6 Sol95.2%

3.Grok 4.694.7%

4.Gemini 3.8 Flash94.44%

5.Gemini 3.7 Flash93.94%

6.Qwen 3.8 Max 090293.69%

7.Claude Fable 5.193.43%

7.Claude Opus 593.43%

7.Gemini 3.6 Flash93.43%

10.Claude Fable 593.18%

10.GPT-5.593.18%

12.Grok 4.592.93%

12.Kimi K392.93%

14.Gemini 3.5 Flash92.68%

14.MiniMax M392.68%

16.Claude Opus 4.892.42%

17.GPT-5.491.67%

17.GPT-5.6 Luna91.67%

19.Grok 4.391.41%

20.GPT-5.6 Terra90.91%

21.Claude Opus 4.790.15%

21.Qwen 3.7 Max90.15%

23.DeepSeek-V4-Pro89.39%

24.Kimi K2.689.14%

25.Claude Sonnet 588.89%

25.Qwen 3.8 27B88.89%

27.Grok 4.20 Reasoning88.64%

28.GLM-5.388.13%

29.Gemini 3 Flash Preview87.88%

30.Inkling87.12%

31.MiniMax M2.786.62%

32.GLM-5.3-Flash86.36%

33.Nemotron 3 Ultra86.11%

34.Claude Sonnet 4.685.61%

34.GLM-5.285.61%

34.GPT-585.61%

37.Ling 3.0 Flash84.85%

38.GLM-5.184.52%

39.Kimi K2.584.09%

39.o384.09%

41.Gemini 3.5 Flash-Lite83.84%

42.Inkling Small83.59%

43.GLM-583.33%

44.GPT-5.4 mini83.08%

45.MiMo-V2.5-Pro82.58%

46.MiniMax M2.582.07%

47.MiMo-V2.581.57%

48.Gemini 3.1 Flash-Lite81.06%

49.GPT-5 mini80.3%

50.GLM-4.780.05%

51.Claude Opus 4.579.55%

52.GPT OSS 120B78.54%

52.Kimi K2 Thinking78.54%

52.MiniMax M2.178.54%

55.GPT-5.4 nano77.53%

56.DeepSeek-V3.276.26%

57.o3-mini75.51%

58.GLM-4.674.5%

58.o4-mini74.5%

60.o173.23%

61.GLM-4.572.22%

62.Kimi K2 071171.46%

63.GPT OSS 20B68.94%

64.Mistral Large 368.43%

65.GPT-4.1 mini67.93%

66.GPT-4.165.4%

67.GPT-5 nano63.38%

68.DeepSeek-V3 032461.62%

69.DeepSeek-V354.55%

70.GPT-4.1 nano50.76%

71.Command A48.49%

72.Llama 4 Scout46.97%

73.GPT-4o mini44.19%

73.Mistral Small 3.144.19%

75.Mistral Medium 3.534.85%

76.Command R+31.06%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-gpqa:1:overall
Metric: accuracy
Scale: 0-100
Participants: 76
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 29.498. Cost per task in USD: 0.001565. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-09-29-8caf7b4807e8. Source payload hash: 8caf7b4807e87a2880c30218acea7855ea3a11721d56f7e049a77fadaf35dd34. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Legal

Harvey's Legal Agent Benchmark

Overall · Task fully resolved

1.Grok 4.615.83%

2.Grok 4.512.92%

3.Grok 4.712.5%

4.Claude Fable 511.25%

4.Qwen 3.8 27B11.25%

6.Kimi K310.83%

7.Qwen 3.8 Max 090210.42%

8.Gemini 3.8 Flash10%

9.Claude Opus 4.89.58%

10.Gemini 3.7 Flash8.75%

11.GLM-5.38.33%

12.GLM-5.27.08%

13.Claude Fable 5.16.67%

13.Claude Opus 4.76.67%

13.Claude Opus 56.67%

13.DeepSeek-V4.1-Flash6.67%

13.GLM-5.3-Flash6.67%

18.GPT-6 Astra5.42%

19.Claude Sonnet 4.65%

19.Claude Sonnet 55%

21.MiniMax M34.17%

22.Claude Opus 5.53.75%

22.DeepSeek-V4-Pro3.75%

22.GPT-5.53.75%

25.Gemini 3.6 Flash3.33%

26.Claude Sonnet 5.52.92%

26.GPT-6 Luna2.92%

28.Gemini 3.5 Flash2.5%

28.GPT-5.6 Sol2.5%

30.Inkling2.08%

30.MiMo-V2.5-Pro2.08%

32.GPT-6 Sol1.67%

32.Inkling Small1.67%

32.Kimi K2.61.67%

32.MiMo-V2.51.67%

32.Qwen 3.7 Max1.67%

37.GPT-5.6 Luna1.25%

37.Ling 3.0 Flash1.25%

39.GPT-5.6 Terra0.83%

40.Grok 4.30.42%

40.Mistral Medium 3.50.42%

40.Nemotron 3 Ultra0.42%

43.Gemini 3 Flash Preview0%

43.Gemini 3.1 Flash-Lite0%

43.Gemini 3.1 Pro Preview0%

43.Gemini 3.5 Flash-Lite0%

43.GLM-5.10%

43.GPT-5.40%

43.GPT-5.4 mini0%

43.GPT-5.4 nano0%

43.Grok 4.20 Reasoning0%

43.Kimi K2.50%

43.Mercury 2.50%

43.MiniMax M2.70%

43.Nemotron 3.5 Lightning0%

43.Qwen 3.7 Plus0%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-legal-agent-benchmark:1:overall
Metric: task resolution rate
Scale: 0-100
Participants: 56
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 363.015. Cost per task in USD: 0.184287. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-09-29-d6fb708e5bcc. Source payload hash: d6fb708e5bccec56c7a5e5ad97918d9cc47c5ebefb88af4a1a5771493dadba2f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

LegalBench

Overall

1.Claude Fable 588.56%

2.Claude Fable 5.188.51%

3.Gemini 3.1 Pro Preview87.4%

4.Gemini 3.7 Flash87.26%

5.Gemini 3.8 Flash86.99%

6.Claude Opus 586.97%

7.GPT-5.6 Sol86.97%

8.Gemini 3 Flash Preview86.86%

9.Gemini 3.6 Flash86.71%

10.GPT-5.586.52%

11.Grok 4.686.31%

12.GPT-5.486.04%

13.GPT-586.02%

14.Kimi K386.02%

15.Grok 4.585.97%

16.MiniMax M385.42%

17.Claude Opus 4.785.25%

18.GPT-5.6 Terra85.11%

19.Qwen 3.7 Max84.91%

20.GLM-5.384.84%

21.Kimi K2.684.74%

22.Grok 4.384.46%

23.GLM-5.184.39%

24.Grok 4.784.39%

25.Gemini 3.5 Flash-Lite84.07%

25.GLM-5.284.07%

27.GLM-584.06%

28.GPT-5.6 Luna84.03%

29.MiniMax M2.783.98%

30.GLM-5.3-Flash83.93%

31.Claude Sonnet 583.92%

32.Gemini 3.1 Flash-Lite83.76%

33.o383.76%

34.Qwen 3.8 Max 090283.61%

35.Gemini 3.5 Flash83.6%

36.Claude Opus 4.883.57%

37.GLM-4.783.36%

38.DeepSeek-V4.1-Flash83.28%

39.GPT-4.183.1%

40.Mercury 2.583.06%

41.Inkling Small82.99%

42.Inkling82.95%

43.Claude Opus 4.582.84%

44.Qwen 3.8 27B82.43%

45.Claude Sonnet 4.682.12%

46.Nemotron 3 Ultra82.07%

47.GPT-5 mini81.77%

48.Kimi K2 071181.45%

49.DeepSeek-V380.76%

50.o180.39%

51.DeepSeek-V4-Pro80.32%

52.Kimi K2 Thinking80.2%

53.MiniMax M2.579.96%

54.Command A79.7%

55.Ling 3.0 Flash79.69%

56.GLM-4.679.61%

57.o4-mini79.19%

58.Mistral Large 379.14%

59.MiMo-V2.578.89%

60.GPT-4.1 mini78.04%

61.GPT-5.4 nano77.92%

62.Grok 4.20 Reasoning77.74%

63.DeepSeek-V3 032477.73%

64.MiMo-V2.5-Pro77.13%

65.GPT OSS 120B75.94%

66.GLM-4.575.63%

67.MiniMax M2.175.45%

68.Llama 4 Scout72.04%

69.o3-mini71.54%

70.GPT OSS 20B70.85%

71.Mistral Small 3.169.16%

72.Command R+68.29%

73.Gemma 2 27B67.98%

74.DeepSeek-R167.32%

75.DeepSeek-V3.265.96%

76.GPT-4.1 nano61.06%

77.GPT-5 nano50.13%

78.Command R32.97%

higher is betterAs of September 29, 2026Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-legal-bench:1:overall
Metric: accuracy
Scale: 0-100
Participants: 78
Methodology and context: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 2.697. Cost per task in USD: 0.000114. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-09-29-c1213019b552. Source payload hash: c1213019b55234a262ae86fd4ebfe291530f315c988a4ee32fd501c13777f85f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source: Vals AI

Recorded measurement details

Taste-Bench: Overall (Show measurement, test conditions, and source)

Source value: 36.6
Score: 36.6
Metric: both-orders-correct, engineering/research macro average
Unit: %
Benchmark version: arxiv:2609.25804v2
Category: agentic
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: taste-bench-paper-v2-502
Source type: independent-evaluator
Evaluator: Taste-Bench paper
Status: active
Retrieved at: 2026-09-26
Methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/gpt-5.4-nano. Source model name: GPT-5.4 nano. Provider reported by source: OpenAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: high. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-ee1a62d2083b. Source payload hash: ee1a62d2083b8b1184c46ee9fe59fc02bd5b2f0a63b87dbef0c3f163690c0542. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.
Context: 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

Source Taste-Bench paper

Code Migration: Overall (Show measurement, test conditions, and source)

Source value: 14.467
Score: 14.467
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-code-migration:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 1594.162. Cost per task in USD: 0.416225. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-09-29-f0b317d5bed3. Source payload hash: f0b317d5bed3a2654a9ea39762d5e57f4649306adce3e6103e845ee1b9b94add. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

CorpFin v2: Exact Pages (Show measurement, test conditions, and source)

Source value: 60.373
Score: 60.373
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: exact_pages
Task label: Exact Pages
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:exact_pages
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-06. Reasoning effort: high. Maximum output tokens: 128000. Latency in seconds: 8.67. Standard error: 1.67. Cost per task in USD: 0.000735. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-08-11-d9e58d191cd9. Source payload hash: d9e58d191cd9f1690e3b59f0a37055813038531410f2d73e765676893c0f9526. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Maximum output tokens: 128000

Source Vals AI

CorpFin v2: Max Fitting Context (Show measurement, test conditions, and source)

Source value: 62.121
Score: 62.121
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: max_fitting_context
Task label: Max Fitting Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:max_fitting_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-06. Reasoning effort: high. Maximum output tokens: 128000. Latency in seconds: 12.158. Standard error: 1.656. Cost per task in USD: 0.035271. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-08-11-d9e58d191cd9. Source payload hash: d9e58d191cd9f1690e3b59f0a37055813038531410f2d73e765676893c0f9526. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Maximum output tokens: 128000

Source Vals AI

CorpFin v2: Shared Max Context (Show measurement, test conditions, and source)

Source value: 61.072
Score: 61.072
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: shared_max_context
Task label: Shared Max Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:shared_max_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-06. Reasoning effort: high. Maximum output tokens: 128000. Latency in seconds: 7.363. Standard error: 1.665. Cost per task in USD: 0.012353. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-08-11-d9e58d191cd9. Source payload hash: d9e58d191cd9f1690e3b59f0a37055813038531410f2d73e765676893c0f9526. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Maximum output tokens: 128000

Source Vals AI

CorpFin v2: Overall (Show measurement, test conditions, and source)

Source value: 61.189
Score: 61.189
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Latency in seconds: 9.397. Cost per task in USD: 0.01612. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-09-29-83186cfafe52. Source payload hash: 83186cfafe526f0b8fe2ae86af05e8209fe799c8bfa48292ef1fef5db66650ee. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

EMB: Overall (Show measurement, test conditions, and source)

Source value: 44.751
Score: 44.751
Metric: accuracy
Unit: %
Benchmark version: 1
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-emb:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 1767.043. Cost per task in USD: 0.57312. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-09-29-90535b2fd4dc. Source payload hash: 90535b2fd4dc213540eac0aff8ae769e34b6b1ba54e1ed6d0a51dcfe307d5598. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Finance Agent (v2): Overall (Show measurement, test conditions, and source)

Source value: 38.217
Score: 38.217
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-finance-agent-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-09-27. Latency in seconds: 334.538. Cost per task in USD: 0.15925. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-09-29-addcd124c670. Source payload hash: addcd124c6706893659c1ae26698bf6ffa29c91018ee95b98ab15b9d4b6edcdb. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

GPQA Diamond: Overall (Show measurement, test conditions, and source)

Source value: 77.526
Score: 77.526
Metric: accuracy
Unit: %
Benchmark version: 1
Category: reasoning
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-gpqa:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 29.498. Cost per task in USD: 0.001565. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-09-29-8caf7b4807e8. Source payload hash: 8caf7b4807e87a2880c30218acea7855ea3a11721d56f7e049a77fadaf35dd34. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

LiveCodeBench v6: Overall (Show measurement, test conditions, and source)

Source value: 84.009
Score: 84.009
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-lcb:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Latency in seconds: 60.089. Cost per task in USD: 0.002468. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-09-29-b6ee1006a41a. Source payload hash: b6ee1006a41aab0c68eef9d3e42a2151634fffaaeb6bdc2230a6d02994198a59. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Harvey's Legal Agent Benchmark: Overall · Task fully resolved (Show measurement, test conditions, and source)

Source value: 0
Score: 0
Metric: task resolution rate
Unit: %
Benchmark version: 1
Category: legal
Task: overall
Task label: Overall · Task fully resolved
Direction: Higher is better
Comparison cohort: vals-legal-agent-benchmark:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 363.015. Cost per task in USD: 0.184287. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-09-29-d6fb708e5bcc. Source payload hash: d6fb708e5bccec56c7a5e5ad97918d9cc47c5ebefb88af4a1a5771493dadba2f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

LegalBench: Overall (Show measurement, test conditions, and source)

Source value: 77.92
Score: 77.92
Metric: accuracy
Unit: %
Benchmark version: 1
Category: legal
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-legal-bench:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-09-29
Methodology: Source model ID: openai/gpt-5.4-nano-2026-03-17. Source model name: GPT-5.4 nano. Model snapshot: gpt-5.4-nano-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-27. Latency in seconds: 2.697. Cost per task in USD: 0.000114. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-09-29-c1213019b552. Source payload hash: c1213019b55234a262ae86fd4ebfe291530f315c988a4ee32fd501c13777f85f. Parser version: vals-astro-v2. Imported at: 2026-09-29

Source Vals AI

Page 1 of 11

Specifications and access

Specification	Value and source
Weights	ProprietarySource
Access	APISource
Context window	400,000 tokensSource
Knowledge cutoff	August 31, 2025Source
Notes	400K context, parameter count not disclosedSource
API model ID	gpt-5.4-nanoSource
Maximum output	128,000 tokensSource
Input	Text, ImageSource
Output	TextSource
Reasoning mode	none, low, medium, high, xhighSource
API interfaces	Responses API, Chat Completions APISource
Tool calling	YesSource
Structured output	YesSource
Release	2026-03-17Source

Predecessor and successor

Sources and data date

Type	Evidence and data date
Speed	OpenAI, xhigh effort. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · GPT-5.4 nano API model documentation · Editorial description reviewed October 4, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · GPT-5.4 nano API model documentation · Editorial description reviewed October 4, 2026
Research status	Research date September 8, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording OpenAI does not publish parameter counts.
Not found in the inspected sources	independent-benchmarks Original wording The evaluated rows in this artifact are the exact OpenAI release-table entries; no independent evaluator row was merged.
Additional source	Provider model lifecycle (retrieved October 2, 2026) · OpenAI API deprecations · Editorial description reviewed October 4, 2026
Additional source	Taste-Bench paper (retrieved September 26, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved August 11, 2026; September 29, 2026)
Additional source	Vals AI (retrieved September 29, 2026)
Additional source	OpenAI (retrieved September 8, 2026) · Introducing GPT-5.4 mini and nano · Editorial description reviewed October 4, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Creative Commons Attribution 4.0

GPT-5.4 nano

Published benchmarks

Taste-Bench

Coding

Code Migration

LiveCodeBench v6

Finance

CorpFin v2

CorpFin v2

EMB

Finance Agent (v2)

GPQA Diamond

Legal

Harvey's Legal Agent Benchmark

LegalBench

Recorded measurement details

Our tests

Pelican on a bicycle

Popularity

Specifications and access

Published prices

Predecessor and successor

Sources and data date