Language modelActive

GPT-5.4 mini

OpenAI

Released: March 17, 2026
Data date: September 8, 2026
Predecessor: GPT-5 mini

Benchmark-Meta-Score: Unavailable
Minimum coverage not met
Speed: 159 tokens/s
Output throughput
Price: $0.75 · $4.5
Input · Output / 1M tokens
Input: Text, Image
Output: Text

GPT-5.4 mini improved coding, reasoning, and tool use compared with GPT-5 mini. OpenAI also reported more than twice the speed at launch. Mini brings capabilities from the larger GPT-5.4 line to a compact tier.

Mini supports native computer use and can handle individual coding tasks as a subagent for a larger model. The co-released GPT-5.4 nano tier handles simpler supporting tasks.

Specifications and access

Specification	Value and source
Weights	ProprietarySource
Access	APISource
Context window	400,000 tokensSource
Knowledge cutoff	August 31, 2025Source
Notes	400K context, parameter count not disclosedSource
API model ID	gpt-5.4-miniSource
Maximum output	128,000 tokensSource
Input	Text, ImageSource
Output	TextSource
Reasoning mode	none, low, medium, high, xhighSource
API interfaces	Responses API, Chat Completions APISource
Tool calling	YesSource
Structured output	YesSource
Release	2026-03-17Source

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-10-15 - 2026-10-19

Overall index

Not enough coverage for a score yet

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 34 %
Test families: 32 / 71
Sources: 3
Mathematical bounds with missing values: 13.3-79.1 / 100

Coding

Not enough coverage for a score yet

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 43 %
Test families: 7 / 15
Sources: 3
Mathematical bounds with missing values: 13.7-70.4 / 100

Reasoning

Not enough coverage for a score yet

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 43 %
Test families: 4 / 7
Sources: 2
Mathematical bounds with missing values: 21.3-78.4 / 100

Knowledge

Index 38.5 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 67 %
Test families: 2 / 3
Sources: 2
Mathematical bounds with missing values: 25.7-59.0 / 100

Agents

Index 41.3 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 50 %
Test families: 6 / 12
Sources: 1
Mathematical bounds with missing values: 20.6-70.6 / 100

General

Not enough coverage for a score yet

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 40 %
Test families: 2 / 5
Sources: 2
Mathematical bounds with missing values: 19.3-79.3 / 100

Finance

Index 30.0 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 86 %
Test families: 6 / 7
Sources: 1
Mathematical bounds with missing values: 25.7-40.0 / 100

Legal

Index 12.0 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 50 %
Test families: 2 / 4
Sources: 1
Mathematical bounds with missing values: 6.0-56.0 / 100

Multimodal

Index 25.8 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 50 %
Test families: 2 / 4
Sources: 3
Mathematical bounds with missing values: 12.9-62.9 / 100

Education

Not enough coverage for a score yet

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Panel-weight coverage: 50 %
Test families: 1 / 2
Sources: 1
Mathematical bounds with missing values: 40.5-90.5 / 100

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v1

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture; when dates match, the wider slice is used. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. When selecting between multiple results, unknown or ambiguous tiers exclude the model from that series. Unambiguous single configurations do not require a tier ordering. The best individual result for each model is not selected. Equal raw values share the same rank percentile. Multiple series from the same evaluator within a family receive equal weight; evaluators within a family, families within an area, and areas in the overall index then receive equal weight. Unmeasured series remain visible in coverage and bounds. An overall score requires at least four areas, six test families, and two sources. An area score requires at least two test families. Every overall and area score also requires at least 50% of its panel weight.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured portions are not entered as zero. They reduce coverage and contribute only to the displayed bounds. Scores with different coverage are therefore not equally complete.

The bounds hold observed percentiles fixed and allow missing contributions anywhere from 0 to 100. They are not confidence intervals or predictions of future recalculations.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 32 test families and 12 of 36 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: AA-AnalystAgent
Agents · Family index: 15.8 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-aa-analyst-agent
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  15 %
  Metric:
  pass^5 reliability
  Version:
  AA cohort: 80 questions
  Participating models:
  39
  Rank percentile within the cohort:
  15.8 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-aa-analyst-agent:AA cohort: 80 questions:standalone:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-aa-analyst-agent","AA cohort: 80 questions","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Harness:
  Artificial Analysis Stirrup
  Agent:
  Stirrup analyst agent
  Tool policy:
  100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission
  Attempts:
  pass^5, 5 independent attempts per question
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 15.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: AA-Briefcase
Agents · Family index: 39.0 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-aa-briefcase
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  719.36 Elo
  Metric:
  AA-Briefcase Elo
  Version:
  v1.1
  Participating models:
  83
  Rank percentile within the cohort:
  39.0 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-aa-briefcase:v1.1:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-aa-briefcase","v1.1","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: AA-LCR
Reasoning · Family index: 46.3 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-aa-lcr
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.02 %
  Raw value:
  77 %
  Metric:
  Pass rate
  Version:
  v1.1
  Participating models:
  83
  Rank percentile within the cohort:
  46.3 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-aa-lcr:v1.1:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-aa-lcr","v1.1","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 77.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: APEX-Agents-AA
Agents · Family index: 57.7 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-apex-agents-aa
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  28.1710914454277 %
  Metric:
  Pass@1 success rate
  Version:
  AA public cohort: 452 tasks
  Participating models:
  27
  Rank percentile within the cohort:
  57.7 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-apex-agents-aa:AA public cohort: 452 tasks:standalone:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-apex-agents-aa","AA public cohort: 452 tasks","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Harness:
  Artificial Analysis Stirrup
  Agent:
  Stirrup agent
  Tool policy:
  200-turn cap; MCP-backed workplace tools in Archipelago
  Attempts:
  pass@1, averaged over 3 repeats per task
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 452 tasks from the public 480-task release, with three repeats per task. A run passes only if every rubric criterion is satisfied; the leaderboard averages pass@1 across repeats.. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: 200-turn cap; MCP-backed workplace tools in Archipelago. Attempts: pass@1, averaged over 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 452 tasks from the public 480-task release, with three repeats per task. A run passes only if every rubric criterion is satisfied; the leaderboard averages pass@1 across repeats.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: 200-turn cap; MCP-backed workplace tools in Archipelago. Attempts: pass@1, averaged over 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 28.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: AutomationBench-AA
Agents · Family index: 57.3 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-automationbench-aa
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  26.204887225711754 %
  Metric:
  Objectives completed without guardrail violations
  Version:
  unspecified
  Participating models:
  83
  Rank percentile within the cohort:
  57.3 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-automationbench-aa:unspecified:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-automationbench-aa","unspecified","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 26.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: GDP.pdf
Reasoning · Family index: 62.2 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-gdp-pdf
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.02 %
  Raw value:
  13.4 %
  Metric:
  All-pass rate
  Version:
  unspecified
  Participating models:
  83
  Rank percentile within the cohort:
  62.2 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-gdp-pdf:unspecified:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-gdp-pdf","unspecified","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 13.4%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: GDPval-AA
Agents · Family index: 46.3 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-gdpval-aa
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  1,011.97 Elo
  Metric:
  GDPval-AA Elo
  Version:
  v2.1
  Participating models:
  83
  Rank percentile within the cohort:
  46.3 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-gdpval-aa:v2.1:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-gdpval-aa","v2.1","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: IFBench
General · Family index: 79.7 / 100
Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-ifbench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.43 %
  Raw value:
  73.265306122449 %
  Metric:
  IFBench prompt-level loose accuracy (pass@1)
  Version:
  AA test set: 294 questions
  Participating models:
  70
  Rank percentile within the cohort:
  79.7 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-ifbench:AA test set: 294 questions:standalone:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-ifbench","AA test set: 294 questions","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Tool policy:
  Single-turn answer evaluated with the official IFBench source code
  Attempts:
  pass@1, 5 repeats per question
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 73.3%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: ITBench-AA (SRE)
Agents · Family index: 31.4 / 100
Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-itbench-aa
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.6 %
  Raw value:
  35.2165725047081 %
  Metric:
  Average precision at full recall
  Version:
  AA cohort: 59 tasks
  Participating models:
  36
  Rank percentile within the cohort:
  31.4 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-itbench-aa:AA cohort: 59 tasks:standalone:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-itbench-aa","AA cohort: 59 tasks","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Harness:
  Artificial Analysis Stirrup
  Agent:
  Stirrup agent
  Tool policy:
  Shell access to an offline Kubernetes incident snapshot
  Attempts:
  Average precision at full recall, 3 repeats per task
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 35.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: AA-Omniscience
Knowledge · Family index: 32.9 / 100
Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-omniscience
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.38 %
  Raw value:
  -18.916666666666668 points
  Metric:
  AA-Omniscience Index
  Version:
  unspecified
  Participating models:
  83
  Rank percentile within the cohort:
  32.9 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-omniscience:unspecified:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-omniscience","unspecified","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: GPQA Diamond
Reasoning · Family index: 43.4 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-gpqa
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.51 %
  Raw value:
  83.08 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  77
  Rank percentile within the cohort:
  43.4 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-gpqa:1:overall
  Reasoning tier:
  xhigh
  Documented comparison setup
  Publication cohort:
  vals-gpqa:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: xhigh. Maximum output tokens: 128000
  Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 63.11. Standard error: 2.457. Cost per task in USD: 0.012743. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-10-15-951eb5224097. Source payload hash: 951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Humanity’s Last Exam
Reasoning · Family index: 37.8 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-humanitys-last-exam
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.51 %
  Raw value:
  28.127896200185397 %
  Metric:
  Text-only pass@1 accuracy
  Version:
  unspecified
  Participating models:
  83
  Rank percentile within the cohort:
  37.8 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-humanitys-last-exam:unspecified:v4.3.2:reasoning:tokens=0:suffix=Xhigh
  Reasoning tier:
  Xhigh
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-humanitys-last-exam","unspecified","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Setup and model parameters
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Xhigh
  Exact AA configuration: GPT-5.4 mini (Xhigh) (release gpt-5-4-mini, row gpt-5-4-mini, UUID ba242e40-83b7-4cd3-a0e0-b56237984914, reasoning=true, source reasoning-token configuration budget=0, suffix=Xhigh). This configured value is not measured token consumption.. Source model ID: artificial-analysis/ba242e40-83b7-4cd3-a0e0-b56237984914/gpt-5-4-mini/gpt-5-4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: GPT-5.4 mini (Xhigh). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Xhigh. Score display: 28.1%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: LMArena Code Arena WebDev
Coding · Family index: 8.3 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LMArena Text, Style Control
General · Family index: 16.7 / 100
Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: LMArena Vision, Style Control
Multimodal · Family index: 20.0 / 100
Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MMMU-Pro
Multimodal · Family index: 31.7 / 100
Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: SWE-bench Verified
Coding · Family index: 35.6 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 50 %
Test family: Terminal-Bench
Coding · Family index: 45.6 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Code Migration
Coding · Family index: 22.6 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: CorpFin v2
Finance · Family index: 43.2 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: EMB
Finance · Family index: 25.4 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Finance Agent (v2)
Finance · Family index: 33.3 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: LiveCodeBench v6
Coding · Family index: 41.6 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Harvey's Legal Agent Benchmark
Legal · Family index: 10.7 / 100
Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: Legal Research Bench
Legal · Family index: 13.3 / 100
Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MMLU-Pro
Knowledge · Family index: 44.2 / 100
Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MortgageTax
Finance · Family index: 26.9 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: ProgramBench
Coding · Family index: 24.5 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: SAGE
Education · Family index: 81.0 / 100
Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: Tax Agent Bench
Finance · Family index: 18.2 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Family index: 32.9 / 100
Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Family index: 45.8 / 100
Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	13	1	25
Vals AI	22	2	195
LMArena	3	0	42

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

Published benchmarks

Taste-Bench

Overall

502 technical decision points from engineering and research workflows. Each choice is presented in both answer orders. The overall score averages engineering and research and requires both answer orders to be correct. Unparseable responses count as wrong. Reasoning settings and output budget are part of the setup.

Note (GPT-5.4 mini): 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

1.GPT-5.6 Sol59.7%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

2.GPT-5.559.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

3.Claude Opus 555.5%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

4.Grok 4.554.6%

Note: 10 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

5.GPT-5.6 Terra54%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

6.GLM-5.253.9%

Note: 17 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

7.Claude Sonnet 551.6%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

8.GPT-5.6 Luna49%

Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

9.MiniMax M345.3%

Note: 3 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

10.DeepSeek-V4-Flash43.3%

Note: 2 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

11.GPT-5.4 mini40.1%

12.Mistral Medium 3.537.7%

Note: 24 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

13.GPT-5.4 nano36.6%

Note: 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

14.Grok 4.20 Reasoning15.7%

Note: 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

higher is betterAs of September 26, 2026Scale: 0% - 100%

Source: Taste-Bench paper

|

gradually.ai

Methodology and context

Comparison cohort: taste-bench-paper-v2-502
Metric: both-orders-correct, engineering/research macro average
Scale: 0-100
Participants: 14
Methodology and context: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/gpt-5.4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: high. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-8d521554a8e2. Source payload hash: 8d521554a8e2145aac33904b3e38d2b8fa2ad428110361d37e0b277ded8478b8. Parser version: curated-v1. Imported at: 2026-10-15

Source: Taste-Bench paper

LMArena Vision, Style Control

Humor

Note (GPT-5.4 mini): The Arena score measures human preference rather than a fixed capability test. The confidence interval, Arena category, and model configuration are part of the result.

1.Claude Opus 4.61,296.35 rating

Note: The Arena score measures human preference rather than a fixed capability test. The confidence interval, Arena category, and model configuration are part of the result.

2.GPT-5.4 mini1,238.17 rating

higher is betterAs of October 15, 2026Scale: 0 rating - 1,296.35 rating

Source: LMArena

|

gradually.ai

Methodology and context

Comparison cohort: lmarena-vision-style-control:2026-10-09:humor
Metric: style-controlled Bradley-Terry rating
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 2
Methodology and context: Arena rating from 869 published pairwise votes.. Source model ID: lmarena/gpt-5.4-mini-high. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-high. Evaluator: LMArena. Dataset type: public-crowdsourced-preferences. Leaderboard date: 2026-10-09. Harness: Arena Vision, style control. Reasoning effort: high. Source ID: lmarena. Evidence capture ID: lmarena-vision_style_control-4589dee83b594891961ae5190c508a9288d2c2ae-6cf1fae8efc9. Source payload hash: 6cf1fae8efc9ef6fe066f27d5bbe558cd360803a9c0fa6d89905aa53934efdb4. Parser version: lmarena-dataset-v1. Imported at: 2026-10-15

Source: LMArena

Coding

Code Migration

Sub-task	GPT-5.4 mini (This model)	GPT-5.5	GPT-5.4
CLI	8.2%	36.88% (best value in this row)	23.79%
COBOL	27.14%	70% (best value in this row)	68.57%
Overall	12.94%	45.16% (best value in this row)	34.98%

higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

Code Migration

Code Quality

1.GPT-5.43.63 rating

2.GPT-5.53.5 rating

3.GPT-5.4 mini3.05 rating

higher is betterAs of October 15, 2026Scale: 0 rating - 3.63 rating

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-code-migration:1:code_quality
Metric: mean reviewer code-quality rating
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 3
Methodology and context: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 1733.471. Standard error: 0.076. Cost per task in USD: 1.953198. Score display: 3.053 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

Finance

CorpFin v2

Sub-task	GPT-5.4 mini (This model)	GPT-5.5	GPT-5.4	GPT-5.6 Terra
Exact Pages	61.54%	65.62% (best value in this row)	64.1%	63.17%
Max Fitting Context	60.96%	70.05% (best value in this row)	64.92%	66.32%
Overall	60.92%	68.42% (best value in this row)	65.27%	65.31%
Shared Max Context	60.26%	69.58% (best value in this row)	66.78%	66.43%

higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

Recorded measurement details

Taste-Bench: Overall (Show measurement, test conditions, and source)

Source value: 40.1
Score: 40.1
Metric: both-orders-correct, engineering/research macro average
Unit: %
Benchmark version: arxiv:2609.25804v2
Category: agentic
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: taste-bench-paper-v2-502
Source type: independent-evaluator
Evaluator: Taste-Bench paper
Status: active
Retrieved at: 2026-09-26
Methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/gpt-5.4-mini. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: high. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-8d521554a8e2. Source payload hash: 8d521554a8e2145aac33904b3e38d2b8fa2ad428110361d37e0b277ded8478b8. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.
Context: 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.

Source Taste-Bench paper

LMArena Vision, Style Control: Humor (Show measurement, test conditions, and source)

Source value: 1238.1732758740177
Score: 1238.1732758740177
Metric: style-controlled Bradley-Terry rating
Unit: rating
Benchmark version: style-control-v1
Category: multimodal
Task: humor
Task label: Humor
Direction: Higher is better
Comparison cohort: lmarena-vision-style-control:2026-10-09:humor
Confidence interval: 21.176563
Source type: official-leaderboard
Evaluator: LMArena
Status: active
Retrieved at: 2026-10-15
Methodology: Arena rating from 869 published pairwise votes.. Source model ID: lmarena/gpt-5.4-mini-high. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-high. Evaluator: LMArena. Dataset type: public-crowdsourced-preferences. Leaderboard date: 2026-10-09. Harness: Arena Vision, style control. Reasoning effort: high. Source ID: lmarena. Evidence capture ID: lmarena-vision_style_control-4589dee83b594891961ae5190c508a9288d2c2ae-6cf1fae8efc9. Source payload hash: 6cf1fae8efc9ef6fe066f27d5bbe558cd360803a9c0fa6d89905aa53934efdb4. Parser version: lmarena-dataset-v1. Imported at: 2026-10-15
Comparison methodology: Arena rating from 869 published pairwise votes.. Harness: Arena Vision, style control. Reasoning effort: high
Context: The Arena score measures human preference rather than a fixed capability test. The confidence interval, Arena category, and model configuration are part of the result.

Code Migration: CLI (Show measurement, test conditions, and source)

Source value: 8.2
Score: 8.2
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: cli
Task label: CLI
Direction: Higher is better
Comparison cohort: vals-code-migration:1:cli
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 1956.927. Standard error: 1.295. Cost per task in USD: 2.31844. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

Code Migration: COBOL (Show measurement, test conditions, and source)

Source value: 27.143
Score: 27.143
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: cobol
Task label: COBOL
Direction: Higher is better
Comparison cohort: vals-code-migration:1:cobol
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 1063.103. Standard error: 14.038. Cost per task in USD: 0.857472. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

Code Migration: Overall (Show measurement, test conditions, and source)

Source value: 12.936
Score: 12.936
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-code-migration:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 1733.471. Standard error: 3.641. Cost per task in USD: 1.953198. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

Code Migration: Code Quality (Show measurement, test conditions, and source)

Source value: 3.053
Score: 3.053
Metric: mean reviewer code-quality rating
Unit: rating
Benchmark version: 1
Category: coding
Task: code_quality
Task label: Code Quality
Direction: Higher is better
Comparison cohort: vals-code-migration:1:code_quality
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 1733.471. Standard error: 0.076. Cost per task in USD: 1.953198. Score display: 3.053 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

CorpFin v2: Exact Pages (Show measurement, test conditions, and source)

Source value: 61.538
Score: 61.538
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: exact_pages
Task label: Exact Pages
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:exact_pages
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 44.668. Standard error: 1.661. Cost per task in USD: 0.014516. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

CorpFin v2: Max Fitting Context (Show measurement, test conditions, and source)

Source value: 60.956
Score: 60.956
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: max_fitting_context
Task label: Max Fitting Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:max_fitting_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 66.943. Standard error: 1.665. Cost per task in USD: 0.146241. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

CorpFin v2: Overall (Show measurement, test conditions, and source)

Source value: 60.917
Score: 60.917
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 52.297. Standard error: 0.962. Cost per task in USD: 0.076368. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

CorpFin v2: Shared Max Context (Show measurement, test conditions, and source)

Source value: 60.256
Score: 60.256
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: shared_max_context
Task label: Shared Max Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:shared_max_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: openai/gpt-5.4-mini-2026-03-17. Source model name: GPT-5.4 mini. Provider reported by source: OpenAI. Model snapshot: gpt-5.4-mini-2026-03-17. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Reasoning effort: xhigh. Maximum output tokens: 128000. Latency in seconds: 45.279. Standard error: 1.671. Cost per task in USD: 0.068348. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Reasoning effort: xhigh. Maximum output tokens: 128000

Page 1 of 21

Predecessor and successor

Sources and data date

Type	Evidence and data date
Speed	OpenAI, xhigh effort. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 7, 2026. Source: Artificial Analysis · Speed (Artificial Analysis)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 7, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · GPT-5.4 mini API model documentation · Editorial description reviewed October 4, 2026
Output	Documented modalities. Retrieved September 7, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · GPT-5.4 mini API model documentation · Editorial description reviewed October 4, 2026
Research status	Research date September 8, 2026. 3 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording OpenAI does not publish parameter counts.
Not found in the inspected sources	independent-benchmarks Original wording The evaluated rows in this artifact are the exact OpenAI release-table entries; no independent evaluator row was merged.
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Taste-Bench paper (retrieved September 26, 2026)
Additional source	LMArena (retrieved October 15, 2026)
Additional source	LMArena (retrieved October 15, 2026)
Additional source	LMArena (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved August 11, 2026; October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	OpenAI (retrieved September 8, 2026) · Introducing GPT-5.4 mini and nano · Editorial description reviewed October 4, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0