Language modelActive

GPT-4o mini

OpenAI

Released: July 18, 2024
Data date: September 8, 2026
Successor: GPT-4.1 mini

Benchmark-Meta-Score: 0 / 100
Provisional, not yet placed. Standard error ± 0
Speed: 149.3 tokens/s
Output throughput
Price: $0.15 · $0.6
Input · Output / 1M tokens
Input: Text, Image
Output: Text

GPT-4o mini brought the GPT-4o generation’s text and image inputs to a compact, low-cost API tier.

GPT-4.1 mini followed with a much larger context window and a stronger focus on coding and instruction following.

Specifications and access

Specification	Value and source
Weights	ProprietarySource not specified
Access	APISource
Parameters	8B (estimated)Source not specified
Context window	128,000 tokensSource
Knowledge cutoff	October 2023Source
Notes	Analyst estimates suggest ~8B parametersSource not specified
API model ID	gpt-4o-miniSource
Maximum output	16,384 tokensSource
Knowledge cutoff	2023-10-01Source
Input	Text, ImageSource
Output	TextSource
Reasoning mode	noneSource
API interfaces	Responses API, Chat Completions APISource
Tool calling	YesSource
Structured output	YesSource
Release	2024-07-18Source

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-08-05 - 2026-10-19

Overall index

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 14
Test families: 13 / 65
Sources: 3
Share of the fixed test panel (informative): 22 %

Coding

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 15
Sources: 1
Share of the fixed test panel (informative): 7 %

Reasoning

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 7
Sources: 1
Share of the fixed test panel (informative): 7 %

Mathematics

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 2
Test families: 2 / 5
Sources: 1
Share of the fixed test panel (informative): 40 %

Knowledge

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 3
Sources: 1
Share of the fixed test panel (informative): 33 %

Agents

Index below 1 / 100

provisional

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 2
Test families: 2 / 12
Sources: 1
Share of the fixed test panel (informative): 17 %

General

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 5
Sources: 1
Share of the fixed test panel (informative): 20 %

Finance

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 3
Test families: 3 / 7
Sources: 1
Share of the fixed test panel (informative): 43 %

Health

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 3
Sources: 1
Share of the fixed test panel (informative): 33 %

Multimodal

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 2
Test families: 1 / 4
Sources: 2
Share of the fixed test panel (informative): 25 %

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v3

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. Unknown or ambiguous tiers exclude the model from that series. Within each series, percentages and rates are logit-transformed, other units stay unchanged, and all values are then standardized. A one-factor model then estimates one shared strength per model and, per series, how sharply it separates the models. Area scores move away from that strength only as far as the model's own results in the area support. The overall score weights the core areas at 75% and finance, law and health at 25%. It requires at least eight test series from two sources and three areas. The 0 to 100 scale refers to a fixed reference group of 29 language models whose average sits at 50.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured series are not entered as zero. Where a model has no own results in a core area, its overall strength stands in, and the note “estimated without own results in …” names the area. Fewer test series lead to a larger standard error.

The 95% interval follows from the standard error of 200 recalculations with randomly redrawn test families. It describes how precisely the recorded test series determine the value. It is no prediction for future tasks or for later recalculations with other participants.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 13 test families and 12 of 14 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: IFBench
General · Mean rank percentile of the family: 8.7 / 100
Family weight in the fixed test panel (informative): 2.14 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-ifbench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.14 %
  Raw value:
  30.952380952381 %
  Metric:
  IFBench prompt-level loose accuracy (pass@1)
  Version:
  AA test set: 294 questions
  Participating models:
  70
  Rank percentile within the cohort:
  8.7 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:tokens=0:suffix=unspecified
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-ifbench","AA test set: 294 questions","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Tool policy:
  Single-turn answer evaluated with the official IFBench source code
  Attempts:
  pass@1, 5 repeats per question
  Setup and model parameters
  Exact AA configuration: GPT-4o mini (release gpt-4o-mini, row gpt-4o-mini, UUID b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning
  Exact AA configuration: GPT-4o mini (release gpt-4o-mini, row gpt-4o-mini, UUID b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6/gpt-4o-mini/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: GPT-4o mini. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Score display: 31.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Test family: GPQA Diamond
Reasoning · Mean rank percentile of the family: 3.3 / 100
Family weight in the fixed test panel (informative): 1.53 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-gpqa
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.77 %
  Raw value:
  44.192 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  77
  Rank percentile within the cohort:
  3.3 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-gpqa:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-gpqa:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 15.32. Standard error: 2.495. Cost per task in USD: 0.000536. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-10-15-951eb5224097. Source payload hash: 951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: GAIA Reliability
Agents · Mean rank percentile of the family: 0.0 / 100
Family weight in the fixed test panel (informative): 0.89 % · Coverage of this test family: 100 %
- Princeton HAL
  Measurement:
  hal-gaia-reliability
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.89 %
  Raw value:
  26.3 %
  Metric:
  exact-match accuracy
  Version:
  final leak-cleaned public validation set
  Participating models:
  9
  Rank percentile within the cohort:
  0.0 %
  Retrieved:
  2026-08-06
  Comparison cohort:
  hal-gaia-reliability-hal-reliability-overall
  Documented comparison setup
  Publication cohort:
  hal-gaia-reliability-hal-reliability-overall
  Dataset type:
  public
  Harness:
  HAL Reliability
  Agent:
  HAL Reliability
  Setup and model parameters
  165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Harness: HAL Reliability. Agent: HAL Reliability
  165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-441190f58bf4. Source payload hash: 441190f58bf43ee1768f98ba21281cc3c97dbd93e50fdde45d9e816fc9137976. Parser version: curated-v1. Imported at: 2026-10-15
Test family: τ-bench Airline Clean
Agents · Mean rank percentile of the family: 0.0 / 100
Family weight in the fixed test panel (informative): 0.89 % · Coverage of this test family: 100 %
- Princeton HAL
  Measurement:
  hal-tau-airline-clean
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.89 %
  Raw value:
  29.5 %
  Metric:
  task success rate
  Version:
  26-task cleaned subset
  Participating models:
  9
  Rank percentile within the cohort:
  0.0 %
  Retrieved:
  2026-08-05
  Comparison cohort:
  hal-tau-airline-clean-hal-reliability-overall
  Documented comparison setup
  Publication cohort:
  hal-tau-airline-clean-hal-reliability-overall
  Dataset type:
  public
  Harness:
  HAL Reliability
  Agent:
  HAL Reliability
  Setup and model parameters
  26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Harness: HAL Reliability. Agent: HAL Reliability
  26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-d92ebee642b2. Source payload hash: d92ebee642b24d08cf3917332b6eb47c5e1332b475553443f7b181dd115c5b91. Parser version: curated-v1. Imported at: 2026-10-15
Test family: MMMU-Pro
Multimodal · Mean rank percentile of the family: 2.8 / 100
Family weight in the fixed test panel (informative): 2.68 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-mmmu-pro
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  1.34 %
  Raw value:
  41.5028901734104 %
  Metric:
  pass@1 accuracy
  Version:
  AA cohort: 1,730 questions
  Participating models:
  57
  Rank percentile within the cohort:
  3.6 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:non-reasoning:tokens=0:suffix=unspecified
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-mmmu-pro","AA cohort: 1,730 questions","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Tool policy:
  Ten-option multiple choice; regex answer extraction
  Attempts:
  pass@1, one run per question
  Setup and model parameters
  Exact AA configuration: GPT-4o mini (release gpt-4o-mini, row gpt-4o-mini, UUID b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning
  Exact AA configuration: GPT-4o mini (release gpt-4o-mini, row gpt-4o-mini, UUID b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/b5c1c91a-7474-4409-9a9c-9c2ac45d9eb6/gpt-4o-mini/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: GPT-4o mini. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Score display: 41.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-mmmu
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  1.34 %
  Raw value:
  56.557 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  48
  Rank percentile within the cohort:
  2.1 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-mmmu:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-mmmu:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 45.016. Standard error: 1.191. Cost per task in USD: 0.00037. Source ID: vals-ai. Evidence capture ID: vals-mmmu-2026-10-15-29395186eddd. Source payload hash: 29395186eddd2c70601228c584a14b3c1d8f9dba4b68256561b46b0dd03c635a. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: CorpFin v2
Finance · Mean rank percentile of the family: 4.1 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-corp-fin-v2
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  45.455 %
  Metric:
  accuracy
  Version:
  2
  Participating models:
  75
  Rank percentile within the cohort:
  4.1 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-corp-fin-v2:2:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-corp-fin-v2:2:overall
  Dataset type:
  private
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 0.041. Standard error: 0.977. Cost per task in USD: 0.008848. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: LiveCodeBench v6
Coding · Mean rank percentile of the family: 1.3 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-lcb
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.71 %
  Raw value:
  26.423 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  78
  Rank percentile within the cohort:
  1.3 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-lcb:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-lcb:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 7.364. Standard error: 1.129. Cost per task in USD: 0.000298. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-10-15-1a7c7dabc9d9. Source payload hash: 1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MATH 500
Mathematics · Mean rank percentile of the family: 4.5 / 100
Family weight in the fixed test panel (informative): 2.14 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-math500
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.14 %
  Raw value:
  72.6 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  23
  Rank percentile within the cohort:
  4.5 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-math500:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-math500:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 0. Maximum output tokens: 8192
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-01-09. Reasoning effort: high. Temperature: 0. Maximum output tokens: 8192. Latency in seconds: 6.304. Standard error: 3.898. Cost per task in USD: 0.00035. Source ID: vals-ai. Evidence capture ID: vals-math500-2026-10-15-435ec4da1112. Source payload hash: 435ec4da1112362c0a6bb4afeab53bf7b0faf2000e1ddbe1a878735dca1fe35b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MedQA
Health · Mean rank percentile of the family: 12.8 / 100
Family weight in the fixed test panel (informative): 2.78 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-medqa
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.78 %
  Raw value:
  72.436 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  40
  Rank percentile within the cohort:
  12.8 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-medqa:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-medqa:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-04-16. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 2.322. Standard error: 0.799. Cost per task in USD: 0.000157. Source ID: vals-ai. Evidence capture ID: vals-medqa-2026-10-15-d67905978d93. Source payload hash: d67905978d932f64698ebe9e230d9519e5bae1c9d7176dda8379dc3874b6f90f. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MGSM
Mathematics · Mean rank percentile of the family: 13.8 / 100
Family weight in the fixed test panel (informative): 2.14 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-mgsm
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.14 %
  Raw value:
  86.182 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  30
  Rank percentile within the cohort:
  13.8 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-mgsm:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-mgsm:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-01-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 4.033. Standard error: 2.133. Cost per task in USD: 0.000187. Source ID: vals-ai. Evidence capture ID: vals-mgsm-2026-10-15-1b87f953e52c. Source payload hash: 1b87f953e52c00bf4bdd90c3add26165f644eeb7a74a8382d2a0324ff0cc1040. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MMLU-Pro
Knowledge · Mean rank percentile of the family: 1.3 / 100
Family weight in the fixed test panel (informative): 3.57 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-mmlu-pro
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  3.57 %
  Raw value:
  62.735 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  78
  Rank percentile within the cohort:
  1.3 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-mmlu-pro:1:overall
  Reasoning tier:
  high
  Documented comparison setup
  Publication cohort:
  vals-mmlu-pro:1:overall
  Dataset type:
  public
  Setup and model parameters
  Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384
  Source model ID: openai/gpt-4o-mini-2024-07-18. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Model snapshot: gpt-4o-mini-2024-07-18. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 5.086. Standard error: 0.461. Cost per task in USD: 0.000412. Source ID: vals-ai. Evidence capture ID: vals-mmlu_pro-2026-10-15-dab025507f46. Source payload hash: dab025507f46c716dd8d145e90ec0a737d9c56219853f41e4b1cc3c5e8c222b4. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MortgageTax
Finance · Mean rank percentile of the family: 11.5 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Mean rank percentile of the family: 2.6 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	2	1	9
Vals AI	11	0	56
Princeton HAL	2	0	12

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

6 measurements from these sources still lack a verified test-family identity and do not increase the test-series count.

Published benchmarks

Agent tasks

GAIA Reliability

Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Sub-task	GPT-4o mini (This model)	Claude Opus 4.5	Claude Opus 4.7	Gemini 3.1 Pro Preview	Gemini 3.5 Flash	Gemini 2.5 Pro	Gemini 2.5 Flash	o1	GPT-5.5
Consistency	0.73	0.81 (best value in this row)	0.81 (best value in this row)	0.76	0.75	0.72	0.7	0.68	0.61
Predictability	0.58	0.84 (best value in this row)	0.73	0.78	0.8	0.71	0.64	0.68	0.8
Reliability	0.76	0.85 (best value in this row)	0.84	0.82	0.84	0.78	0.74	0.79	0.79
Robustness	0.97	0.91	0.99	0.94	0.96	0.91	0.87	1 (best value in this row)	0.95
Safety	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)

Unit: ratinghigher is betterAs of August 6, 2026Source: Princeton HALbars scaled within each row, not across rows

GAIA Reliability

Accuracy

Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

1.Gemini 3.5 Flash79.2%

2.Gemini 3.1 Pro Preview76.2%

3.Claude Opus 4.773.3%

4.Claude Opus 4.568.5%

5.GPT-5.562.8%

6.Gemini 2.5 Pro52.5%

7.Gemini 2.5 Flash46.7%

8.o134.4%

9.GPT-4o mini26.3%

higher is betterAs of August 6, 2026Scale: 0% - 100%

Source: Princeton HAL

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Metric: exact-match accuracy
Scale: 0-100
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-441190f58bf4. Source payload hash: 441190f58bf43ee1768f98ba21281cc3c97dbd93e50fdde45d9e816fc9137976. Parser version: curated-v1. Imported at: 2026-10-15

Source: Princeton HAL

τ-bench Airline Clean

26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools. The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.

Sub-task	GPT-4o mini (This model)	Claude Opus 4.7	Claude Opus 4.5	GPT-5.5	Gemini 3.1 Pro Preview	Gemini 3.5 Flash	Gemini 2.5 Flash	Gemini 2.5 Pro	o1
Consistency	0.74	0.88 (best value in this row)	0.84	0.84	0.81	0.78	0.75	0.74	0.71
Predictability	0.35	0.87 (best value in this row)	0.85	0.84	0.83	0.82	0.6	0.78	0.74
Reliability	0.67	0.89 (best value in this row)	0.88	0.89 (best value in this row)	0.86	0.86	0.72	0.81	0.81
Robustness	0.93	0.91	0.97	0.97	0.94	0.97	0.82	0.93	0.98 (best value in this row)
Safety	0.85	0.94	0.98	0.96	0.97	0.99 (best value in this row)	0.9	0.92	0.91

Unit: ratinghigher is betterAs of August 5, 2026Source: Princeton HALbars scaled within each row, not across rows

τ-bench Airline Clean

Accuracy

1.Claude Opus 4.784.6%

2.Gemini 3.1 Pro Preview82.1%

3.Claude Opus 4.580.8%

3.Gemini 3.5 Flash80.8%

5.GPT-5.579.5%

6.Gemini 2.5 Pro71.8%

7.o166.2%

8.Gemini 2.5 Flash59%

9.GPT-4o mini29.5%

higher is betterAs of August 5, 2026Scale: 0% - 100%

Source: Princeton HAL

gradually.ai

Methodology and context

Comparison cohort: hal-tau-airline-clean-hal-reliability-overall
Metric: task success rate
Scale: 0-100
Participants: 9
Methodology and context: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-d92ebee642b2. Source payload hash: d92ebee642b24d08cf3917332b6eb47c5e1332b475553443f7b181dd115c5b91. Parser version: curated-v1. Imported at: 2026-10-15

Source: Princeton HAL

Recorded measurement details

GAIA Reliability: Consistency (Show measurement, test conditions, and source)

Source value: 0.73
Score: 0.73
Metric: aggregate consistency
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-b9a95297d4ec. Source payload hash: b9a95297d4ece6abc36f31aac8cf976f92d4246a3ed4b99cf2e842f06b58c87b. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Predictability (Show measurement, test conditions, and source)

Source value: 0.58
Score: 0.58
Metric: aggregate predictability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-7b241e589f41. Source payload hash: 7b241e589f419187e043f7adb2ced034d2473c897385c649ba050b1bdb0b3229. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Reliability (Show measurement, test conditions, and source)

Source value: 0.76
Score: 0.76
Metric: aggregate reliability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-da0b6b1749b6. Source payload hash: da0b6b1749b66a7ae012a8ccbd11b9177f90c132f5afacfcc96b8dd9297396ea. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Robustness (Show measurement, test conditions, and source)

Source value: 0.97
Score: 0.97
Metric: aggregate robustness
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-bf43b4155a46. Source payload hash: bf43b4155a4669508574513fe077bd40298dd1e3bb98afa4ec694839481574a1. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Safety (Show measurement, test conditions, and source)

Source value: 1
Score: 1
Metric: aggregate safety
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: safety
Task label: Safety
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-safety
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ecc218e3d73e. Source payload hash: ecc218e3d73e8416cb96ec0b8cf34ce2c4d21f205c4ab9d1229d78121c4725e0. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Accuracy (Show measurement, test conditions, and source)

Source value: 26.3
Score: 26.3
Metric: exact-match accuracy
Unit: %
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-441190f58bf4. Source payload hash: 441190f58bf43ee1768f98ba21281cc3c97dbd93e50fdde45d9e816fc9137976. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

τ-bench Airline Clean: Consistency (Show measurement, test conditions, and source)

Source value: 0.74
Score: 0.74
Metric: aggregate consistency
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-8400878e788c. Source payload hash: 8400878e788c6664ed84932f7fca367058ab95c3e531fcd1c51e93e57a4dd233. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Predictability (Show measurement, test conditions, and source)

Source value: 0.35
Score: 0.35
Metric: aggregate predictability
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-c35619e391d3. Source payload hash: c35619e391d36c7d2812660952978da8fb1f54d02512d7c7757a4361611e6382. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Reliability (Show measurement, test conditions, and source)

Source value: 0.67
Score: 0.67
Metric: aggregate reliability
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-1269bcbf9a8c. Source payload hash: 1269bcbf9a8c4ef53772153b789f62ea2ad09834c781605a213cad2df6524c81. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Robustness (Show measurement, test conditions, and source)

Source value: 0.93
Score: 0.93
Metric: aggregate robustness
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-6d10b4cb6839. Source payload hash: 6d10b4cb68392cf777aaac6a06fee2bddff25e9ee793486587e866d09158b5ac. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Safety (Show measurement, test conditions, and source)

Source value: 0.85
Score: 0.85
Metric: aggregate safety
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: safety
Task label: Safety
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-safety
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-c133a6448b92. Source payload hash: c133a6448b92ae287d22a270b7228dc9c155982dd9f47c2e15616b61d2f94df3. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Accuracy (Show measurement, test conditions, and source)

Source value: 29.5
Score: 29.5
Metric: task success rate
Unit: %
Benchmark version: 26-task cleaned subset
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gpt-4o-mini. Source model name: GPT-4o mini. Provider reported by source: OpenAI. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-d92ebee642b2. Source payload hash: d92ebee642b24d08cf3917332b6eb47c5e1332b475553443f7b181dd115c5b91. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

Page 1 of 8

Predecessor and successor

Sources and data date

Type	Evidence and data date
Speed	OpenAI. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing (retrieved September 8, 2026) · GPT-4o mini API model documentation · Editorial description reviewed October 2, 2026
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · API pricing (retrieved September 8, 2026) · GPT-4o mini API model documentation · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · API pricing (retrieved September 8, 2026) · GPT-4o mini API model documentation · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording OpenAI does not publish parameter counts.
Not found in the inspected sources	independent-benchmarks Original wording This pass retained the exact OpenAI launch-page results; no independent evaluator row was merged.
Additional source	Context window data
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Princeton HAL (retrieved August 6, 2026)
Additional source	Princeton HAL (retrieved August 5, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	OpenAI (retrieved September 8, 2026) · GPT-4o mini: advancing cost-efficient intelligence · Editorial description reviewed October 2, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0
Additional source	Introducing GPT-4.1 in the API (2026-09-19)

Specifications and access

Overall and area scores

Overall index

Coding

Reasoning

Mathematics

Knowledge

Agents

General

Finance

Health

Multimodal

Test family: IFBench

Test family: GPQA Diamond

Test family: GAIA Reliability

Test family: τ-bench Airline Clean

Test family: MMMU-Pro

Test family: CorpFin v2

Test family: LiveCodeBench v6

Test family: MATH 500

Test family: MedQA

Test family: MGSM

Test family: MMLU-Pro

Test family: MortgageTax

Test family: TaxEval v2

Benchmark sources at a glance

Published benchmarks

Agent tasks

GAIA Reliability

GAIA Reliability

τ-bench Airline Clean

τ-bench Airline Clean

Recorded measurement details

Our tests

SVG short test

GPT-4o mini

Pelican on a bicycle

Popularity

Published prices

Predecessor and successor

Sources and data date