Claude Opus 5
Unavailable in this round
Language modelActive
Claude Opus 5
Anthropic
- Released
- July 24, 2026
- Data date
- September 8, 2026
- Predecessor
- Claude Opus 4.8
- Successor
- Claude Opus 5.5
- Benchmark-Meta-Score
- Provisional, 68 % panel coverage
- Speed
- 54.6 tokens/sOutput throughput
- Price
- Input · Output / 1M tokens
- Input
- Text, Image
- Output
- Text
At launch, Claude Opus 5 was the default model for Claude Max and the strongest model in Claude Pro. It followed Claude Opus 4.8 and, according to Anthropic, brought much of Claude Fable 5’s capability into a generally available Opus model.
The provider reported more than twice Opus 4.8’s Frontier-Bench v0.1 result and a 0.5% gap to Fable 5 on CursorBench 3.2 at maximum effort. These are vendor comparisons. For controlled cybersecurity work, Claude Mythos 5 remained the more specialized option.
- Position
- Rank 3 of 24
- Index score
- 78.8 / 100
Methodology and source
The LLM comparison hub's selected-panel index averages rank percentiles from 10 documented comparison cohorts. Benchmark meta-scores are shown in the benchmark section.
Benchmark meta-analysis
Overall and area scores
These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.
Retrieval dates of the sources used 2026-10-15 - 2026-10-19
Overall index
Index 81.7 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 68 %
- Test families
- 46 / 71
- Sources
- 4
- Mathematical bounds with missing values
- 55.6-87.6 / 100
Coding
Index 89.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 62 %
- Test families
- 10 / 15
- Sources
- 3
- Mathematical bounds with missing values
- 55.5-93.3 / 100
Reasoning
Index 70.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 64 %
- Test families
- 5 / 7
- Sources
- 3
- Mathematical bounds with missing values
- 45.1-80.9 / 100
Mathematics
Index 77.1 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 60 %
- Test families
- 3 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 46.3-86.3 / 100
Knowledge
Index 97.5 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 67 %
- Test families
- 2 / 3
- Sources
- 2
- Mathematical bounds with missing values
- 65.0-98.3 / 100
Agents
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 42 %
- Test families
- 5 / 12
- Sources
- 2
- Mathematical bounds with missing values
- 36.6-95.0 / 100
General
Index 78.3 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 80 %
- Test families
- 4 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 62.7-82.7 / 100
Finance
Index 93.6 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 86 %
- Test families
- 6 / 7
- Sources
- 1
- Mathematical bounds with missing values
- 80.2-94.5 / 100
Legal
Index 87.3 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 75 %
- Test families
- 3 / 4
- Sources
- 1
- Mathematical bounds with missing values
- 65.5-90.5 / 100
Health
Index 97.8 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 67 %
- Test families
- 2 / 3
- Sources
- 1
- Mathematical bounds with missing values
- 65.2-98.5 / 100
Cybersecurity
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 1 / 1
- Sources
- 1
- Mathematical bounds with missing values
- 42.4-42.4 / 100
Multimodal
Index 87.7 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 2 / 4
- Sources
- 3
- Mathematical bounds with missing values
- 43.8-93.8 / 100
Search
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 1 / 1
- Sources
- 1
- Mathematical bounds with missing values
- 84.6-84.6 / 100
Productivity
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 1 / 2
- Sources
- 1
- Mathematical bounds with missing values
- 50.0-100.0 / 100
Education
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 1 / 2
- Sources
- 1
- Mathematical bounds with missing values
- 35.3-85.3 / 100
Documented method · Included test families and measurements
Method version: benchmark-meta-score-v1
The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture; when dates match, the wider slice is used. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. When selecting between multiple results, unknown or ambiguous tiers exclude the model from that series. Unambiguous single configurations do not require a tier ordering. The best individual result for each model is not selected. Equal raw values share the same rank percentile. Multiple series from the same evaluator within a family receive equal weight; evaluators within a family, families within an area, and areas in the overall index then receive equal weight. Unmeasured series remain visible in coverage and bounds. An overall score requires at least four areas, six test families, and two sources. An area score requires at least two test families. Every overall and area score also requires at least 50% of its panel weight.
Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.
Unmeasured portions are not entered as zero. They reduce coverage and contribute only to the displayed bounds. Scores with different coverage are therefore not equally complete.
The bounds hold observed percentiles fixed and allow missing contributions anywhere from 0 to 100. They are not confidence intervals or predictions of future recalculations.
The source count is not a count of independent experiments.
OECD handbook on composite indicatorsAll 46 test families and 12 of 52 raw measurements are listed here. The complete calculation for every model is available in the JSON export.
Test family: AIME
Mathematics · Family index: 61.4 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-otis-mock-aime-2024-2025
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.43 %
- Raw value:
- 0.9888888888888889 ratio
- Metric:
- model_extracted_exact_match
- Version:
- 1.1.6
- Participating models:
- 23
- Rank percentile within the cohort:
- 61.4 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: claude-opus-5_max; Epoch model label: Claude Opus 5; Display label field: Claude Opus 5 (max); Reasoning setting encoded in the alias: max. Task ID: OTIS Mock AIME 2024-2025. Task version: 1.1.6. Run ID: KxvHAT9SV2dTDn7jBnTk6f. Started at 2026-07-24T18:40:39.000Z. Original task label: OTIS Mock AIME 2024-2025. The published row reports mean_score=0.9888888888888889 and stderr=0.011111111111111112. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: AA-AnalystAgent
Agents · Family index: 88.2 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-analyst-agent
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 53.75 %
- Metric:
- pass^5 reliability
- Version:
- AA cohort: 80 questions
- Participating models:
- 39
- Rank percentile within the cohort:
- 88.2 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-analyst-agent:AA cohort: 80 questions:standalone:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-analyst-agent","AA cohort: 80 questions","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup analyst agent
- Tool policy:
- 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission
- Attempts:
- pass^5, 5 independent attempts per question
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 53.8%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Briefcase
Agents · Family index: 98.8 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-briefcase
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,659.48 Elo
- Metric:
- AA-Briefcase Elo
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 98.8 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-briefcase:v1.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-briefcase","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-LCR
Reasoning · Family index: 58.5 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-lcr
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 79.3333333333333 %
- Metric:
- Pass rate
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 58.5 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-lcr:v1.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-lcr","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 79.3%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AutomationBench-AA
Agents · Family index: 75.6 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-automationbench-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 56.57355576493251 %
- Metric:
- Objectives completed without guardrail violations
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 75.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-automationbench-aa:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-automationbench-aa","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 56.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDP.pdf
Reasoning · Family index: 81.7 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdp-pdf
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 21.6 %
- Metric:
- All-pass rate
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 81.7 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdp-pdf:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdp-pdf","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 21.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDPval-AA
Agents · Family index: 97.6 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdpval-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,721.15 Elo
- Metric:
- GDPval-AA Elo
- Version:
- v2.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 97.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdpval-aa:v2.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdpval-aa","v2.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Omniscience
Knowledge · Family index: 96.3 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-omniscience
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 2.38 %
- Raw value:
- 37.06666666666667 points
- Metric:
- AA-Omniscience Index
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 96.3 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-omniscience:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-omniscience","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: Claude Opus 5 (Max) (release claude-opus-5, row claude-opus-5, UUID b8fc61f7-5e9a-49e6-8547-6ac56db24627, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/b8fc61f7-5e9a-49e6-8547-6ac56db24627/claude-opus-5/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Model snapshot: Claude Opus 5 (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: Chess Puzzles
Reasoning · Family index: 47.6 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-chess-puzzles
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 0.43 ratio
- Metric:
- exact
- Version:
- 2.0.0
- Participating models:
- 22
- Rank percentile within the cohort:
- 47.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: claude-opus-5_max; Epoch model label: Claude Opus 5; Display label field: Claude Opus 5 (max); Reasoning setting encoded in the alias: max. Task ID: Chess Puzzles. Task version: 2.0.0. Run ID: RLN98bmQSMnc5Zv39BF4M2. Started at 2026-10-13T13:19:00.000Z. Original task label: Chess Puzzles. The published row reports mean_score=0.43 and stderr=0.0497569851956243. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: FrontierMath
Mathematics · Family index: 80.3 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-frontiermath-tier-4-v2-private
- Series weight within the test family:
- 50 %
- Nominal contribution to the overall panel:
- 0.71 %
- Raw value:
- 0.7317073170731707 ratio
- Metric:
- verification_code
- Version:
- 2.0.0
- Participating models:
- 19
- Rank percentile within the cohort:
- 83.3 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["FrontierMath-Tier-4-v2-Private","FrontierMath-Tier-4-v2-Private","2.0.0","verification_code"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["FrontierMath-Tier-4-v2-Private","FrontierMath-Tier-4-v2-Private","2.0.0","verification_code"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: claude-opus-5_max; Epoch model label: Claude Opus 5; Display label field: Claude Opus 5 (max); Reasoning setting encoded in the alias: max. Task ID: FrontierMath-Tier-4-v2-Private. Task version: 2.0.0. Run ID: ji3kMn49Jcr9t4whxcpjj4. Started at 2026-07-24T19:54:20.000Z. Original task label: FrontierMath-Tier-4-v2-Private. The published row reports mean_score=0.7317073170731707 and stderr=0.07005564203095158. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max. - Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-frontiermath-tiers-1-3-v2-private
- Series weight within the test family:
- 50 %
- Nominal contribution to the overall panel:
- 0.71 %
- Raw value:
- 0.856140350877193 ratio
- Metric:
- verification_code
- Version:
- 2.0.0
- Participating models:
- 23
- Rank percentile within the cohort:
- 77.3 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["FrontierMath-Tiers-1-3-v2-Private","FrontierMath-Tiers-1-3-v2-Private","2.0.0","verification_code"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["FrontierMath-Tiers-1-3-v2-Private","FrontierMath-Tiers-1-3-v2-Private","2.0.0","verification_code"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: claude-opus-5_max; Epoch model label: Claude Opus 5; Display label field: Claude Opus 5 (max); Reasoning setting encoded in the alias: max. Task ID: FrontierMath-Tiers-1-3-v2-Private. Task version: 2.0.0. Run ID: ff7GeS7rdperjxSBZ7MHZY. Started at 2026-07-24T19:54:20.000Z. Original task label: FrontierMath-Tiers-1-3-v2-Private. The published row reports mean_score=0.856140350877193 and stderr=0.020824894575343943. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: GPQA Diamond
Reasoning · Family index: 78.7 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-gpqa-diamond
- Series weight within the test family:
- 50 %
- Nominal contribution to the overall panel:
- 0.51 %
- Raw value:
- 0.9387626262626263 ratio
- Metric:
- choice
- Version:
- 1.0.11
- Participating models:
- 22
- Rank percentile within the cohort:
- 66.7 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["GPQA diamond","GPQA diamond","1.0.11","choice"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["GPQA diamond","GPQA diamond","1.0.11","choice"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: claude-opus-5_max; Epoch model label: Claude Opus 5; Display label field: Claude Opus 5 (max); Reasoning setting encoded in the alias: max. Task ID: GPQA diamond. Task version: 1.0.11. Run ID: D3mtasTumfvjR4fjmnHWRy. Started at 2026-07-24T18:40:39.000Z. Original task label: GPQA diamond. The published row reports mean_score=0.9387626262626263 and stderr=0.014782015056089346. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: Humanity’s Last Exam
Reasoning · Family index: 98.8 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
Test family: LMArena Code Arena WebDev
Coding · Family index: 86.1 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LMArena Search
Search · Family index: 84.6 / 100Nominal family weight in the overall panel: 7.14 % · Coverage of this test family: 100 %
Test family: LMArena Text, Style Control
General · Family index: 83.3 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: LMArena Vision, Style Control
Multimodal · Family index: 80.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MMMU-Pro
Multimodal · Family index: 95.4 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: SWE-bench Verified
Coding · Family index: 100.0 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 50 %
Test family: Terminal-Bench
Coding · Family index: 92.4 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 83 %
Test family: Terminal-Bench Science
Coding · Family index: 80.0 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: BioMysteryBench
General · Family index: 84.2 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: Code Migration
Coding · Family index: 90.3 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: CorpFin v2
Finance · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: CyberBench v1.1
Cybersecurity · Family index: 42.4 / 100Nominal family weight in the overall panel: 7.14 % · Coverage of this test family: 100 %
Test family: EMB
Finance · Family index: 93.2 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Finance Agent (v2)
Finance · Family index: 91.7 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: IOI
Coding · Family index: 81.8 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LiveCodeBench v6
Coding · Family index: 96.1 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Harvey's Legal Agent Benchmark
Legal · Family index: 70.5 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: LegalBench
Legal · Family index: 92.3 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: Legal Research Bench
Legal · Family index: 99.2 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MedCode
Health · Family index: 100.0 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MedScribe
Health · Family index: 95.6 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MMLU-Pro
Knowledge · Family index: 98.7 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MortgageTax
Finance · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: MysteryMechanism
General · Family index: 75.0 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: ProgramBench
Coding · Family index: 90.8 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: ProofBench
Mathematics · Family index: 89.7 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: SAGE
Education · Family index: 70.7 / 100Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: SkillsBench
Agents · Family index: 79.3 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
Test family: Public Benefits Bench v1.1
Productivity · Family index: 100.0 / 100Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: SRE Bench
General · Family index: 70.8 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: Tax Agent Bench
Finance · Family index: 96.4 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Family index: 80.3 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench 1-100
Coding · Family index: 90.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Family index: 90.3 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Benchmark sources at a glance
Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.
| Evaluator | Test series | Composite indices | Recorded active measurements |
|---|---|---|---|
| Artificial Analysis | 11 | 1 | 62 |
| Vals AI | 34 | 2 | 230 |
| LMArena | 4 | 0 | 76 |
| Epoch AI | 6 | 0 | 11 |
| ARC Prize | 1 | 0 | 1 |
Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.
Page 1 of 35
Published benchmarks
General tasks
AlmanBench v0.1
Accepted cases
1,029 German tasks test conversion from standard German into a defined simplified-language specification. It measures the Alman specification, not general German writing quality.
higher is betterAs of August 9, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- almanbench-v0-1:official:overall
- Metric
- accepted cases
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- 1,029 public tasks with one direct run per row. The target is the Alman language specification.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Alman Institut. Dataset type: public. Leaderboard date: 2026-08-05. Reasoning effort: max. Standard error: 0.682851. Source ID: alman-institut. Evidence capture ID: curated-alman-institut-2026-08-09-dae32c26388f. Source payload hash: dae32c26388f764c4290ce78ea52d53e2d999561717fbbda2fafb71ab61604bc. Parser version: curated-v1. Imported at: 2026-10-15
BioMysteryBench
| Sub-task | Claude Opus 5 (This model) | Claude Opus 5.5 | Claude Sonnet 5.5 |
|---|---|---|---|
| Hard | 43.14% | 47.06% (best value in this row) | 45.1% |
| Human-solvable | 87.67% | 86.76% | 89.5% (best value in this row) |
| Overall | 79.26% | 79.26% | 81.11% (best value in this row) |
higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows
Methodology and context
- Metric
- accuracy
SimpleQA Verified v2
Kaggle score
1,000 verified short-answer prompts test factual knowledge without search or other tools. It does not measure document grounding, web research, or a RAG system.
higher is betterAs of August 9, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- simpleqa-verified-v2:official:overall
- Metric
- Kaggle leaderboard score
- Scale
- 0-100
- Participants
- 7
- Methodology and context
- 1,000 verified prompts without tools. Kaggle independently reproduced the results.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Kaggle. Dataset type: public. Leaderboard date: 2026-08-03. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-c9e07857b2b7. Source payload hash: c9e07857b2b7f674a2630039744be359b3da134b3edfe1d58547169b7a6a4a69. Parser version: curated-v1. Imported at: 2026-10-15
Agent tasks
Taste-Bench
Overall
502 technical decision points from engineering and research workflows. Each choice is presented in both answer orders. The overall score averages engineering and research and requires both answer orders to be correct. Unparseable responses count as wrong. Reasoning settings and output budget are part of the setup.
Note (Claude Opus 5): 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
1.GPT-5.6 Sol59.7%
Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
2.GPT-5.559.5%
Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
4.Grok 4.554.6%
Note: 10 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
5.GPT-5.6 Terra54%
Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
6.GLM-5.253.9%
Note: 17 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
7.Claude Sonnet 551.6%
Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
8.GPT-5.6 Luna49%
Note: 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
9.MiniMax M345.3%
Note: 3 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
10.DeepSeek-V4-Flash43.3%
Note: 2 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
11.GPT-5.4 mini40.1%
Note: 112 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
12.Mistral Medium 3.537.7%
Note: 24 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
13.GPT-5.4 nano36.6%
Note: 93 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
14.Grok 4.20 Reasoning15.7%
Note: 459 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
higher is betterAs of September 26, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- taste-bench-paper-v2-502
- Metric
- both-orders-correct, engineering/research macro average
- Scale
- 0-100
- Participants
- 14
- Methodology and context
- Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-4b814a15c9d8. Source payload hash: 4b814a15c9d86735eebaab2abbe0870cf5e70fb9dbcf2daad8953ba9982f1ae3. Parser version: curated-v1. Imported at: 2026-10-15
Terminal-Bench 4.0
Agentic evaluation of tasks in reproducible terminal environments. The agent, tools, environment, and reasoning level are part of the result. Scores from older Terminal-Bench versions are not directly comparable.
| Sub-task | Claude Opus 5 (This model) | Claude Fable 5.1 |
|---|---|---|
| Overall | 50.3% | 54.55% (best value in this row) |
| Overall | 53.94% | 57.88% (best value in this row) |
higher is betterAs of October 15, 2026Source: Terminal-Benchbars scaled within each row, not across rows
Note (Overall): Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
Note (Overall): Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
Methodology and context
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Metric
- trial pass rate
Terminal-Bench 4.0
Overall
Agentic evaluation of tasks in reproducible terminal environments. The agent, tools, environment, and reasoning level are part of the result. Scores from older Terminal-Bench versions are not directly comparable.
Note (Claude Opus 5): Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
1.Claude Opus 5.564.85%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
2.Claude Sonnet 5.561.82%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
3.Claude Fable 5.157.88%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
5.Claude Fable 544.55%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
6.Claude Opus 4.823.64%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
7.Claude Sonnet 512.42%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Metric
- trial pass rate
- Scale
- 0-100
- Participants
- 7
- Methodology and context
- 330 graded trials, Claude Code agent, max reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: Claude Code. Attempts: 330 graded trials. Reasoning effort: max. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-2a2d33bd5e8c. Source payload hash: 2a2d33bd5e8cfa913ee83e3e372608d1b9b2c5111a91370f9459d34899d2ee6d. Parser version: curated-v1. Imported at: 2026-10-15
Coding
Code Migration
| Sub-task | Claude Opus 5 (This model) | Claude Sonnet 5.5 | Claude Opus 5.5 | Claude Fable 5 | Claude Fable 5.1 | Claude Opus 4.8 | Claude Sonnet 5 | Claude Opus 4.7 |
|---|---|---|---|---|---|---|---|---|
| CLI | 53.3% | 69.77% (best value in this row) | 65.53% | 60.09% | 52.81% | 40.14% | 36.33% | 35.17% |
| COBOL | 70% (best value in this row) | 70% (best value in this row) | 70% (best value in this row) | 40% | 60% | 68.57% | 68.57% | 70% (best value in this row) |
| Overall | 57.47% | 69.83% (best value in this row) | 66.65% | 55.07% | 54.61% | 47.25% | 44.39% | 43.88% |
higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows
Methodology and context
- Metric
- accuracy
Recorded measurement details
AlmanBench v0.1: Accepted cases (Show measurement, test conditions, and source)
- Source value
- 94.9465500486
- Score
- 94.9465500486
- Metric
- accepted cases
- Unit
- %
- Benchmark version
- v0.1, results commit f34ddbf
- Category
- general
- Task
- overall
- Task label
- Accepted cases
- Direction
- Higher is better
- Comparison cohort
- almanbench-v0-1:official:overall
- Source type
- independent-evaluator
- Evaluator
- Alman Institut
- Status
- active
- Retrieved at
- 2026-08-09
- Methodology
- 1,029 public tasks with one direct run per row. The target is the Alman language specification.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Alman Institut. Dataset type: public. Leaderboard date: 2026-08-05. Reasoning effort: max. Standard error: 0.682851. Source ID: alman-institut. Evidence capture ID: curated-alman-institut-2026-08-09-dae32c26388f. Source payload hash: dae32c26388f764c4290ce78ea52d53e2d999561717fbbda2fafb71ab61604bc. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 1,029 public tasks with one direct run per row. The target is the Alman language specification.
SimpleQA Verified v2: Kaggle score (Show measurement, test conditions, and source)
- Source value
- 63.3
- Score
- 63.3
- Metric
- Kaggle leaderboard score
- Unit
- %
- Benchmark version
- Kaggle leaderboard 2026-08-03
- Category
- knowledge
- Task
- overall
- Task label
- Kaggle score
- Direction
- Higher is better
- Comparison cohort
- simpleqa-verified-v2:official:overall
- Confidence interval
- 3.2
- Source type
- independent-evaluator
- Evaluator
- Kaggle
- Status
- active
- Retrieved at
- 2026-08-09
- Methodology
- 1,000 verified prompts without tools. Kaggle independently reproduced the results.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Kaggle. Dataset type: public. Leaderboard date: 2026-08-03. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-c9e07857b2b7. Source payload hash: c9e07857b2b7f674a2630039744be359b3da134b3edfe1d58547169b7a6a4a69. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 1,000 verified prompts without tools. Kaggle independently reproduced the results.
Taste-Bench: Overall (Show measurement, test conditions, and source)
- Source value
- 55.5
- Score
- 55.5
- Metric
- both-orders-correct, engineering/research macro average
- Unit
- %
- Benchmark version
- arxiv:2609.25804v2
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- taste-bench-paper-v2-502
- Source type
- independent-evaluator
- Evaluator
- Taste-Bench paper
- Status
- active
- Retrieved at
- 2026-09-26
- Methodology
- Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Taste-Bench paper. Dataset type: public. Leaderboard date: 2026-09-23. Attempts: 502 questions, two option orders each. Reasoning effort: default. Maximum output tokens: 65536. Source ID: taste-bench-paper. Evidence capture ID: curated-taste-bench-paper-2026-09-26-4b814a15c9d8. Source payload hash: 4b814a15c9d86735eebaab2abbe0870cf5e70fb9dbcf2daad8953ba9982f1ae3. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Same prompt for all models. Each of 502 questions is shown twice with the options reversed. Average of engineering and research success rates, requiring both orders to be correct. 65,536-token output budget; unparseable answers count as wrong.
- Context
- 0 of 1,004 responses were unparseable and counted as wrong. Not a Gradually test.
Terminal-Bench 4.0: Overall (Show measurement, test conditions, and source)
- Source value
- 50.3
- Score
- 50.3
- Metric
- trial pass rate
- Unit
- %
- Benchmark version
- 4.0
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Confidence interval
- 3.73
- Source type
- official-leaderboard
- Evaluator
- Terminal-Bench
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 330 graded trials, Claude Code agent, high reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: Claude Code. Attempts: 330 graded trials. Reasoning effort: high. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-865fc280366f. Source payload hash: 865fc280366f8ee3f063bc6b86853d1ef0570083b038756c724d7e704843f670. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 330 graded trials, Claude Code agent, high reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.
- Context
- Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
Terminal-Bench 4.0: Overall (Show measurement, test conditions, and source)
- Source value
- 53.94
- Score
- 53.94
- Metric
- trial pass rate
- Unit
- %
- Benchmark version
- 4.0
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Confidence interval
- 3.17
- Source type
- official-leaderboard
- Evaluator
- Terminal-Bench
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 330 graded trials, Claude Code agent, xhigh reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: Claude Code. Attempts: 330 graded trials. Reasoning effort: xhigh. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-0cf13774bc38. Source payload hash: 0cf13774bc384234c6d3ee86e7641036aacbed1de60e71bed826db24aaa27c21. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 330 graded trials, Claude Code agent, xhigh reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.
- Context
- Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
Terminal-Bench 4.0: Overall (Show measurement, test conditions, and source)
- Source value
- 51.82
- Score
- 51.82
- Metric
- trial pass rate
- Unit
- %
- Benchmark version
- 4.0
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Confidence interval
- 3.39
- Source type
- official-leaderboard
- Evaluator
- Terminal-Bench
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 330 graded trials, Claude Code agent, max reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: Claude Code. Attempts: 330 graded trials. Reasoning effort: max. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-2a2d33bd5e8c. Source payload hash: 2a2d33bd5e8cfa913ee83e3e372608d1b9b2c5111a91370f9459d34899d2ee6d. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 330 graded trials, Claude Code agent, max reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.
- Context
- Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
BioMysteryBench: Hard (Show measurement, test conditions, and source)
- Source value
- 43.137
- Score
- 43.137
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- human_difficult
- Task label
- Hard
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:human_difficult
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 2195.964. Standard error: 1.961. Cost per task in USD: 6.01311. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000
BioMysteryBench: Human-solvable (Show measurement, test conditions, and source)
- Source value
- 87.671
- Score
- 87.671
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- human_solvable
- Task label
- Human-solvable
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:human_solvable
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 2195.964. Standard error: 2.093. Cost per task in USD: 2.64447. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000
BioMysteryBench: Overall (Show measurement, test conditions, and source)
- Source value
- 79.259
- Score
- 79.259
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:overall
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 2195.964. Standard error: 2.062. Cost per task in USD: 3.292774. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Compute: max. Temperature: 1. Maximum output tokens: 128000
Code Migration: CLI (Show measurement, test conditions, and source)
- Source value
- 53.297
- Score
- 53.297
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- cli
- Task label
- CLI
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:cli
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 12019.037. Standard error: 2.837. Cost per task in USD: 74.846365. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Compute: max. Temperature: 1. Maximum output tokens: 128000
Code Migration: COBOL (Show measurement, test conditions, and source)
- Source value
- 70
- Score
- 70
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- cobol
- Task label
- COBOL
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:cobol
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 5017.804. Standard error: 15.275. Cost per task in USD: 17.499254. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Compute: max. Temperature: 1. Maximum output tokens: 128000
Code Migration: Overall (Show measurement, test conditions, and source)
- Source value
- 57.473
- Score
- 57.473
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:overall
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: anthropic/claude-opus-5. Source model name: Claude Opus 5. Provider reported by source: Anthropic. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Compute: max. Temperature: 1. Maximum output tokens: 128000. Latency in seconds: 10268.729. Standard error: 4.371. Cost per task in USD: 60.509587. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Compute: max. Temperature: 1. Maximum output tokens: 128000
Page 1 of 35
Our tests
SVG short test
One SVG prompt and fixed output settings. Captures are shown only with 0 reported reasoning tokens or documented lack of reasoning controls. Identical test conditions do not make the models' internal compute identical.
Earlier tests
Pelican on a bicycle
A community classic for free-form SVG drawing.
Claude Opus 5
Reasoning (requested): high
Current attempt
- Run date
- Oct 1, 2026
- API provider
- openrouter
- Requested API model
- anthropic/claude-opus-5
- API model reported in response
- anthropic/claude-opus-5
- Reasoning setting
- high
- Pinned endpoint
- Anthropic | anthropic/claude-opus-5-20260723
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Test protocol
- community-visual-tests-v1
- Technical checks
- Passed, not a quality rating
- Response time
- 63 s
Visually reviewed. This individual test does not establish overall model quality.
Earlier attempts (1)
- Sep 7, 2026API request failedReasoning setting: high
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)Popularity
62.8 / 100
Specifications and access
| Specification | Value and source |
|---|---|
| Weights | ProprietarySource |
| Access | APISource |
| Context window | 1,000,000 tokensSource |
| Knowledge cutoff | May 2026Source |
| Notes | Released July 24, 2026. Near-frontier intelligence at launch, close to Fable 5 at half the price ($5/$25). 1M context, 128K output, and a May 2026 knowledge cutoff. It became the new default on Claude Max and strongest model on Claude Pro at launch. It led Frontier-Bench v0.1, surpassed Fable 5 on OSWorld 2.0, came within 0.5% of Fable 5 on CursorBench 3.2, and trailed Mythos 5 on cybersecurity. Those results describe the pre-Fable-5.1 and pre-Mythos-5.1 cohort.Source |
| API model ID | claude-opus-5Source |
| Source status | activeSource |
| Release | 2026-07-24Source |
| Maximum output | 128,000 tokensSource |
| Input | Text, imagesSource |
| Output | TextSource |
| Reasoning mode | adaptive; default highSource |
| Knowledge cutoff | 2026-05Source |
| Tool calling | YesSource |
| Structured output | YesSource |