Language modelActive

Claude Opus 4.5

Anthropic

Released: November 2025
Data date: September 8, 2026

Intelligence: Not ranked
Overall score
Speed: 48.1 tokens/s
Output throughput
Price: $5 · $25
Input · Output / 1M tokens
Input: Text, Image
Output: Text

No published category-wide ranking is available for this model.

Entries 1-12 of 46. Page 1 of 4.

Published benchmarks

Each chart compares models in the same documented test and highlights the current model.

Agent tasks

6 measurements

BFCL v4

Overall

What this benchmark measures: Berkeley's function-calling test combines single and multi-turn tool calls with web search, memory, and hallucination measurement.

Context: The overall score is an unweighted mix of subcategories and depends on native function calling or prompt workarounds.

Higher is better2 modelsClaude Opus 4.5: Rank 1 of 2Data date: September 2, 2026

1.Claude Opus 4.577.47%

2.Claude Sonnet 4.573.24%

Scale: 0% - 100%

Source: Berkeley BFCL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: bfcl-v4-f7cf735-overall
Metric: unweighted overall accuracy
Scale: 0-100
Participants: 2
Methodology and context: Unweighted overall score across agentic, multi-turn, and single-turn function-calling tests.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Model snapshot: claude-opus-4-5-20251101-fc. Evaluator: Berkeley BFCL. Dataset type: public. Leaderboard date: 2026-04-12. Source ID: berkeley-bfcl. Evidence capture ID: curated-berkeley-bfcl-2026-09-02-8f6a222c9fa4. Source payload hash: 8f6a222c9fa497de3d8ea8610998d9e80881ac37f553de6987bb056de8e691d4. Parser version: curated-v1. Imported at: 2026-09-04

Source: Berkeley BFCL

GAIA Reliability

Consistency

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsClaude Opus 4.5: Rank 1 of 9Data date: August 6, 2026

1.Claude Opus 4.50.81 rating

1.Claude Opus 4.70.81 rating

3.Gemini 3.1 Pro Preview0.76 rating

4.Gemini 3.5 Flash0.75 rating

5.GPT-4o mini0.73 rating

6.Gemini 2.5 Pro0.72 rating

7.Gemini 2.5 Flash0.7 rating

8.o10.68 rating

9.GPT-5.50.61 rating

Scale: 0 rating - 0.81 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Metric: aggregate consistency
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-6dc9a7372aa5. Source payload hash: 6dc9a7372aa57246baa347fe9a721357c12568b43a736c4f29267d3e8099e829. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Accuracy

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsClaude Opus 4.5: Rank 4 of 9Data date: August 6, 2026

1.Gemini 3.5 Flash79.2%

2.Gemini 3.1 Pro Preview76.2%

3.Claude Opus 4.773.3%

4.Claude Opus 4.568.5%

5.GPT-5.562.8%

6.Gemini 2.5 Pro52.5%

7.Gemini 2.5 Flash46.7%

8.o134.4%

9.GPT-4o mini26.3%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Metric: exact-match accuracy
Scale: 0-100
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c9e8d3dda73e. Source payload hash: c9e8d3dda73e7db4cf1f7f27fd17b5b9cf99cd306a2010121415045b692de2d0. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Predictability

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsClaude Opus 4.5: Rank 1 of 9Data date: August 6, 2026

1.Claude Opus 4.50.84 rating

2.Gemini 3.5 Flash0.8 rating

2.GPT-5.50.8 rating

4.Gemini 3.1 Pro Preview0.78 rating

5.Claude Opus 4.70.73 rating

6.Gemini 2.5 Pro0.71 rating

7.o10.68 rating

8.Gemini 2.5 Flash0.64 rating

9.GPT-4o mini0.58 rating

Scale: 0 rating - 0.84 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Metric: aggregate predictability
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-0810969a2419. Source payload hash: 0810969a241953f49cf21d98e373922e0582ee7a48723f0134230fefcc05296e. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Reliability

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsClaude Opus 4.5: Rank 1 of 9Data date: August 6, 2026

1.Claude Opus 4.50.85 rating

2.Claude Opus 4.70.84 rating

2.Gemini 3.5 Flash0.84 rating

4.Gemini 3.1 Pro Preview0.82 rating

5.GPT-5.50.79 rating

5.o10.79 rating

7.Gemini 2.5 Pro0.78 rating

8.GPT-4o mini0.76 rating

9.Gemini 2.5 Flash0.74 rating

Scale: 0 rating - 0.85 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Metric: aggregate reliability
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-3b3c7e95ef29. Source payload hash: 3b3c7e95ef2907a9449e379b4710e314ca2e332ff72478241e313050b8046c1d. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Robustness

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsClaude Opus 4.5: Rank 7 of 9Data date: August 6, 2026

1.o11 rating

2.Claude Opus 4.70.99 rating

3.GPT-4o mini0.97 rating

4.Gemini 3.5 Flash0.96 rating

5.GPT-5.50.95 rating

6.Gemini 3.1 Pro Preview0.94 rating

7.Claude Opus 4.50.91 rating

7.Gemini 2.5 Pro0.91 rating

9.Gemini 2.5 Flash0.87 rating

Scale: 0 rating - 1 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Metric: aggregate robustness
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-0bf85b7d6298. Source payload hash: 0bf85b7d6298735dacaf396cff511bab2654394cf3860535e6b12966bba882a2. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

Knowledge

5 measurements

FACTS Benchmark Suite

Document grounding

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsClaude Opus 4.5: Rank 3 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro74.2%

2.GPT-569.6%

3.Claude Opus 4.562.1%

4.GPT-4.145.6%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:grounding
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-6870a864ed52. Source payload hash: 6870a864ed520fe945d72701b2edd1d36f281604f98ca462d8238b0b98b87b1b. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Multimodal factuality

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsClaude Opus 4.5: Rank 4 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro46.9%

2.GPT-544.1%

3.GPT-4.140.1%

4.Claude Opus 4.539.2%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:multimodal
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-19f1c1a4d1b3. Source payload hash: 19f1c1a4d1b3b8b81bec5b3e0a4aed7ae7e1d1e9680a4ee8bb86285e4eef4773. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

FACTS overall score

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsClaude Opus 4.5: Rank 3 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro62.1%

2.GPT-561.8%

3.Claude Opus 4.551.3%

4.GPT-4.150.5%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:overall
Metric: FACTS score
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-2c437e957aba. Source payload hash: 2c437e957aba94ab458935bcae2984b84e3bc4b502bc492175dcdf274f94f4a8. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Parametric knowledge

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsClaude Opus 4.5: Rank 4 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro63.2%

2.GPT-555.8%

3.GPT-4.151.5%

4.Claude Opus 4.530.6%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:parametric
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-d96cd61e22ee. Source payload hash: d96cd61e22eeebd3c5b3b4ef95c92d1d01097631696428ff7b2532fb8c07d34c. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Factuality with search

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsClaude Opus 4.5: Rank 2 of 4Data date: August 9, 2026

1.GPT-577.7%

2.Claude Opus 4.573.2%

3.GPT-4.164.6%

4.Gemini 2.5 Pro63.9%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:search
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-587f77440d38. Source payload hash: 587f77440d38dbca82dd2fe811a20538a5a2757c30484e14b0d894fa0ff58fa0. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

CORE-Bench Hard

Automatic validation

What this benchmark measures: 45 public tasks for reproducing research results with code, data, and dependencies.

Context: Automatic and manually validated scores are different metrics. The agent scaffold strongly affects the score.

Higher is better2 modelsClaude Opus 4.5: Rank 1 of 2Data date: August 6, 2026

1.Claude Opus 4.577.78%

2.Claude Sonnet 4.562.22%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-corebench-hard-claude-code-automatic
Metric: automatic accuracy
Scale: 0-100
Participants: 2
Methodology and context: Claude Code on 45 public paper reproductions. Total cost is divided by 45.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: Claude Code. Agent: Claude Code. Cost per task in USD: 1.9368888888888889. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-a1a31dd8d645. Source payload hash: a1a31dd8d6455bb089695d0099ec1a57f1633370c20f0cf91962fe7dfb06ae33. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

Recorded measurement details

Every observation retains its source value and published test conditions.

BFCL v4: Overall (Show measurement, test conditions, and source)

Source value: 77.47
Score: 77.47
Metric: unweighted overall accuracy
Unit: %
Benchmark version: v4, commit f7cf735
Category: agentic
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: bfcl-v4-f7cf735-overall
Source type: official-leaderboard
Evaluator: Berkeley BFCL
Status: active
Retrieved at: 2026-09-02
Methodology: Unweighted overall score across agentic, multi-turn, and single-turn function-calling tests.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Model snapshot: claude-opus-4-5-20251101-fc. Evaluator: Berkeley BFCL. Dataset type: public. Leaderboard date: 2026-04-12. Source ID: berkeley-bfcl. Evidence capture ID: curated-berkeley-bfcl-2026-09-02-8f6a222c9fa4. Source payload hash: 8f6a222c9fa497de3d8ea8610998d9e80881ac37f553de6987bb056de8e691d4. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Unweighted overall score across agentic, multi-turn, and single-turn function-calling tests.

Source Berkeley BFCL

FACTS Benchmark Suite: Document grounding (Show measurement, test conditions, and source)

Source value: 62.1
Score: 62.1
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: grounding
Task label: Document grounding
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:grounding
Confidence interval: 3.3
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-6870a864ed52. Source payload hash: 6870a864ed520fe945d72701b2edd1d36f281604f98ca462d8238b0b98b87b1b. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Multimodal factuality (Show measurement, test conditions, and source)

Source value: 39.2
Score: 39.2
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: multimodal
Task label: Multimodal factuality
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:multimodal
Confidence interval: 3.5
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-19f1c1a4d1b3. Source payload hash: 19f1c1a4d1b3b8b81bec5b3e0a4aed7ae7e1d1e9680a4ee8bb86285e4eef4773. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: FACTS overall score (Show measurement, test conditions, and source)

Source value: 51.3
Score: 51.3
Metric: FACTS score
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: overall
Task label: FACTS overall score
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:overall
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-2c437e957aba. Source payload hash: 2c437e957aba94ab458935bcae2984b84e3bc4b502bc492175dcdf274f94f4a8. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Parametric knowledge (Show measurement, test conditions, and source)

Source value: 30.6
Score: 30.6
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: parametric
Task label: Parametric knowledge
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:parametric
Confidence interval: 2
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-d96cd61e22ee. Source payload hash: d96cd61e22eeebd3c5b3b4ef95c92d1d01097631696428ff7b2532fb8c07d34c. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Factuality with search (Show measurement, test conditions, and source)

Source value: 73.2
Score: 73.2
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: search
Task label: Factuality with search
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:search
Confidence interval: 1.7
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-587f77440d38. Source payload hash: 587f77440d38dbca82dd2fe811a20538a5a2757c30484e14b0d894fa0ff58fa0. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

CORE-Bench Hard: Automatic validation (Show measurement, test conditions, and source)

Source value: 77.78
Score: 77.78
Metric: automatic accuracy
Unit: %
Benchmark version: 45-paper public hard set
Category: coding
Task: automatic
Task label: Automatic validation
Direction: Higher is better
Comparison cohort: hal-corebench-hard-claude-code-automatic
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: Claude Code on 45 public paper reproductions. Total cost is divided by 45.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: Claude Code. Agent: Claude Code. Cost per task in USD: 1.9368888888888889. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-a1a31dd8d645. Source payload hash: a1a31dd8d6455bb089695d0099ec1a57f1633370c20f0cf91962fe7dfb06ae33. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Claude Code on 45 public paper reproductions. Total cost is divided by 45.

Source Princeton HAL

GAIA Reliability: Consistency (Show measurement, test conditions, and source)

Source value: 0.81
Score: 0.81
Metric: aggregate consistency
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-6dc9a7372aa5. Source payload hash: 6dc9a7372aa57246baa347fe9a721357c12568b43a736c4f29267d3e8099e829. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Accuracy (Show measurement, test conditions, and source)

Source value: 68.5
Score: 68.5
Metric: exact-match accuracy
Unit: %
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c9e8d3dda73e. Source payload hash: c9e8d3dda73e7db4cf1f7f27fd17b5b9cf99cd306a2010121415045b692de2d0. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Predictability (Show measurement, test conditions, and source)

Source value: 0.84
Score: 0.84
Metric: aggregate predictability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-0810969a2419. Source payload hash: 0810969a241953f49cf21d98e373922e0582ee7a48723f0134230fefcc05296e. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Reliability (Show measurement, test conditions, and source)

Source value: 0.85
Score: 0.85
Metric: aggregate reliability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-3b3c7e95ef29. Source payload hash: 3b3c7e95ef2907a9449e379b4710e314ca2e332ff72478241e313050b8046c1d. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Robustness (Show measurement, test conditions, and source)

Source value: 0.91
Score: 0.91
Metric: aggregate robustness
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/claude-opus-4.5. Source model name: Claude Opus 4.5. Provider reported by source: Anthropic. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-0bf85b7d6298. Source payload hash: 0bf85b7d6298735dacaf396cff511bab2654394cf3860535e6b12966bba882a2. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

Our model tests

Community tests you can inspect

Compare how the models respond to the same prompt. Each test shows the first attempt, with no subsequent fixes to the generated code. These results do not contribute to an overall score.

Pelican on a bicycle

A community classic for free-form SVG drawing.

Claude Opus 4.5

Not tested yet

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

Claude Opus 4.5

Published benchmarks

Agent tasks

BFCL v4

GAIA Reliability

GAIA Reliability

GAIA Reliability

GAIA Reliability

GAIA Reliability

Knowledge

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

CORE-Bench Hard

Recorded measurement details

Community tests you can inspect

Pelican on a bicycle

Specifications and access

Additional source observations

Published prices

100 API requests

Compare this model

Sources and data date