Language modelActive

Gemini 2.5 Pro

Google

Released: March 2025
Data date: September 8, 2026

Intelligence: Not ranked
Overall score
Speed: 131.3 tokens/s
Output throughput
Price: $1.25 · $10
Input · Output / 1M tokens
Input: Text, Image, Audio, Video
Output: Text

No published category-wide ranking is available for this model.

Entries 1-12 of 86. Page 1 of 8.

Published benchmarks

Each chart compares models in the same documented test and highlights the current model.

Aider Polyglot

Overall

What this benchmark measures: 225 code-editing tasks from Exercism across C++, Go, Java, JavaScript, Python, and Rust.

Context: Aider, the edit format, reasoning budget, and two attempts are part of the result.

Higher is better4 modelsGemini 2.5 Pro: Rank 2 of 4Data date: September 2, 2026

1.o3-pro84.9%

2.Gemini 2.5 Pro83.1%

3.o381.3%

4.DeepSeek-R171.4%

Scale: 0% - 100%

Source: Aider

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: aider-polyglot-pass2-diff
Metric: percent correct after two attempts
Scale: 0-100
Participants: 4
Methodology and context: 225 Exercism tasks across six languages using Aider and a documented edit format.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Model snapshot: gemini-2.5-pro-preview-2025-06-05. Evaluator: Aider. Dataset type: public. Leaderboard date: 2025-06-06. Attempts: pass 2. Reasoning effort: 32k thinking tokens. Latency in seconds: 200.3. Cost per task in USD: 0.22169866666666663. Source ID: aider. Evidence capture ID: curated-aider-2026-09-02-b94f7308e188. Source payload hash: b94f7308e188d90f6864b22fd13d3d933378668e9fc981a7655173a4c3545a4e. Parser version: curated-v1. Imported at: 2026-09-04

Source: Aider

Knowledge

5 measurements

FACTS Benchmark Suite

Document grounding

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsGemini 2.5 Pro: Rank 1 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro74.2%

2.GPT-569.6%

3.Claude Opus 4.562.1%

4.GPT-4.145.6%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:grounding
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-8a8b1ae4abae. Source payload hash: 8a8b1ae4abae2b077a109193965fb09bce92092c2be809bfae78d4cfa219dc27. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Multimodal factuality

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsGemini 2.5 Pro: Rank 1 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro46.9%

2.GPT-544.1%

3.GPT-4.140.1%

4.Claude Opus 4.539.2%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:multimodal
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-26c476d1beb7. Source payload hash: 26c476d1beb7b7772cf5b5a2e60452d78ae516f5f936f804b1cccfead5529b32. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

FACTS overall score

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsGemini 2.5 Pro: Rank 1 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro62.1%

2.GPT-561.8%

3.Claude Opus 4.551.3%

4.GPT-4.150.5%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:overall
Metric: FACTS score
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-f90e68e563fe. Source payload hash: f90e68e563fe354373076ca491cfa45c3d12df9ca257aa0eff56a191c63c53fa. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Parametric knowledge

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsGemini 2.5 Pro: Rank 1 of 4Data date: August 9, 2026

1.Gemini 2.5 Pro63.2%

2.GPT-555.8%

3.GPT-4.151.5%

4.Claude Opus 4.530.6%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:parametric
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-68c34aa514b7. Source payload hash: 68c34aa514b740c6f80ada1a284136c938a1f97cd1e65c0f87138497fc92708c. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

FACTS Benchmark Suite

Factuality with search

What this benchmark measures: 3,513 public examples test factuality with documents, images, parametric knowledge, and web search.

Context: The suite score averages four different capabilities. The component scores matter more for decisions.

Higher is better4 modelsGemini 2.5 Pro: Rank 4 of 4Data date: August 9, 2026

1.GPT-577.7%

2.Claude Opus 4.573.2%

3.GPT-4.164.6%

4.Gemini 2.5 Pro63.9%

Scale: 0% - 100%

Source: Google DeepMind

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: facts-suite-v2:official:search
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-36ddfd1bf586. Source payload hash: 36ddfd1bf586c681280b54cf509f9fbb4f8f41eeebb44d9a562d34aed96401d9. Parser version: curated-v1. Imported at: 2026-09-04

Source: Google DeepMind

Agent tasks

6 measurements

GAIA Reliability

Consistency

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 6 of 9Data date: August 6, 2026

1.Claude Opus 4.50.81 rating

1.Claude Opus 4.70.81 rating

3.Gemini 3.1 Pro Preview0.76 rating

4.Gemini 3.5 Flash0.75 rating

5.GPT-4o mini0.73 rating

6.Gemini 2.5 Pro0.72 rating

7.Gemini 2.5 Flash0.7 rating

8.o10.68 rating

9.GPT-5.50.61 rating

Scale: 0 rating - 0.81 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Metric: aggregate consistency
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c31d0b9d9ce4. Source payload hash: c31d0b9d9ce4ae3176dd4e9d861282248ef948babbb34bde532eb1ccbbc96780. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Accuracy

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 6 of 9Data date: August 6, 2026

1.Gemini 3.5 Flash79.2%

2.Gemini 3.1 Pro Preview76.2%

3.Claude Opus 4.773.3%

4.Claude Opus 4.568.5%

5.GPT-5.562.8%

6.Gemini 2.5 Pro52.5%

7.Gemini 2.5 Flash46.7%

8.o134.4%

9.GPT-4o mini26.3%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Metric: exact-match accuracy
Scale: 0-100
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-5c16f4ffaaa5. Source payload hash: 5c16f4ffaaa5ac8a5c682da01aa3fab9655de1625348891876a58ad2be1d678f. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Predictability

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 6 of 9Data date: August 6, 2026

1.Claude Opus 4.50.84 rating

2.Gemini 3.5 Flash0.8 rating

2.GPT-5.50.8 rating

4.Gemini 3.1 Pro Preview0.78 rating

5.Claude Opus 4.70.73 rating

6.Gemini 2.5 Pro0.71 rating

7.o10.68 rating

8.Gemini 2.5 Flash0.64 rating

9.GPT-4o mini0.58 rating

Scale: 0 rating - 0.84 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Metric: aggregate predictability
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-1c53c174caef. Source payload hash: 1c53c174caef16e9fce637ae32349d954da5c8a7493eb5ccbfc1b3b74b7df1b2. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Reliability

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 7 of 9Data date: August 6, 2026

1.Claude Opus 4.50.85 rating

2.Claude Opus 4.70.84 rating

2.Gemini 3.5 Flash0.84 rating

4.Gemini 3.1 Pro Preview0.82 rating

5.GPT-5.50.79 rating

5.o10.79 rating

7.Gemini 2.5 Pro0.78 rating

8.GPT-4o mini0.76 rating

9.Gemini 2.5 Flash0.74 rating

Scale: 0 rating - 0.85 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Metric: aggregate reliability
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-8d19bbb49250. Source payload hash: 8d19bbb4925070a4e6ec390b4d5684ae50db13a01227057bc90bb40f1a762e09. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Robustness

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 7 of 9Data date: August 6, 2026

1.o11 rating

2.Claude Opus 4.70.99 rating

3.GPT-4o mini0.97 rating

4.Gemini 3.5 Flash0.96 rating

5.GPT-5.50.95 rating

6.Gemini 3.1 Pro Preview0.94 rating

7.Claude Opus 4.50.91 rating

7.Gemini 2.5 Pro0.91 rating

9.Gemini 2.5 Flash0.87 rating

Scale: 0 rating - 1 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Metric: aggregate robustness
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-1796d18fe25c. Source payload hash: 1796d18fe25c8013919dfe86d7063843be59e912ef58bfb042e346d61d24613e. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

GAIA Reliability

Safety

What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.

Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Higher is better9 modelsGemini 2.5 Pro: Rank 1 of 9Data date: August 6, 2026

1.Claude Opus 4.51 rating

1.Claude Opus 4.71 rating

1.Gemini 2.5 Flash1 rating

1.Gemini 2.5 Pro1 rating

1.Gemini 3.1 Pro Preview1 rating

1.Gemini 3.5 Flash1 rating

1.GPT-4o mini1 rating

1.GPT-5.51 rating

1.o11 rating

Scale: 0 rating - 1 rating

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-safety
Metric: aggregate safety
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c63a794d3d54. Source payload hash: c63a794d3d549f51d32f25a48655db8386502ffd23bb1bd44974e9db28f6bdc8. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

Recorded measurement details

Every observation retains its source value and published test conditions.

Aider Polyglot: Overall (Show measurement, test conditions, and source)

Source value: 83.1
Score: 83.1
Metric: percent correct after two attempts
Unit: %
Benchmark version: 225-case polyglot suite
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: aider-polyglot-pass2-diff
Source type: official-leaderboard
Evaluator: Aider
Status: active
Retrieved at: 2026-09-02
Methodology: 225 Exercism tasks across six languages using Aider and a documented edit format.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Model snapshot: gemini-2.5-pro-preview-2025-06-05. Evaluator: Aider. Dataset type: public. Leaderboard date: 2025-06-06. Attempts: pass 2. Reasoning effort: 32k thinking tokens. Latency in seconds: 200.3. Cost per task in USD: 0.22169866666666663. Source ID: aider. Evidence capture ID: curated-aider-2026-09-02-b94f7308e188. Source payload hash: b94f7308e188d90f6864b22fd13d3d933378668e9fc981a7655173a4c3545a4e. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 225 Exercism tasks across six languages using Aider and a documented edit format.

Source Aider

FACTS Benchmark Suite: Document grounding (Show measurement, test conditions, and source)

Source value: 74.2
Score: 74.2
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: grounding
Task label: Document grounding
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:grounding
Confidence interval: 2
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-8a8b1ae4abae. Source payload hash: 8a8b1ae4abae2b077a109193965fb09bce92092c2be809bfae78d4cfa219dc27. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Multimodal factuality (Show measurement, test conditions, and source)

Source value: 46.9
Score: 46.9
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: multimodal
Task label: Multimodal factuality
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:multimodal
Confidence interval: 3.6
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-26c476d1beb7. Source payload hash: 26c476d1beb7b7772cf5b5a2e60452d78ae516f5f936f804b1cccfead5529b32. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: FACTS overall score (Show measurement, test conditions, and source)

Source value: 62.1
Score: 62.1
Metric: FACTS score
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: overall
Task label: FACTS overall score
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:overall
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-f90e68e563fe. Source payload hash: f90e68e563fe354373076ca491cfa45c3d12df9ca257aa0eff56a191c63c53fa. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Parametric knowledge (Show measurement, test conditions, and source)

Source value: 63.2
Score: 63.2
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: parametric
Task label: Parametric knowledge
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:parametric
Confidence interval: 2
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-68c34aa514b7. Source payload hash: 68c34aa514b740c6f80ada1a284136c938a1f97cd1e65c0f87138497fc92708c. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

FACTS Benchmark Suite: Factuality with search (Show measurement, test conditions, and source)

Source value: 63.9
Score: 63.9
Metric: accuracy
Unit: %
Benchmark version: FACTS Benchmark Suite v2
Category: knowledge
Task: search
Task label: Factuality with search
Direction: Higher is better
Comparison cohort: facts-suite-v2:official:search
Confidence interval: 2
Source type: official-leaderboard
Evaluator: Google DeepMind
Status: active
Retrieved at: 2026-08-09
Methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Google DeepMind. Dataset type: public. Leaderboard date: 2025-12-11. Source ID: google-deepmind. Evidence capture ID: curated-google-deepmind-2026-08-09-36ddfd1bf586. Source payload hash: 36ddfd1bf586c681280b54cf509f9fbb4f8f41eeebb44d9a562d34aed96401d9. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Public and private splits. Document, image, parametric knowledge, and standardized web search remain separate.

Source Google DeepMind

GAIA Reliability: Consistency (Show measurement, test conditions, and source)

Source value: 0.72
Score: 0.72
Metric: aggregate consistency
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c31d0b9d9ce4. Source payload hash: c31d0b9d9ce4ae3176dd4e9d861282248ef948babbb34bde532eb1ccbbc96780. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Accuracy (Show measurement, test conditions, and source)

Source value: 52.5
Score: 52.5
Metric: exact-match accuracy
Unit: %
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-5c16f4ffaaa5. Source payload hash: 5c16f4ffaaa5ac8a5c682da01aa3fab9655de1625348891876a58ad2be1d678f. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Predictability (Show measurement, test conditions, and source)

Source value: 0.71
Score: 0.71
Metric: aggregate predictability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-1c53c174caef. Source payload hash: 1c53c174caef16e9fce637ae32349d954da5c8a7493eb5ccbfc1b3b74b7df1b2. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Reliability (Show measurement, test conditions, and source)

Source value: 0.78
Score: 0.78
Metric: aggregate reliability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-8d19bbb49250. Source payload hash: 8d19bbb4925070a4e6ec390b4d5684ae50db13a01227057bc90bb40f1a762e09. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Robustness (Show measurement, test conditions, and source)

Source value: 0.91
Score: 0.91
Metric: aggregate robustness
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-1796d18fe25c. Source payload hash: 1796d18fe25c8013919dfe86d7063843be59e912ef58bfb042e346d61d24613e. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Safety (Show measurement, test conditions, and source)

Source value: 1
Score: 1
Metric: aggregate safety
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: safety
Task label: Safety
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-safety
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-pro. Source model name: Gemini 2.5 Pro. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-c63a794d3d54. Source payload hash: c63a794d3d549f51d32f25a48655db8386502ffd23bb1bd44974e9db28f6bdc8. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

Our model tests

Community tests you can inspect

Compare how the models respond to the same prompt. Each test shows the first attempt, with no subsequent fixes to the generated code. These results do not contribute to an overall score.

Pelican on a bicycle

A community classic for free-form SVG drawing.

Gemini 2.5 Pro

Not tested yet

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

Evidence

Sources and data date

Every statement links to its underlying documentation or leaderboard.

Speed: Google. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026.
Artificial Analysis Methodology
Price: Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs.
Pricing
Input: Documented modalities. Retrieved September 8, 2026.
Model documentation
Output: Documented modalities. Retrieved September 8, 2026.
Model documentation

Research date September 8, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.

Open evidence questions and research notes (original wording)

Not found in the inspected sources

parameters-billions. Google does not publish a parameter count on the Gemini API model page.

Not found in the inspected sources

active-parameters-billions. Google does not publish active parameter counts for Gemini API models.

Model metadataSource
Context window data (retrieved September 8, 2026)Source
Knowledge cutoff dataSource
API pricingSource
Aider (retrieved September 2, 2026)Source
DeepResearch Bench (retrieved August 9, 2026)Source
Google DeepMind (retrieved August 9, 2026)Source
Princeton HAL (retrieved August 6, 2026)Source
Princeton HAL (retrieved August 5, 2026)Source
LongBench (retrieved September 2, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Google DeepMind (retrieved September 8, 2026)Source
Artificial Analysis (retrieved September 8, 2026)Source
Model metadata (retrieved September 8, 2026)Source
Speed measurement methodologySource

Gemini 2.5 Pro

Published benchmarks

Aider Polyglot

Knowledge

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

FACTS Benchmark Suite

Agent tasks

GAIA Reliability

GAIA Reliability

GAIA Reliability

GAIA Reliability

GAIA Reliability

GAIA Reliability

Recorded measurement details

Community tests you can inspect

Pelican on a bicycle

Specifications and access

Additional source observations

Published prices

100 API requests

Compare this model

Sources and data date