Language modelActive

Gemini 2.5 Flash

Google

Released: June 17, 2025
Data date: September 8, 2026
Successor: Gemini 3 Flash Preview

Benchmark-Meta-Score: Unavailable
Fewer than eight test series
Speed: 215.1 tokens/s
Output throughput
Price: $0.3 · $2.5
Input · Output / 1M tokens
Input: Text, Image, Audio, Video
Output: Text

What is new in Gemini 2.5 Flash is hybrid reasoning. Google lets developers turn thinking on or off and control it through a budget. The stable GA model remains served, but API access is now limited to users with prior 2.5 usage. Gemini 3 Flash Preview is the later model in the documented line.

The model card lists limitations in causal understanding, complex logical deduction, and counterfactual reasoning.

Specifications and access

Specification	Value and source
Weights	ProprietarySource
Access	APISource
Context window	1,048,576 tokensSource
Knowledge cutoff	January 2025Source
Notes	1M context, parameter count not disclosedSource
API model ID	gemini-2.5-flashSource
Source status	stableSource
Maximum output	65,536 tokensSource
Input	Text, Image, Video, Audio, pdfSource
Output	TextSource
Reasoning mode	low, medium, highSource
Tool calling	YesSource
Structured output	YesSource
Release	2025-06-17Source
Knowledge cutoff	2025-01Source
Capabilities	function calling, structured outputs, search grounding, Google Maps grounding, URL context, code execution, file searchSource
General availability	2025-06-17 (General availability (GA).)Source

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-08-05 - 2026-08-26

Overall index

Not enough independent results for a score yet

An overall score needs at least eight test series.

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Test series: 5
Test families: 5 / 65
Sources: 2
Share of the fixed test panel (informative): 8 %

Agents

Index below 1 / 100

provisional

Standard error: ± 0.0
95% interval: 0.0 to 0.2
Test series: 2
Test families: 2 / 12
Sources: 1
Share of the fixed test panel (informative): 17 %

Finance

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.1
95% interval: 0.0 to 0.6
Test series: 1
Test families: 1 / 7
Sources: 1
Share of the fixed test panel (informative): 14 %

Legal

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.2
95% interval: 0.0 to 1.1
Test series: 1
Test families: 1 / 4
Sources: 1
Share of the fixed test panel (informative): 25 %

Multimodal

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.1
Test series: 1
Test families: 1 / 4
Sources: 1
Share of the fixed test panel (informative): 25 %

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v3

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. Unknown or ambiguous tiers exclude the model from that series. Within each series, percentages and rates are logit-transformed, other units stay unchanged, and all values are then standardized. A one-factor model then estimates one shared strength per model and, per series, how sharply it separates the models. Area scores move away from that strength only as far as the model's own results in the area support. The overall score weights the core areas at 75% and finance, law and health at 25%. It requires at least eight test series from two sources and three areas. The 0 to 100 scale refers to a fixed reference group of 29 language models whose average sits at 50.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured series are not entered as zero. Where a model has no own results in a core area, its overall strength stands in, and the note “estimated without own results in …” names the area. Fewer test series lead to a larger standard error.

The 95% interval follows from the standard error of 200 recalculations with randomly redrawn test families. It describes how precisely the recorded test series determine the value. It is no prediction for future tasks or for later recalculations with other participants.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 5 test families and 5 of 5 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: GAIA Reliability
Agents · Mean rank percentile of the family: 25.0 / 100
Family weight in the fixed test panel (informative): 0.89 % · Coverage of this test family: 100 %
- Princeton HAL
  Measurement:
  hal-gaia-reliability
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.89 %
  Raw value:
  46.7 %
  Metric:
  exact-match accuracy
  Version:
  final leak-cleaned public validation set
  Participating models:
  9
  Rank percentile within the cohort:
  25.0 %
  Retrieved:
  2026-08-06
  Comparison cohort:
  hal-gaia-reliability-hal-reliability-overall
  Documented comparison setup
  Publication cohort:
  hal-gaia-reliability-hal-reliability-overall
  Dataset type:
  public
  Harness:
  HAL Reliability
  Agent:
  HAL Reliability
  Setup and model parameters
  165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Harness: HAL Reliability. Agent: HAL Reliability
  165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ec0350e989af. Source payload hash: ec0350e989af492497edfa7fb858ea21c4693c375e9212b1ad488dc3f8eb81f8. Parser version: curated-v1. Imported at: 2026-10-15
Test family: τ-bench Airline Clean
Agents · Mean rank percentile of the family: 12.5 / 100
Family weight in the fixed test panel (informative): 0.89 % · Coverage of this test family: 100 %
- Princeton HAL
  Measurement:
  hal-tau-airline-clean
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.89 %
  Raw value:
  59 %
  Metric:
  task success rate
  Version:
  26-task cleaned subset
  Participating models:
  9
  Rank percentile within the cohort:
  12.5 %
  Retrieved:
  2026-08-05
  Comparison cohort:
  hal-tau-airline-clean-hal-reliability-overall
  Documented comparison setup
  Publication cohort:
  hal-tau-airline-clean-hal-reliability-overall
  Dataset type:
  public
  Harness:
  HAL Reliability
  Agent:
  HAL Reliability
  Setup and model parameters
  26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Harness: HAL Reliability. Agent: HAL Reliability
  26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-781c5011daf3. Source payload hash: 781c5011daf31b3367fa81ab29685b1d3b7b0e7f082a01f9888879c8a7223b33. Parser version: curated-v1. Imported at: 2026-10-15
Test family: PRBench Finance
Finance · Mean rank percentile of the family: 13.3 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Scale AI
  Published configurations differ within this cohort.
  Measurement:
  prbench-finance
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  38.41 %
  Metric:
  weighted rubric score
  Version:
  full finance set
  Participating models:
  16
  Rank percentile within the cohort:
  13.3 %
  Retrieved:
  2026-08-26
  Comparison cohort:
  scale-prbench-finance-full
  Documented comparison setup
  Publication cohort:
  scale-prbench-finance-full
  Dataset type:
  public
  Attempts:
  pass@1
  Setup and model parameters
  600 finance tasks with task-specific rubrics and o4-mini as judge.. Attempts: pass@1
  600 finance tasks with task-specific rubrics and o4-mini as judge.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Scale AI. Dataset type: public. Attempts: pass@1. Score display: 38.41% ± 0.62. Source ID: scale-ai. Evidence capture ID: curated-scale-ai-2026-08-26-88c18b250ab8. Source payload hash: 88c18b250ab815175025671db93668dc330c2e53f517c9e8a1de37d51f27b988. Parser version: curated-v1. Imported at: 2026-10-15
Test family: PRBench Legal
Legal · Mean rank percentile of the family: 26.7 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Scale AI
  Published configurations differ within this cohort.
  Measurement:
  prbench-legal
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  41.02 %
  Metric:
  weighted rubric score
  Version:
  full legal set
  Participating models:
  16
  Rank percentile within the cohort:
  26.7 %
  Retrieved:
  2026-08-26
  Comparison cohort:
  scale-prbench-legal-full
  Documented comparison setup
  Publication cohort:
  scale-prbench-legal-full
  Dataset type:
  public
  Attempts:
  pass@1
  Setup and model parameters
  500 legal tasks with task-specific rubrics and o4-mini as judge.. Attempts: pass@1
  500 legal tasks with task-specific rubrics and o4-mini as judge.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Scale AI. Dataset type: public. Attempts: pass@1. Score display: 41.02% ± 0.61. Source ID: scale-ai. Evidence capture ID: curated-scale-ai-2026-08-26-4e04093a101b. Source payload hash: 4e04093a101b1d3499b6333e4267cf0de410540678436d292d53e2bdc6fdb3ca. Parser version: curated-v1. Imported at: 2026-10-15
Test family: VisualToolBench
Multimodal · Mean rank percentile of the family: 22.2 / 100
Family weight in the fixed test panel (informative): 2.68 % · Coverage of this test family: 100 %
- Scale AI
  Published configurations differ within this cohort.
  Measurement:
  visual-toolbench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.68 %
  Raw value:
  4.69 %
  Metric:
  average pass rate
  Version:
  1,204-task release
  Participating models:
  10
  Rank percentile within the cohort:
  22.2 %
  Retrieved:
  2026-08-26
  Comparison cohort:
  scale-visual-toolbench-apr
  Documented comparison setup
  Publication cohort:
  scale-visual-toolbench-apr
  Dataset type:
  public
  Attempts:
  pass@1
  Setup and model parameters
  1,204 tasks, six tools, and at most 20 tool calls per task.. Attempts: pass@1
  1,204 tasks, six tools, and at most 20 tool calls per task.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Scale AI. Dataset type: public. Attempts: pass@1. Score display: 4.69% ± 0.13. Source ID: scale-ai. Evidence capture ID: curated-scale-ai-2026-08-26-d08ed1b114e6. Source payload hash: d08ed1b114e6589943992e9b9a599fec181130395dc31d8682e7dc5ca15a7130. Parser version: curated-v1. Imported at: 2026-10-15

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	3	1	16
Vals AI	3	0	14
Princeton HAL	2	0	12
Scale AI Evals	3	0	3

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

6 measurements from these sources still lack a verified test-family identity and do not increase the test-series count.

Published benchmarks

Agent tasks

GAIA Reliability

Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

Sub-task	Gemini 2.5 Flash (This model)	Claude Opus 4.5	Claude Opus 4.7	Gemini 3.1 Pro Preview	Gemini 3.5 Flash	GPT-4o mini	Gemini 2.5 Pro	o1	GPT-5.5
Consistency	0.7	0.81 (best value in this row)	0.81 (best value in this row)	0.76	0.75	0.73	0.72	0.68	0.61
Predictability	0.64	0.84 (best value in this row)	0.73	0.78	0.8	0.58	0.71	0.68	0.8
Reliability	0.74	0.85 (best value in this row)	0.84	0.82	0.84	0.76	0.78	0.79	0.79
Robustness	0.87	0.91	0.99	0.94	0.96	0.97	0.91	1 (best value in this row)	0.95
Safety	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)	1 (best value in this row)

Unit: ratinghigher is betterAs of August 6, 2026Source: Princeton HALbars scaled within each row, not across rows

GAIA Reliability

Accuracy

Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.

1.Gemini 3.5 Flash79.2%

2.Gemini 3.1 Pro Preview76.2%

3.Claude Opus 4.773.3%

4.Claude Opus 4.568.5%

5.GPT-5.562.8%

6.Gemini 2.5 Pro52.5%

7.Gemini 2.5 Flash46.7%

8.o134.4%

9.GPT-4o mini26.3%

higher is betterAs of August 6, 2026Scale: 0% - 100%

Source: Princeton HAL

gradually.ai

Methodology and context

Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Metric: exact-match accuracy
Scale: 0-100
Participants: 9
Methodology and context: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ec0350e989af. Source payload hash: ec0350e989af492497edfa7fb858ea21c4693c375e9212b1ad488dc3f8eb81f8. Parser version: curated-v1. Imported at: 2026-10-15

Source: Princeton HAL

τ-bench Airline Clean

26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools. The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.

Sub-task	Gemini 2.5 Flash (This model)	Claude Opus 4.7	Claude Opus 4.5	GPT-5.5	Gemini 3.1 Pro Preview	Gemini 3.5 Flash	Gemini 2.5 Pro	GPT-4o mini	o1
Consistency	0.75	0.88 (best value in this row)	0.84	0.84	0.81	0.78	0.74	0.74	0.71
Predictability	0.6	0.87 (best value in this row)	0.85	0.84	0.83	0.82	0.78	0.35	0.74
Reliability	0.72	0.89 (best value in this row)	0.88	0.89 (best value in this row)	0.86	0.86	0.81	0.67	0.81
Robustness	0.82	0.91	0.97	0.97	0.94	0.97	0.93	0.93	0.98 (best value in this row)
Safety	0.9	0.94	0.98	0.96	0.97	0.99 (best value in this row)	0.92	0.85	0.91

Unit: ratinghigher is betterAs of August 5, 2026Source: Princeton HALbars scaled within each row, not across rows

τ-bench Airline Clean

Accuracy

1.Claude Opus 4.784.6%

2.Gemini 3.1 Pro Preview82.1%

3.Claude Opus 4.580.8%

3.Gemini 3.5 Flash80.8%

5.GPT-5.579.5%

6.Gemini 2.5 Pro71.8%

7.o166.2%

8.Gemini 2.5 Flash59%

9.GPT-4o mini29.5%

higher is betterAs of August 5, 2026Scale: 0% - 100%

Source: Princeton HAL

gradually.ai

Methodology and context

Comparison cohort: hal-tau-airline-clean-hal-reliability-overall
Metric: task success rate
Scale: 0-100
Participants: 9
Methodology and context: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-781c5011daf3. Source payload hash: 781c5011daf31b3367fa81ab29685b1d3b7b0e7f082a01f9888879c8a7223b33. Parser version: curated-v1. Imported at: 2026-10-15

Source: Princeton HAL

Recorded measurement details

GAIA Reliability: Consistency (Show measurement, test conditions, and source)

Source value: 0.7
Score: 0.7
Metric: aggregate consistency
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-8fabb56ccf92. Source payload hash: 8fabb56ccf92853585e6cf40f25f4bd6eb7b3ed4d70a3a9f50d8b97eb56f4047. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Predictability (Show measurement, test conditions, and source)

Source value: 0.64
Score: 0.64
Metric: aggregate predictability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-d99783ab6f78. Source payload hash: d99783ab6f787e6c1457af9dde7cf159a8716a1f7e941533dfd47bd0dd47fdd0. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Reliability (Show measurement, test conditions, and source)

Source value: 0.74
Score: 0.74
Metric: aggregate reliability
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-168466553152. Source payload hash: 16846655315224fa440fcd0321202b7dfa8a4130573fe58faac6da71b1146e97. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Robustness (Show measurement, test conditions, and source)

Source value: 0.87
Score: 0.87
Metric: aggregate robustness
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-2b2fc2a6fed2. Source payload hash: 2b2fc2a6fed208076717b36345d3d979236a239b53816d51f5e6de909ab6e921. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Safety (Show measurement, test conditions, and source)

Source value: 1
Score: 1
Metric: aggregate safety
Unit: rating
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: safety
Task label: Safety
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-safety
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-7b0b29385a9a. Source payload hash: 7b0b29385a9adcf56735e8978a0e9f3a9c5ef2d1c29fc664c87ba01894f30f03. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

GAIA Reliability: Accuracy (Show measurement, test conditions, and source)

Source value: 46.7
Score: 46.7
Metric: exact-match accuracy
Unit: %
Benchmark version: final leak-cleaned public validation set
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-gaia-reliability-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ec0350e989af. Source payload hash: ec0350e989af492497edfa7fb858ea21c4693c375e9212b1ad488dc3f8eb81f8. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.

Source Princeton HAL

τ-bench Airline Clean: Consistency (Show measurement, test conditions, and source)

Source value: 0.75
Score: 0.75
Metric: aggregate consistency
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: consistency
Task label: Consistency
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-consistency
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-abed2a7158fa. Source payload hash: abed2a7158fa39004f9a9b2a459865dcf4cb79871117ca5d41fb73a82449378e. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Predictability (Show measurement, test conditions, and source)

Source value: 0.6
Score: 0.6
Metric: aggregate predictability
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: predictability
Task label: Predictability
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-predictability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-b5a35a49844a. Source payload hash: b5a35a49844a501da2d8ef2ec0e8f6f66be49c4889ff6c4cd7082dc165bf797e. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Reliability (Show measurement, test conditions, and source)

Source value: 0.72
Score: 0.72
Metric: aggregate reliability
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: reliability
Task label: Reliability
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-reliability
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-90e450de5dc4. Source payload hash: 90e450de5dc466dc08ea690da49ce7848c3ec293974c718f0631028afdc650b9. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Robustness (Show measurement, test conditions, and source)

Source value: 0.82
Score: 0.82
Metric: aggregate robustness
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: robustness
Task label: Robustness
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-robustness
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-9fdb5eb35d92. Source payload hash: 9fdb5eb35d92aac92a3e215d4bea6422b90dc35736dba1b0308299783017bfff. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Safety (Show measurement, test conditions, and source)

Source value: 0.9
Score: 0.9
Metric: aggregate safety
Unit: rating
Benchmark version: 26-task cleaned subset
Category: agentic
Task: safety
Task label: Safety
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-safety
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-6b877739ac96. Source payload hash: 6b877739ac96766c81a5b7b2e5ab1c275ede0f70561c6c04797bf264879d9461. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

τ-bench Airline Clean: Accuracy (Show measurement, test conditions, and source)

Source value: 59
Score: 59
Metric: task success rate
Unit: %
Benchmark version: 26-task cleaned subset
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-tau-airline-clean-hal-reliability-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-781c5011daf3. Source payload hash: 781c5011daf31b3367fa81ab29685b1d3b7b0e7f082a01f9888879c8a7223b33. Parser version: curated-v1. Imported at: 2026-10-15
Comparison methodology: 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.

Source Princeton HAL

Page 1 of 6

Predecessor and successor

Sources and data date

Type	Evidence and data date
Speed	Google, Non-reasoning. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Context window data (retrieved September 8, 2026) · Gemini 2.5 Flash API documentation · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Context window data (retrieved September 8, 2026) · Gemini 2.5 Flash API documentation · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	parameters-billions Original wording Google does not publish a parameter count on the Gemini API model page.
Not found in the inspected sources	active-parameters-billions Original wording Google does not publish active parameter counts for Gemini API models.
Additional source	Model metadata
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Princeton HAL (retrieved August 6, 2026)
Additional source	Princeton HAL (retrieved August 5, 2026)
Additional source	LongBench (retrieved September 2, 2026)
Additional source	Scale AI (retrieved August 26, 2026)
Additional source	Scale AI (retrieved August 26, 2026)
Additional source	Scale AI (retrieved August 26, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Google DeepMind (retrieved September 8, 2026) · Gemini 2.5 Flash model card · Editorial description reviewed October 2, 2026
Additional source	Model metadata (retrieved September 8, 2026)
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0
Additional source	Google, "Gemini 3 Flash" (2026-09-22)

Gemini 2.5 Flash

Specifications and access

Overall and area scores

Overall index

Agents

Finance

Legal

Multimodal

Test family: GAIA Reliability

Test family: τ-bench Airline Clean

Test family: PRBench Finance

Test family: PRBench Legal

Test family: VisualToolBench

Benchmark sources at a glance

Published benchmarks

Agent tasks

GAIA Reliability

GAIA Reliability

τ-bench Airline Clean

τ-bench Airline Clean

Recorded measurement details

Our tests

SVG short test

Gemini 2.5 Flash

Pelican on a bicycle

Popularity

Published prices

Predecessor and successor

Sources and data date