Gemini 2.5 Flash
Not tested yet
Language modelActive
Gemini 2.5 Flash
- Released
- April 2025
- Data date
- September 8, 2026
- Intelligence
- Overall score
- Speed
- 215.1 tokens/sOutput throughput
- Price
- Input · Output / 1M tokens
- Input
- Text, Image, Audio, Video
- Output
- Text
No published category-wide ranking is available for this model.
Entries 1-12 of 57. Page 1 of 5.
Published benchmarks
Each chart compares models in the same documented test and highlights the current model.
Agent tasks
12 measurementsGAIA Reliability
Consistency
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 7 of 9Data date: August 6, 2026
Scale: 0 rating - 0.81 rating
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-consistency
- Metric
- aggregate consistency
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-8fabb56ccf92. Source payload hash: 8fabb56ccf92853585e6cf40f25f4bd6eb7b3ed4d70a3a9f50d8b97eb56f4047. Parser version: curated-v1. Imported at: 2026-09-04
GAIA Reliability
Accuracy
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 7 of 9Data date: August 6, 2026
Scale: 0% - 100%
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-overall
- Metric
- exact-match accuracy
- Scale
- 0-100
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ec0350e989af. Source payload hash: ec0350e989af492497edfa7fb858ea21c4693c375e9212b1ad488dc3f8eb81f8. Parser version: curated-v1. Imported at: 2026-09-04
GAIA Reliability
Predictability
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 8 of 9Data date: August 6, 2026
Scale: 0 rating - 0.84 rating
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-predictability
- Metric
- aggregate predictability
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-d99783ab6f78. Source payload hash: d99783ab6f787e6c1457af9dde7cf159a8716a1f7e941533dfd47bd0dd47fdd0. Parser version: curated-v1. Imported at: 2026-09-04
GAIA Reliability
Reliability
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 9 of 9Data date: August 6, 2026
Scale: 0 rating - 0.85 rating
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-reliability
- Metric
- aggregate reliability
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-168466553152. Source payload hash: 16846655315224fa440fcd0321202b7dfa8a4130573fe58faac6da71b1146e97. Parser version: curated-v1. Imported at: 2026-09-04
GAIA Reliability
Robustness
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 9 of 9Data date: August 6, 2026
Scale: 0 rating - 1 rating
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-robustness
- Metric
- aggregate robustness
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-2b2fc2a6fed2. Source payload hash: 2b2fc2a6fed208076717b36345d3d979236a239b53816d51f5e6de909ab6e921. Parser version: curated-v1. Imported at: 2026-09-04
GAIA Reliability
Safety
What this benchmark measures: Reliability evaluation on 165 GAIA tasks with repetitions and perturbations.
Context: Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Higher is better9 modelsGemini 2.5 Flash: Rank 1 of 9Data date: August 6, 2026
Scale: 0 rating - 1 rating
Methodology and context
- Comparison cohort
- hal-gaia-reliability-hal-reliability-safety
- Metric
- aggregate safety
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-7b0b29385a9a. Source payload hash: 7b0b29385a9adcf56735e8978a0e9f3a9c5ef2d1c29fc664c87ba01894f30f03. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Consistency
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 6 of 9Data date: August 5, 2026
Scale: 0 rating - 0.88 rating
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-consistency
- Metric
- aggregate consistency
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-abed2a7158fa. Source payload hash: abed2a7158fa39004f9a9b2a459865dcf4cb79871117ca5d41fb73a82449378e. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Accuracy
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 8 of 9Data date: August 5, 2026
Scale: 0% - 100%
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-overall
- Metric
- task success rate
- Scale
- 0-100
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-781c5011daf3. Source payload hash: 781c5011daf31b3367fa81ab29685b1d3b7b0e7f082a01f9888879c8a7223b33. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Predictability
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 8 of 9Data date: August 5, 2026
Scale: 0 rating - 0.87 rating
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-predictability
- Metric
- aggregate predictability
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-b5a35a49844a. Source payload hash: b5a35a49844a501da2d8ef2ec0e8f6f66be49c4889ff6c4cd7082dc165bf797e. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Reliability
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 8 of 9Data date: August 5, 2026
Scale: 0 rating - 0.89 rating
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-reliability
- Metric
- aggregate reliability
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-90e450de5dc4. Source payload hash: 90e450de5dc466dc08ea690da49ce7848c3ec293974c718f0631028afdc650b9. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Robustness
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 9 of 9Data date: August 5, 2026
Scale: 0 rating - 0.98 rating
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-robustness
- Metric
- aggregate robustness
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-9fdb5eb35d92. Source payload hash: 9fdb5eb35d92aac92a3e215d4bea6422b90dc35736dba1b0308299783017bfff. Parser version: curated-v1. Imported at: 2026-09-04
τ-bench Airline Clean
Safety
What this benchmark measures: 26 cleaned customer-service tasks involving dialogue, policy compliance, and mutating backend tools.
Context: The cleaned subset removes 24 broken or ambiguous tasks. Accuracy and reliability dimensions measure different properties.
Higher is better9 modelsGemini 2.5 Flash: Rank 8 of 9Data date: August 5, 2026
Scale: 0 rating - 0.99 rating
Methodology and context
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-safety
- Metric
- aggregate safety
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 9
- Methodology and context
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-6b877739ac96. Source payload hash: 6b877739ac96766c81a5b7b2e5ab1c275ede0f70561c6c04797bf264879d9461. Parser version: curated-v1. Imported at: 2026-09-04
Recorded measurement details
Every observation retains its source value and published test conditions.
GAIA Reliability: Consistency (Show measurement, test conditions, and source)
- Source value
- 0.7
- Score
- 0.7
- Metric
- aggregate consistency
- Unit
- rating
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- consistency
- Task label
- Consistency
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-consistency
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-8fabb56ccf92. Source payload hash: 8fabb56ccf92853585e6cf40f25f4bd6eb7b3ed4d70a3a9f50d8b97eb56f4047. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Accuracy (Show measurement, test conditions, and source)
- Source value
- 46.7
- Score
- 46.7
- Metric
- exact-match accuracy
- Unit
- %
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- overall
- Task label
- Accuracy
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-overall
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-ec0350e989af. Source payload hash: ec0350e989af492497edfa7fb858ea21c4693c375e9212b1ad488dc3f8eb81f8. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Predictability (Show measurement, test conditions, and source)
- Source value
- 0.64
- Score
- 0.64
- Metric
- aggregate predictability
- Unit
- rating
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- predictability
- Task label
- Predictability
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-predictability
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-d99783ab6f78. Source payload hash: d99783ab6f787e6c1457af9dde7cf159a8716a1f7e941533dfd47bd0dd47fdd0. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Reliability (Show measurement, test conditions, and source)
- Source value
- 0.74
- Score
- 0.74
- Metric
- aggregate reliability
- Unit
- rating
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- reliability
- Task label
- Reliability
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-reliability
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-168466553152. Source payload hash: 16846655315224fa440fcd0321202b7dfa8a4130573fe58faac6da71b1146e97. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Robustness (Show measurement, test conditions, and source)
- Source value
- 0.87
- Score
- 0.87
- Metric
- aggregate robustness
- Unit
- rating
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- robustness
- Task label
- Robustness
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-robustness
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-2b2fc2a6fed2. Source payload hash: 2b2fc2a6fed208076717b36345d3d979236a239b53816d51f5e6de909ab6e921. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Safety (Show measurement, test conditions, and source)
- Source value
- 1
- Score
- 1
- Metric
- aggregate safety
- Unit
- rating
- Benchmark version
- final leak-cleaned public validation set
- Category
- agentic
- Task
- safety
- Task label
- Safety
- Direction
- Higher is better
- Comparison cohort
- hal-gaia-reliability-hal-reliability-safety
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-06
- Methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-7b0b29385a9a. Source payload hash: 7b0b29385a9adcf56735e8978a0e9f3a9c5ef2d1c29fc664c87ba01894f30f03. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
τ-bench Airline Clean: Consistency (Show measurement, test conditions, and source)
- Source value
- 0.75
- Score
- 0.75
- Metric
- aggregate consistency
- Unit
- rating
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- consistency
- Task label
- Consistency
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-consistency
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-abed2a7158fa. Source payload hash: abed2a7158fa39004f9a9b2a459865dcf4cb79871117ca5d41fb73a82449378e. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
τ-bench Airline Clean: Accuracy (Show measurement, test conditions, and source)
- Source value
- 59
- Score
- 59
- Metric
- task success rate
- Unit
- %
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- overall
- Task label
- Accuracy
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-overall
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-781c5011daf3. Source payload hash: 781c5011daf31b3367fa81ab29685b1d3b7b0e7f082a01f9888879c8a7223b33. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
τ-bench Airline Clean: Predictability (Show measurement, test conditions, and source)
- Source value
- 0.6
- Score
- 0.6
- Metric
- aggregate predictability
- Unit
- rating
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- predictability
- Task label
- Predictability
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-predictability
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-b5a35a49844a. Source payload hash: b5a35a49844a501da2d8ef2ec0e8f6f66be49c4889ff6c4cd7082dc165bf797e. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
τ-bench Airline Clean: Reliability (Show measurement, test conditions, and source)
- Source value
- 0.72
- Score
- 0.72
- Metric
- aggregate reliability
- Unit
- rating
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- reliability
- Task label
- Reliability
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-reliability
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-90e450de5dc4. Source payload hash: 90e450de5dc466dc08ea690da49ce7848c3ec293974c718f0631028afdc650b9. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
τ-bench Airline Clean: Robustness (Show measurement, test conditions, and source)
- Source value
- 0.82
- Score
- 0.82
- Metric
- aggregate robustness
- Unit
- rating
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- robustness
- Task label
- Robustness
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-robustness
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-9fdb5eb35d92. Source payload hash: 9fdb5eb35d92aac92a3e215d4bea6422b90dc35736dba1b0308299783017bfff. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
τ-bench Airline Clean: Safety (Show measurement, test conditions, and source)
- Source value
- 0.9
- Score
- 0.9
- Metric
- aggregate safety
- Unit
- rating
- Benchmark version
- 26-task cleaned subset
- Category
- agentic
- Task
- safety
- Task label
- Safety
- Direction
- Higher is better
- Comparison cohort
- hal-tau-airline-clean-hal-reliability-safety
- Source type
- independent-evaluator
- Evaluator
- Princeton HAL
- Status
- active
- Retrieved at
- 2026-08-05
- Methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.. Source model ID: curated/gemini-2.5-flash. Source model name: Gemini 2.5 Flash. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-6b877739ac96. Source payload hash: 6b877739ac96766c81a5b7b2e5ab1c275ede0f70561c6c04797bf264879d9461. Parser version: curated-v1. Imported at: 2026-09-04
- Comparison methodology
- 26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset.
Our model tests
Community tests you can inspect
Compare how the models respond to the same prompt. Each test shows the first attempt, with no subsequent fixes to the generated code. These results do not contribute to an overall score.
Pelican on a bicycle
A community classic for free-form SVG drawing.
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)Profile
Specifications and access
Published information about this model. Existing estimates are explicitly labeled.
Additional source observations
Dated documentation and model-card observations. Provider limits, native context and extended context can differ. Configuration notes retain the source wording.
- API model ID
- gemini-2.5-flashSource
- Source status
- stableSource
- Maximum output
- 65,536 tokensSource
- Input
- Text, Image, Video, Audio, pdfSource
- Output
- TextSource
- Reasoning mode
- low, medium, highSource
- Tool calling
- YesSource
- Structured output
- YesSource
- Release
- 2025-06-17Source
- Knowledge cutoff
- 2025-01Source
- Capabilities
- function calling, structured outputs, search grounding, Google Maps grounding, URL context, code execution, file searchSource
- General availability
- 2025-06-17 (General availability (GA).)Source
Pricing
Published prices
Prices apply to the stated unit. Resolution, output length, and provider can change the cost.
Cost example
100 API requests
$0.155
100 requests with 1,000 input and 500 output tokens each calculate to $0.155. Input accounts for $0.03; output accounts for $0.125.
- Input price
- $0.3 / 1M tokensBase tier for each request
- Output price
- $2.5 / 1M tokensBase tier for each request
The calculation uses documented token prices and no cache discount. It excludes extra tools, tax, reasoning tokens, and further hidden output tokens.
Open sourceHead-to-head comparisons
Compare this model
Each matchup compares this model with exactly one other model from the same category.
There are no published direct comparisons for this model yet.
Evidence
Sources and data date
Every statement links to its underlying documentation or leaderboard.
- Speed
Google, Non-reasoning. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026.
- Price
Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs.
- Input
Documented modalities. Retrieved September 8, 2026.
- Output
Documented modalities. Retrieved September 8, 2026.
Research date September 8, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Open evidence questions and research notes (original wording)
Not found in the inspected sources
parameters-billions. Google does not publish a parameter count on the Gemini API model page.
Not found in the inspected sources
active-parameters-billions. Google does not publish active parameter counts for Gemini API models.
- Model metadataSource
- Context window data (retrieved September 8, 2026)Source
- API pricingSource
- Princeton HAL (retrieved August 6, 2026)Source
- Princeton HAL (retrieved August 5, 2026)Source
- LongBench (retrieved September 2, 2026)Source
- Scale AI (retrieved August 26, 2026)Source
- Scale AI (retrieved August 26, 2026)Source
- Scale AI (retrieved August 26, 2026)Source
- Vals AI (retrieved August 11, 2026; September 4, 2026)Source
- Vals AI (retrieved September 4, 2026)Source
- Vals AI (retrieved September 4, 2026)Source
- Vals AI (retrieved August 11, 2026; September 4, 2026)Source
- Google DeepMind (retrieved September 8, 2026)Source
- Artificial Analysis (retrieved September 8, 2026)Source
- Model metadata (retrieved September 8, 2026)Source
- Speed measurement methodologySource