What is new in Gemini 3.1 Pro Preview is Google’s stated improvement in thinking, token efficiency, and factual grounding. Its API model name remains gemini-3.1-pro-preview. The preview suffix describes provisional availability and must not be read as GA.
It accepts multimodal input within a 1,048,576-token context and produces up to 65,536 text tokens. Google publishes neither the parameter count nor the architecture.
1,266 hard-to-find research tasks with short, unambiguous answers. The benchmark rewards persistent fact finding and only partly represents open-ended research reports or typical user questions.
Note (Gemini 3.1 Pro Preview): Editorial selection from the vendor table, not a complete extract of the comparison cohort.
Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
Unit: rating·higher is better·As of August 6, 2026·Source: Princeton HAL·bars scaled within each row, not across rows
GAIA Reliability
Accuracy
Reliability evaluation on 165 GAIA tasks with repetitions and perturbations. Accuracy, reliability, consistency, predictability, robustness, and safety are separate metrics.
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-5beb5d872d82. Source payload hash: 5beb5d872d8245204ec7f426f3063f94418e0cac67065ec663ddda909522a958. Parser version: curated-v1. Imported at: 2026-09-29
GAIA Reliability: Consistency (Show measurement, test conditions, and source)
Source value
0.76
Score
0.76
Metric
aggregate consistency
Unit
rating
Benchmark version
final leak-cleaned public validation set
Category
agentic
Task
consistency
Task label
Consistency
Direction
Higher is better
Comparison cohort
hal-gaia-reliability-hal-reliability-consistency
Source type
independent-evaluator
Evaluator
Princeton HAL
Status
active
Retrieved at
2026-08-06
Methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-5975efd2a7dd. Source payload hash: 5975efd2a7dde0fdac9d188612551abd16f9a35afccab8d423cc87e15ed79cb6. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-1ba765b3e748. Source payload hash: 1ba765b3e7483d1dfc42bdfb3314edd1fa8b393800a9cc76e26ed1fbbbd6c9f6. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Reliability (Show measurement, test conditions, and source)
Source value
0.82
Score
0.82
Metric
aggregate reliability
Unit
rating
Benchmark version
final leak-cleaned public validation set
Category
agentic
Task
reliability
Task label
Reliability
Direction
Higher is better
Comparison cohort
hal-gaia-reliability-hal-reliability-reliability
Source type
independent-evaluator
Evaluator
Princeton HAL
Status
active
Retrieved at
2026-08-06
Methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-7cad1cd2cba8. Source payload hash: 7cad1cd2cba8062a366befcbb0b7b6af15c287c05ba0e7dfba45f494c7973e84. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Robustness (Show measurement, test conditions, and source)
Source value
0.94
Score
0.94
Metric
aggregate robustness
Unit
rating
Benchmark version
final leak-cleaned public validation set
Category
agentic
Task
robustness
Task label
Robustness
Direction
Higher is better
Comparison cohort
hal-gaia-reliability-hal-reliability-robustness
Source type
independent-evaluator
Evaluator
Princeton HAL
Status
active
Retrieved at
2026-08-06
Methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-83df5a085c25. Source payload hash: 83df5a085c25186a09c3c488ed657d137f4dd3e70e0d82bbc805faa52620fbfa. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Safety (Show measurement, test conditions, and source)
Source value
1
Score
1
Metric
aggregate safety
Unit
rating
Benchmark version
final leak-cleaned public validation set
Category
agentic
Task
safety
Task label
Safety
Direction
Higher is better
Comparison cohort
hal-gaia-reliability-hal-reliability-safety
Source type
independent-evaluator
Evaluator
Princeton HAL
Status
active
Retrieved at
2026-08-06
Methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-6b1585568e22. Source payload hash: 6b1585568e22bcfd30d0dcb258148c811925b9d304059f33cf26228e7ae5399d. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
GAIA Reliability: Accuracy (Show measurement, test conditions, and source)
Source value
76.2
Score
76.2
Metric
exact-match accuracy
Unit
%
Benchmark version
final leak-cleaned public validation set
Category
agentic
Task
overall
Task label
Accuracy
Direction
Higher is better
Comparison cohort
hal-gaia-reliability-hal-reliability-overall
Source type
independent-evaluator
Evaluator
Princeton HAL
Status
active
Retrieved at
2026-08-06
Methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.. Source model ID: curated/gemini-3.1-pro. Source model name: Gemini 3.1 Pro Preview. Provider reported by source: Google. Evaluator: Princeton HAL. Dataset type: public. Harness: HAL Reliability. Agent: HAL Reliability. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-5beb5d872d82. Source payload hash: 5beb5d872d8245204ec7f426f3063f94418e0cac67065ec663ddda909522a958. Parser version: curated-v1. Imported at: 2026-09-29
Comparison methodology
165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation.
Each test shows the latest attempt, with no subsequent fixes to the generated code. Earlier runs remain available in the test details. These results do not contribute to an overall score.
Pelican on a bicycle
A community classic for free-form SVG drawing.
Gemini 3.1 Pro Preview
The response reached the token limit. No complete recording is available.
Reasoning (requested): high
Current attempt
Run date
2 Oct 2026
API provider
openrouter
Requested API model
google/gemini-3.1-pro-preview
API model reported in response
google/gemini-3.1-pro-preview
Reasoning setting
high
Pinned endpoint
Google AI Studio | google/gemini-3.1-pro-preview-20260219
Temperature
Provider default
Token limit including reasoning
8,192 tokens
Test protocol
community-visual-tests-v1
Technical checks
Failed
Response time
61.6 s
Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.
Earlier attempts (2)
1 Oct 2026API request failedReasoning setting: high
7 Sept 2026API request failedReasoning setting: high
The table lists `gemini-3-pro-preview` (released November 18, 2025) discontinued March 9, 2026 with recommended replacement `gemini-3.1-pro-preview`, which matches this directory's Gemini 3.1 Pro Preview profile (released February 19, 2026 per the same table); the GA `gemini-2.5-pro` id itself carries no shutdown date or replacement, so the documented predecessor is the untracked Gemini 3 Pro Preview, not Gemini 2.5 Pro directly.