Gemini 3.8 Flash
Unavailable in this round
Language modelActive
Gemini 3.8 Flash
- Released
- September 2, 2026
- Data date
- September 8, 2026
- Predecessor
- Gemini 3.7 Flash
- Benchmark-Meta-Score
- Provisional, 63.8 % panel coverage
- Speed
- 280.8 tokens/sOutput throughput
- Price
- Input · Output / 1M tokens
- Input
- Text, Image, Audio, Video
- Output
- Text
Gemini 3.8 Flash improves over Gemini 3.7 Flash mainly on longer software-engineering tasks. Google’s DeepSWE v1.1 result rises from 65.3% to 73.7%. On Terminal-Bench 2.1, Google reports a gain from 85.8% to 89.4%. These results come from the provider’s evaluations.
For computer use, Gemini 3.8 Flash scores 59.0% against its predecessor’s 50.6% in the published OSWorld 2.0 test. Google measures partial credit with batch tool calls enabled. Computer use remains a preview capability in the API.
- Position
- Rank 7 of 24
- Index score
- 66.3 / 100
Methodology and source
The LLM comparison hub's selected-panel index averages rank percentiles from 10 documented comparison cohorts. Benchmark meta-scores are shown in the benchmark section.
Benchmark meta-analysis
Overall and area scores
These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.
Retrieval dates of the sources used 2026-10-15 - 2026-10-19
Overall index
Index 61.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 64 %
- Test families
- 48 / 71
- Sources
- 4
- Mathematical bounds with missing values
- 39.5-75.7 / 100
Coding
Index 57.5 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 76 %
- Test families
- 12 / 15
- Sources
- 3
- Mathematical bounds with missing values
- 43.5-67.9 / 100
Reasoning
Index 81.8 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 64 %
- Test families
- 5 / 7
- Sources
- 3
- Mathematical bounds with missing values
- 52.6-88.3 / 100
Mathematics
Index 44.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 60 %
- Test families
- 3 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 26.5-66.5 / 100
Knowledge
Index 89.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 3 / 3
- Sources
- 3
- Mathematical bounds with missing values
- 89.9-89.9 / 100
Agents
Index 75.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 6 / 12
- Sources
- 2
- Mathematical bounds with missing values
- 37.9-87.9 / 100
General
Index 60.6 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 80 %
- Test families
- 4 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 48.5-68.5 / 100
Finance
Index 75.0 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 71 %
- Test families
- 5 / 7
- Sources
- 1
- Mathematical bounds with missing values
- 53.6-82.1 / 100
Legal
Index 79.6 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 75 %
- Test families
- 3 / 4
- Sources
- 1
- Mathematical bounds with missing values
- 59.7-84.7 / 100
Health
Index 65.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 67 %
- Test families
- 2 / 3
- Sources
- 1
- Mathematical bounds with missing values
- 44.0-77.3 / 100
Cybersecurity
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 1 / 1
- Sources
- 1
- Mathematical bounds with missing values
- 13.6-13.6 / 100
Multimodal
Index 89.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 2 / 4
- Sources
- 3
- Mathematical bounds with missing values
- 44.6-94.6 / 100
Productivity
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 1 / 2
- Sources
- 1
- Mathematical bounds with missing values
- 30.3-80.3 / 100
Education
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 1 / 2
- Sources
- 1
- Mathematical bounds with missing values
- 7.8-57.8 / 100
Documented method · Included test families and measurements
Method version: benchmark-meta-score-v1
The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture; when dates match, the wider slice is used. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. When selecting between multiple results, unknown or ambiguous tiers exclude the model from that series. Unambiguous single configurations do not require a tier ordering. The best individual result for each model is not selected. Equal raw values share the same rank percentile. Multiple series from the same evaluator within a family receive equal weight; evaluators within a family, families within an area, and areas in the overall index then receive equal weight. Unmeasured series remain visible in coverage and bounds. An overall score requires at least four areas, six test families, and two sources. An area score requires at least two test families. Every overall and area score also requires at least 50% of its panel weight.
Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.
Unmeasured portions are not entered as zero. They reduce coverage and contribute only to the displayed bounds. Scores with different coverage are therefore not equally complete.
The bounds hold observed percentiles fixed and allow missing contributions anywhere from 0 to 100. They are not confidence intervals or predictions of future recalculations.
The source count is not a count of independent experiments.
OECD handbook on composite indicatorsAll 48 test families and 12 of 54 raw measurements are listed here. The complete calculation for every model is available in the JSON export.
Test family: AIME
Mathematics · Family index: 61.4 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-otis-mock-aime-2024-2025
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.43 %
- Raw value:
- 0.9888888888888889 ratio
- Metric:
- model_extracted_exact_match
- Version:
- 1.1.6
- Participating models:
- 23
- Rank percentile within the cohort:
- 61.4 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Reasoning tier:
- high
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: high. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gemini-3.8-flash_high; Epoch model label: Gemini 3.8 Flash; Display label field: Gemini 3.8 Flash (high); Reasoning setting encoded in the alias: high. Task ID: OTIS Mock AIME 2024-2025. Task version: 1.1.6. Run ID: NeYuqgQyWtFWAHEjQYPm5Z. Started at 2026-09-02T17:15:36.000Z. Original task label: OTIS Mock AIME 2024-2025. The published row reports mean_score=0.9888888888888889 and stderr=0.008724007409153888. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: high.
- Epoch AI benchmark run dataset snapshot
Test family: AA-AnalystAgent
Agents · Family index: 67.1 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-analyst-agent
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 46.25 %
- Metric:
- pass^5 reliability
- Version:
- AA cohort: 80 questions
- Participating models:
- 39
- Rank percentile within the cohort:
- 67.1 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-analyst-agent:AA cohort: 80 questions:standalone:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-analyst-agent","AA cohort: 80 questions","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup analyst agent
- Tool policy:
- 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission
- Attempts:
- pass^5, 5 independent attempts per question
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 46.3%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Briefcase
Agents · Family index: 64.6 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-briefcase
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,202.59 Elo
- Metric:
- AA-Briefcase Elo
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 64.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-briefcase:v1.1:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-briefcase","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-LCR
Reasoning · Family index: 77.4 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-lcr
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 81.3333333333333 %
- Metric:
- Pass rate
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 77.4 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-lcr:v1.1:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-lcr","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 81.3%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AutomationBench-AA
Agents · Family index: 85.4 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-automationbench-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 59.93009432118242 %
- Metric:
- Objectives completed without guardrail violations
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 85.4 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-automationbench-aa:unspecified:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-automationbench-aa","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 59.9%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: FrontierCode 1.1 Main
Coding · Family index: 26.7 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-frontiercode
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.48 %
- Raw value:
- 36.3585 %
- Metric:
- Mean score over three attempts per task
- Version:
- 1.1 Main
- Participating models:
- 16
- Rank percentile within the cohort:
- 26.7 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-frontiercode:1.1 Main:standalone:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-frontiercode","1.1 Main","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup coding agent
- Tool policy:
- 250-turn cap; repository worktree graded as left by the agent
- Attempts:
- Mean score, 3 attempts per task
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. The primary 100-task FrontierCode 1.1 set is run three times per task, with equal task weights. Failing any blocking check or being flagged by the trajectory scanner scores zero; runs passing every blocking check score 70 to 100 percent.. Harness: Artificial Analysis Stirrup. Agent: Stirrup coding agent. Tool policy: 250-turn cap; repository worktree graded as left by the agent. Attempts: Mean score, 3 attempts per task. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. The primary 100-task FrontierCode 1.1 set is run three times per task, with equal task weights. Failing any blocking check or being flagged by the trajectory scanner scores zero; runs passing every blocking check score 70 to 100 percent.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup coding agent. Tool policy: 250-turn cap; repository worktree graded as left by the agent. Attempts: Mean score, 3 attempts per task. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 36.4%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDP.pdf
Reasoning · Family index: 79.3 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdp-pdf
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 21 %
- Metric:
- All-pass rate
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 79.3 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdp-pdf:unspecified:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdp-pdf","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 21.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDPval-AA
Agents · Family index: 72.0 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdpval-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,436.3 Elo
- Metric:
- GDPval-AA Elo
- Version:
- v2.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 72.0 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdpval-aa:v2.1:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdpval-aa","v2.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: ITBench-AA (SRE)
Agents · Family index: 97.1 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-itbench-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 52.5423728813559 %
- Metric:
- Average precision at full recall
- Version:
- AA cohort: 59 tasks
- Participating models:
- 36
- Rank percentile within the cohort:
- 97.1 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-itbench-aa:AA cohort: 59 tasks:standalone:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-itbench-aa","AA cohort: 59 tasks","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup agent
- Tool policy:
- Shell access to an offline Kubernetes incident snapshot
- Attempts:
- Average precision at full recall, 3 repeats per task
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: High. Score display: 52.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Omniscience
Knowledge · Family index: 91.5 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-omniscience
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 2.38 %
- Raw value:
- 29.55 points
- Metric:
- AA-Omniscience Index
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 91.5 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-omniscience:unspecified:v4.3.2:reasoning:tokens=0:suffix=High
- Reasoning tier:
- High
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-omniscience","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: High
Exact AA configuration: Gemini 3.8 Flash (High) (release gemini-3-8-flash, row gemini-3-8-flash, UUID 8a999846-4c1d-4ce7-a8b7-1310a7166fd7, reasoning=true, source reasoning-token configuration budget=0, suffix=High). This configured value is not measured token consumption.. Source model ID: artificial-analysis/8a999846-4c1d-4ce7-a8b7-1310a7166fd7/gemini-3-8-flash/gemini-3-8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Model snapshot: Gemini 3.8 Flash (High). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: High. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: Chess Puzzles
Reasoning · Family index: 76.2 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-chess-puzzles
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 0.58 ratio
- Metric:
- exact
- Version:
- 2.0.0
- Participating models:
- 22
- Rank percentile within the cohort:
- 76.2 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Reasoning tier:
- high
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: high. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gemini-3.8-flash_high; Epoch model label: Gemini 3.8 Flash; Display label field: Gemini 3.8 Flash (high); Reasoning setting encoded in the alias: high. Task ID: Chess Puzzles. Task version: 2.0.0. Run ID: Hpyy6CpmXmd5q7Zbdq63CR. Started at 2026-10-13T12:39:05.000Z. Original task label: Chess Puzzles. The published row reports mean_score=0.58 and stderr=0.04960449637488583. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: high.
- Epoch AI benchmark run dataset snapshot
Test family: SimpleQA Verified
Knowledge · Family index: 83.3 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-simpleqa-verified
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 2.38 %
- Raw value:
- 0.697 ratio
- Metric:
- simpleqa_scorer
- Version:
- 1.2.0
- Participating models:
- 19
- Rank percentile within the cohort:
- 83.3 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["SimpleQA Verified","SimpleQA Verified (anti-abstention)","1.2.0","simpleqa_scorer"]
- Reasoning tier:
- high
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["SimpleQA Verified","SimpleQA Verified (anti-abstention)","1.2.0","simpleqa_scorer"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: high. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gemini-3.8-flash_high; Epoch model label: Gemini 3.8 Flash; Display label field: Gemini 3.8 Flash (high); Reasoning setting encoded in the alias: high. Task ID: SimpleQA Verified. Task version: 1.2.0. Run ID: gZ4bwozTKtpmZTqnqX3a33. Started at 2026-09-02T17:15:36.000Z. Original task label: SimpleQA Verified (anti-abstention). The published row reports mean_score=0.697 and stderr=0.014539683710535193. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: high.
- Epoch AI benchmark run dataset snapshot
Test family: FrontierMath
Mathematics · Family index: 30.2 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: GPQA Diamond
Reasoning · Family index: 92.1 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Humanity’s Last Exam
Reasoning · Family index: 86.6 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
Test family: LMArena Code Arena WebDev
Coding · Family index: 47.2 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LMArena Text, Style Control
General · Family index: 88.9 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: LMArena Vision, Style Control
Multimodal · Family index: 83.3 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MMMU-Pro
Multimodal · Family index: 95.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: SWE-bench Verified
Coding · Family index: 69.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 50 %
Test family: Terminal-Bench
Coding · Family index: 73.8 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 83 %
Test family: Terminal-Bench Science
Coding · Family index: 52.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: BioMysteryBench
General · Family index: 21.1 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: Code Migration
Coding · Family index: 58.1 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: CyberBench v1.1
Cybersecurity · Family index: 13.6 / 100Nominal family weight in the overall panel: 7.14 % · Coverage of this test family: 100 %
Test family: EMB
Finance · Family index: 89.8 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Finance Agent (v2)
Finance · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: IOI
Coding · Family index: 51.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LiveCodeBench v6
Coding · Family index: 97.4 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Harvey's Legal Agent Benchmark
Legal · Family index: 82.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: LegalBench
Legal · Family index: 93.6 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: Legal Research Bench
Legal · Family index: 63.3 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MedCode
Health · Family index: 67.2 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MedScribe
Health · Family index: 64.7 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MMLU-Pro
Knowledge · Family index: 94.8 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MortgageTax
Finance · Family index: 38.5 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: MysteryMechanism
General · Family index: 70.0 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: ProgramBench
Coding · Family index: 73.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: ProofBench
Mathematics · Family index: 41.0 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: SAGE
Education · Family index: 15.5 / 100Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: SkillsBench
Agents · Family index: 69.0 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
Test family: Public Benefits Bench v1.1
Productivity · Family index: 60.5 / 100Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: SRE Bench
General · Family index: 62.5 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: SWE-bench Multimodal
Coding · Family index: 23.1 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Tax Agent Bench
Finance · Family index: 78.2 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Family index: 68.4 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench 1-100
Coding · Family index: 52.4 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Family index: 73.6 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Benchmark sources at a glance
Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.
| Evaluator | Test series | Composite indices | Recorded active measurements |
|---|---|---|---|
| Artificial Analysis | 13 | 1 | 32 |
| Vals AI | 34 | 1 | 227 |
| LMArena | 3 | 0 | 41 |
| Epoch AI | 6 | 0 | 6 |
Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.
Page 1 of 28
Published benchmarks
Humanity's Last Exam
Overall
2,500 difficult, partly multimodal expert questions across more than 100 subjects. The benchmark measures closed-ended expert knowledge and reasoning, not open research or general intelligence.
Note (Gemini 3.8 Flash): Editorial selection from the vendor table, not a complete extract of the comparison cohort.
1.Gemini 3.7 Flash45.7%
Note: Editorial selection from the vendor table, not a complete extract of the comparison cohort.
higher is betterAs of September 2, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- google-gemini-38-hle-release
- Metric
- accuracy
- Scale
- 0-100
- Participants
- 2
- Methodology and context
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Google Cloud. Dataset type: public. Leaderboard date: 2026-09-02. Source ID: google-cloud. Evidence capture ID: curated-google-cloud-2026-09-02-abe383616de5. Source payload hash: abe383616de579a422bf59de0a0bfbbe19fc7feea92553c2cbc4ff5d33466fc7. Parser version: curated-v1. Imported at: 2026-10-15
Agent tasks
Terminal-Bench 2.1
Overall
Note (Gemini 3.8 Flash): Editorial selection from the vendor table, not a complete extract of the comparison cohort.
2.Gemini 3.7 Flash81.6%
Note: Editorial selection from the vendor table, not a complete extract of the comparison cohort.
higher is betterAs of September 2, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- google-gemini-38-terminal-bench-2-1-release
- Metric
- pass@1
- Scale
- 0-100
- Participants
- 2
- Methodology and context
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Google Cloud. Dataset type: public. Leaderboard date: 2026-09-02. Source ID: google-cloud. Evidence capture ID: curated-google-cloud-2026-09-02-0899d9ea04b8. Source payload hash: 0899d9ea04b8bc3ad8e9b37e004ed90bfa495ad9ccec45a2ad7927deb52e8eed. Parser version: curated-v1. Imported at: 2026-10-15
Terminal-Bench 4.0
Overall
Agentic evaluation of tasks in reproducible terminal environments. The agent, tools, environment, and reasoning level are part of the result. Scores from older Terminal-Bench versions are not directly comparable.
Note (Gemini 3.8 Flash): Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
2.Gemini 3.7 Flash11.21%
Note: Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Metric
- trial pass rate
- Scale
- 0-100
- Participants
- 2
- Methodology and context
- 330 graded trials, mini-SWE-agent agent, high reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: mini-SWE-agent. Attempts: 330 graded trials. Reasoning effort: high. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-1290fc94851c. Source payload hash: 1290fc94851c07e4997be903e42b324b2b37e0fc0a2ef9b70b34656a832eadfe. Parser version: curated-v1. Imported at: 2026-10-15
General tasks
BioMysteryBench
| Sub-task | Gemini 3.8 Flash (This model) | Gemini 3.1 Pro Preview | Gemini 3.6 Flash |
|---|---|---|---|
| Hard | 21.57% | 27.45% (best value in this row) | 21.57% |
| Human-solvable | 71.69% (best value in this row) | 67.58% | 67.12% |
| Overall | 62.22% (best value in this row) | 60% | 58.52% |
higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows
Methodology and context
- Metric
- accuracy
Coding
Code Migration
| Sub-task | Gemini 3.8 Flash (This model) | Gemini 3.7 Flash | Gemini 3.6 Flash | Gemini 3.5 Flash | Gemini 3.1 Pro Preview | Gemini 3 Flash Preview | Gemini 3.5 Flash-Lite | Gemini 3.1 Flash-Lite |
|---|---|---|---|---|---|---|---|---|
| CLI | 25.87% (best value in this row) | 23.54% | 19.33% | 17.18% | 10.83% | 8.52% | 5.88% | 5.73% |
| COBOL | 68.57% (best value in this row) | 68.57% (best value in this row) | 65.71% | 55.46% | 36.75% | 0% | 6.96% | 1.25% |
| Overall | 36.55% (best value in this row) | 34.8% | 30.93% | 26.75% | 17.31% | 6.39% | 6.15% | 4.61% |
higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows
Methodology and context
- Metric
- accuracy
Code Migration
Code Quality
higher is betterAs of October 15, 2026Scale: 0 rating - 3.74 rating
Methodology and context
- Comparison cohort
- vals-code-migration:1:code_quality
- Metric
- mean reviewer code-quality rating
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 8
- Methodology and context
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 9443.316. Standard error: 0.053. Cost per task in USD: 18.491664. Score display: 3.742 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Recorded measurement details
Humanity's Last Exam: Overall (Show measurement, test conditions, and source)
- Source value
- 45.4
- Score
- 45.4
- Metric
- accuracy
- Unit
- %
- Benchmark version
- Gemini 3.8 Flash release table
- Category
- knowledge
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- google-gemini-38-hle-release
- Source type
- vendor-reported
- Evaluator
- Google Cloud
- Status
- active
- Retrieved at
- 2026-09-02
- Methodology
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Google Cloud. Dataset type: public. Leaderboard date: 2026-09-02. Source ID: google-cloud. Evidence capture ID: curated-google-cloud-2026-09-02-abe383616de5. Source payload hash: abe383616de579a422bf59de0a0bfbbe19fc7feea92553c2cbc4ff5d33466fc7. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.
- Context
- Editorial selection from the vendor table, not a complete extract of the comparison cohort.
Terminal-Bench 2.1: Overall (Show measurement, test conditions, and source)
- Source value
- 90.8
- Score
- 90.8
- Metric
- pass@1
- Unit
- %
- Benchmark version
- 2.1, Gemini 3.8 Flash release table
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- google-gemini-38-terminal-bench-2-1-release
- Source type
- vendor-reported
- Evaluator
- Google Cloud
- Status
- active
- Retrieved at
- 2026-09-02
- Methodology
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Google Cloud. Dataset type: public. Leaderboard date: 2026-09-02. Source ID: google-cloud. Evidence capture ID: curated-google-cloud-2026-09-02-0899d9ea04b8. Source payload hash: 0899d9ea04b8bc3ad8e9b37e004ed90bfa495ad9ccec45a2ad7927deb52e8eed. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Google-published comparison of Gemini 3.8 Flash and 3.7 Flash. Not a Gradually test.
- Context
- Editorial selection from the vendor table, not a complete extract of the comparison cohort.
Terminal-Bench 4.0: Overall (Show measurement, test conditions, and source)
- Source value
- 19.09
- Score
- 19.09
- Metric
- trial pass rate
- Unit
- %
- Benchmark version
- 4.0
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- terminal-bench-4-leaderboard-9f966760-00f1-424e-90f5-c964fb6f6091-2026-10-06
- Confidence interval
- 3.36
- Source type
- official-leaderboard
- Evaluator
- Terminal-Bench
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 330 graded trials, mini-SWE-agent agent, high reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.. Source model ID: curated/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Terminal-Bench. Dataset type: public. Leaderboard date: 2026-10-06. Agent: mini-SWE-agent. Attempts: 330 graded trials. Reasoning effort: high. Source ID: terminal-bench. Evidence capture ID: curated-terminal-bench-2026-10-15-1290fc94851c. Source payload hash: 1290fc94851c07e4997be903e42b324b2b37e0fc0a2ef9b70b34656a832eadfe. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 330 graded trials, mini-SWE-agent agent, high reasoning. Dataset version 1922072f-a433-429a-8929-350d5e1bcf02.
- Context
- Selection of exactly identified models from the official 35-row leaderboard. Agents and reasoning efforts differ. These values do not belong to the separate Vals Mini-SWE-agent cohort.
BioMysteryBench: Hard (Show measurement, test conditions, and source)
- Source value
- 21.569
- Score
- 21.569
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- human_difficult
- Task label
- Hard
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:human_difficult
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 576.331. Standard error: 7.07. Cost per task in USD: 2.896591. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
BioMysteryBench: Human-solvable (Show measurement, test conditions, and source)
- Source value
- 71.689
- Score
- 71.689
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- human_solvable
- Task label
- Human-solvable
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:human_solvable
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 267.243. Standard error: 1.826. Cost per task in USD: 1.233913. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
BioMysteryBench: Overall (Show measurement, test conditions, and source)
- Source value
- 62.222
- Score
- 62.222
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- vals-biomysterybench:1:overall
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 326.755. Standard error: 1.697. Cost per task in USD: 1.5539. Source ID: vals-ai. Evidence capture ID: vals-biomysterybench-2026-10-15-59bf79d38ec4. Source payload hash: 59bf79d38ec415a34ffb1c2901a16ecbbe101b7fa864e86104d2571fc7cfc23d. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Harness: Terminus 2. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
Code Migration: CLI (Show measurement, test conditions, and source)
- Source value
- 25.871
- Score
- 25.871
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- cli
- Task label
- CLI
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:cli
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 10040.43. Standard error: 2.456. Cost per task in USD: 22.86835. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
Code Migration: COBOL (Show measurement, test conditions, and source)
- Source value
- 68.571
- Score
- 68.571
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- cobol
- Task label
- COBOL
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:cobol
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 7651.975. Standard error: 15.028. Cost per task in USD: 5.361607. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
Code Migration: Overall (Show measurement, test conditions, and source)
- Source value
- 36.546
- Score
- 36.546
- Metric
- accuracy
- Unit
- %
- Benchmark version
- 1
- Category
- coding
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:overall
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 9443.316. Standard error: 4.184. Cost per task in USD: 18.491664. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
Code Migration: Code Quality (Show measurement, test conditions, and source)
- Source value
- 3.742
- Score
- 3.742
- Metric
- mean reviewer code-quality rating
- Unit
- rating
- Benchmark version
- 1
- Category
- coding
- Task
- code_quality
- Task label
- Code Quality
- Direction
- Higher is better
- Comparison cohort
- vals-code-migration:1:code_quality
- Source type
- independent-evaluator
- Evaluator
- Vals AI
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Source model ID: google/gemini-3.8-flash. Source model name: Gemini 3.8 Flash. Provider reported by source: Google. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536. Latency in seconds: 9443.316. Standard error: 0.053. Cost per task in USD: 18.491664. Score display: 3.742 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
- Comparison methodology
- Reasoning effort: high. Temperature: 1. Maximum output tokens: 65536
Page 1 of 28
Our tests
SVG short test
One SVG prompt and fixed output settings. Captures are shown only with 0 reported reasoning tokens or documented lack of reasoning controls. Identical test conditions do not make the models' internal compute identical.
Earlier tests
Pelican on a bicycle
A community classic for free-form SVG drawing.
Gemini 3.8 Flash
The response reached the token limit. No complete recording is available.
Reasoning (requested): high
Current attempt
- Run date
- Oct 5, 2026
- API provider
- openrouter
- Requested API model
- google/gemini-3.8-flash
- API model reported in response
- google/gemini-3.8-flash
- Reasoning setting
- high
- Pinned endpoint
- Google AI Studio | google/gemini-3.8-flash-20260902
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Test protocol
- community-visual-tests-v1
- Technical checks
- Failed
- Response time
- 40.2 s
Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.
Earlier attempts (3)
- Oct 2, 2026API request failedReasoning setting: high
- Oct 1, 2026API request failedReasoning setting: high
- Sep 7, 2026API request failedReasoning setting: high
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)Popularity
57.6 / 100
Specifications and access
| Specification | Value and source |
|---|---|
| Weights | ProprietarySource |
| Access | APISource |
| Context window | 1,048,576 tokensSource |
| Knowledge cutoff | March 2026Source |
| Notes | Released September 2, 2026. Multimodal Flash model with a 1,048,576-token context window, up to 65,536 output tokens, and three reasoning levels. Google does not publish its parameter count or architecture.Source |
| API model ID | gemini-3.8-flashSource |
| Source status | stableSource |
| Maximum output | 65,536 tokensSource |
| Input | Text, Image, Video, Audio, pdfSource |
| Output | TextSource |
| Reasoning mode | low, medium, highSource |
| Tool calling | YesSource |
| Structured output | YesSource |
| Release | 2026-09-02Source |
| Knowledge cutoff | 2026-03Source |
| Capabilities | function calling, structured outputs, search grounding, Google Maps grounding, URL context, code execution, file search, computer useSource |