GoogleLanguage model
Gemma 4 12B IT
Gemma 4 12B IT has no overall index value yet because there are too few independent benchmark results. From the vendor-reported results we estimate about 55.3 points (95% range 35.1 to 75.5). This is not an independent measurement.
PopularityRank 219 of 785
43out of 100same as last month
based on search, usage and attention
Summary
Google Gemma 4 12B IT is distinguished by the 12B Unified variant’s direct multimodal path without separate audio or vision encoders. Released on June 3, 2026, the checkpoint accepts text, image, audio, and video input, generates text, and uses Apache 2.0 weights.
The model card lists up to 32,768 output tokens and more than 140 training languages. Gemma 4 31B IT is larger and dense, while 12B Unified routes modalities more directly through one model.
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 42 published benchmark results
Page 1 of 4
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
CritPt v2: Overall (Show measurement, test conditions, and source)
- Source value
- 0
- Score
- 0
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- v2
- Category
- reasoning
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- disputed
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/025ec6b6-df4a-449f-9a03-24e4388d6863/gemma-4-12b/gemma-4-12b-non-reasoning. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Non-reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none. Score display: 0.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:non-reasoning:effort=unspecified:suffix=Non-reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none
- Context
- This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.
CritPt v2: Overall (Show measurement, test conditions, and source)
- Source value
- 0
- Score
- 0
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- v2
- Category
- reasoning
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- disputed
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/feb02d3b-ff8d-4ed4-b165-13f8d4a7192c/gemma-4-12b/gemma-4-12b. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled. Score display: 0.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled
- Context
- This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 45.1700680272109
- Score
- 45.1700680272109
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/025ec6b6-df4a-449f-9a03-24e4388d6863/gemma-4-12b/gemma-4-12b-non-reasoning. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Non-reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 45.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=Non-reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.. This result is retained as historical evidence and is not interchangeable with current results.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 45.1700680272109
- Score
- 45.1700680272109
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/025ec6b6-df4a-449f-9a03-24e4388d6863/gemma-4-12b/gemma-4-12b-non-reasoning. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Non-reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 45.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=Non-reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 73.5374149659864
- Score
- 73.5374149659864
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/feb02d3b-ff8d-4ed4-b165-13f8d4a7192c/gemma-4-12b/gemma-4-12b. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled. Score display: 73.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.. This result is retained as historical evidence and is not interchangeable with current results.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 73.5374149659864
- Score
- 73.5374149659864
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/feb02d3b-ff8d-4ed4-b165-13f8d4a7192c/gemma-4-12b/gemma-4-12b. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled. Score display: 73.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: reasoning enabled
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 61.965317919075105
- Score
- 61.965317919075105
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/025ec6b6-df4a-449f-9a03-24e4388d6863/gemma-4-12b/gemma-4-12b-non-reasoning. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Non-reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 62.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:non-reasoning:effort=unspecified:suffix=Non-reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.. This result is retained as historical evidence and is not interchangeable with current results.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 61.965317919075105
- Score
- 61.965317919075105
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/025ec6b6-df4a-449f-9a03-24e4388d6863/gemma-4-12b/gemma-4-12b-non-reasoning. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Non-reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 62.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:non-reasoning:effort=unspecified:suffix=Non-reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Non-reasoning) (release gemma-4-12b, row gemma-4-12b-non-reasoning, UUID 025ec6b6-df4a-449f-9a03-24e4388d6863, reasoning=false, effort=not specified, suffix=Non-reasoning). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 69.6531791907514
- Score
- 69.6531791907514
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/feb02d3b-ff8d-4ed4-b165-13f8d4a7192c/gemma-4-12b/gemma-4-12b. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: reasoning enabled. Score display: 69.7%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: reasoning enabled
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.. This result is retained as historical evidence and is not interchangeable with current results.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 69.6531791907514
- Score
- 69.6531791907514
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/feb02d3b-ff8d-4ed4-b165-13f8d4a7192c/gemma-4-12b/gemma-4-12b. Source model name: Gemma 4 12B IT. Provider reported by source: Google. Model snapshot: Gemma 4 12B (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: reasoning enabled. Score display: 69.7%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Gemma 4 12B (Reasoning) (release gemma-4-12b, row gemma-4-12b, UUID feb02d3b-ff8d-4ed4-b165-13f8d4a7192c, reasoning=true, effort=not specified, suffix=Reasoning). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: reasoning enabled
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.
MMLU Pro (Show measurement, test conditions, and source)
- Source value
- 77.2
- Score
- 77.2
- Metric
- percent
- Unit
- %
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Google Gemma 4 official model card
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Gemma 4 12B Unified instruction-tuned column; the card does not specify per-task prompt, decoding setup, or table run date.
AIME 2026 no tools (Show measurement, test conditions, and source)
- Source value
- 77.5
- Score
- 77.5
- Metric
- percent
- Unit
- %
- Benchmark version
- AIME 2026
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Google Gemma 4 official model card
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Gemma 4 12B Unified instruction-tuned; explicitly labeled no tools; other prompt and decoding details are not stated.
Page 1 of 4
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 101 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 8, 2026
Gemma 4 12B ITLing-3.0-flash-Fin 44Mercury 2 44Qwen 3 32B 43Muse Glimmer 30B 43
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
- 2November 1, 2026: Method version 4 (common scale, shrinkage)
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as Gemma 4 12B IT, chosen by model line, developer, size, price and release date.
- Gemma 4 E4B ITGoogle · same developer
- DiffusionGemma 26B A4B ITGoogle · same developer
- Gemma 3 12BGoogle · same developer
- Gemma 4 26B A4BGoogle · same developer
- Gemma 4 E2B ITGoogle · same developer
- Gemma 4 31BGoogle · same developer
- Gemini 3.8 FlashGoogle · same developer
- Gemini 3.5 Flash-LiteGoogle · same developer
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
- Shutdown
- None announced
- no source
Frequently asked questions about Gemma 4 12B IT
How good is Gemma 4 12B IT in benchmarks?
Independent results are not yet enough for the overall index. An estimate from 5 vendor-reported results (GPQA Diamond 78.8%, Humanity's Last Exam (no tools) 5.2%, LiveCodeBench v6 72%, MMLU-Pro 77.2%, MMMU-Pro 69.1%) gives about 55.3 points, with a 95% range of 35.1 to 75.5. The estimate is calibrated on models with both vendor-reported and independent results. Vendor numbers run clearly above independent measurements on some tests; the calibration offsets that on average.
Vendor estimate, not an overall index value
How large is the context window of Gemma 4 12B IT?
262,144 tokens. That is how much text Gemma 4 12B IT reads at once, about 390 A4 pages.
Page count as a rule of thumb from Artificial Analysis
Gemma 4 12B ITRank 67 of 103262K
Kimi K2.6262K
Qwen 3.8 27B262K
Kimi K2.7 Code262K
Kimi K2.5262K
Nemotron 3 Ultra262K
Kimi K2 Thinking262K
Can I run Gemma 4 12B IT locally?
Yes, the weights are open.
Stated by Google
- Open weights
- yes
- License
- Apache 2.0
- Parameters
- 12B
- active per token
- n/a
Sources and data date
Show all 17 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
| Evaluator | Test series | Composite indices | Recorded active measurements |
|---|---|---|---|
| Artificial Analysis | 3 | 0 | 5 |
Without vendor claims and historical or disputed results; those stay labelled in the single values.
| Type | Evidence and data date |
|---|---|
| Input | Documented modalities. Retrieved October 2, 2026. |
| Output | Documented modalities. Retrieved October 27, 2026. |
| Research status | Research date October 15, 2026. 5 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication. |
| Not published | benchmark-protocol-details Note The card identifies the instruction-tuned cohort and benchmark labels but omits per-task prompts, decoding settings, and evaluation dates. |
| Unresolved | ExtractBench measurement provenance Note The exact card shows Mean, Short, and Medium values and the named pipeline; the contributor PR identifies unified value F1 and says the overall mean covers 370 documents. The split-specific sample sizes, benchmark revision, immutable evaluated checkpoint, confidence interval, score scale, and full run settings are not stated. The captured card shows an additional unexpanded result control without exposing its label or value, so no hidden result is inferred. |
| Additional source | blog.google (retrieved October 2, 2026) · Google Gemma 4 12B release · Editorial description reviewed October 2, 2026 |
| Additional source | Artificial Analysis (retrieved November 2, 2026) |
| Additional source | Artificial Analysis (retrieved October 29, 2026; November 2, 2026; November 7, 2026) |
| Additional source | Artificial Analysis (retrieved November 2, 2026; November 7, 2026)12 |
| Additional source | Gemma 4 model card (retrieved October 17, 2026) |
| Additional source | Gemma Team, Google DeepMind (retrieved October 18, 2026) |
| Additional source | Releasing the Inner Jev: A Simple Recipe for Open Decision Models, Table 3 (retrieved October 19, 2026) |
| Additional source | Gemma Team, Google DeepMind (retrieved October 17, 2026; October 22, 2026) |
| Additional source | OpenRouter: Popularity index (derived), CC BY 4.0 |
| Additional source | Vercel AI Gateway: Popularity index (derived), CC BY 4.0 |
| Additional source | Ollama: cumulative downloads for the popularity index |
| Additional source | Replicate: cumulative runs for the popularity index |
| Additional source | Google Trending Now: limited feed selection for Germany and the US |
| Additional source | Creative Commons Attribution 4.0 |