Language modelActive

Phi-4

Microsoft

Released: December 2024
Data date: September 8, 2026
Predecessor: Phi-3 (Official documentation)

Benchmark-Meta-Score: Unavailable
Fewer than eight test series
Speed: 41.1 tokens/s
Output throughput
Price: $0.07 · $0.14
Input · Output / 1M tokens
Input: Text
Output: Text

Phi-4 combines 14 billion dense parameters with a 16,384-token context. Microsoft trained the checkpoint on 9.8 trillion tokens, including a substantial share of synthetic and curated data. The weights use the MIT license.

Microsoft describes Phi-4 as a development of the Phi-3 line with only small architectural changes. The gains are attributed mainly to data mixture, curriculum, and post-training. The model card identifies English as the primary language.

Specifications and access

Specification	Value and source
Weights	Open weightsSource
Access	API, LocalSource
Parameters	14BSource
Context window	16,384 tokensSource
Knowledge cutoff	June 2024Source
Notes	Trained on 9.8T tokens, excels at math/reasoningSource
License	MITSource
Release	2024-12-12Source
Parameters	14 BSource
Knowledge cutoff	2024-06Source
Training tokens	9.8TSource
Architecture	dense decoder-only TransformerSource
Input	TextSource
Output	TextSource
Languages	EnglishSource

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-10-19 - 2026-10-19

Overall index

Not enough independent results for a score yet

An overall score needs at least eight test series.

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Test series: 1
Test families: 1 / 65
Sources: 1
Share of the fixed test panel (informative): 2 %

General

Index below 1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.0
95% interval: 0.0 to 0.0
Test series: 1
Test families: 1 / 5
Sources: 1
Share of the fixed test panel (informative): 20 %

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v3

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. Unknown or ambiguous tiers exclude the model from that series. Within each series, percentages and rates are logit-transformed, other units stay unchanged, and all values are then standardized. A one-factor model then estimates one shared strength per model and, per series, how sharply it separates the models. Area scores move away from that strength only as far as the model's own results in the area support. The overall score weights the core areas at 75% and finance, law and health at 25%. It requires at least eight test series from two sources and three areas. The 0 to 100 scale refers to a fixed reference group of 29 language models whose average sits at 50.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured series are not entered as zero. Where a model has no own results in a core area, its overall strength stands in, and the note “estimated without own results in …” names the area. Fewer test series lead to a larger standard error.

The 95% interval follows from the standard error of 200 recalculations with randomly redrawn test families. It describes how precisely the recorded test series determine the value. It is no prediction for future tasks or for later recalculations with other participants.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 1 test families and 1 of 1 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: IFBench
General · Mean rank percentile of the family: 0.0 / 100
Family weight in the fixed test panel (informative): 2.14 % · Coverage of this test family: 100 %
- Artificial Analysis
  Published configurations differ within this cohort.
  Measurement:
  artificial-analysis-ifbench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.14 %
  Raw value:
  23.5374149659864 %
  Metric:
  IFBench prompt-level loose accuracy (pass@1)
  Version:
  AA test set: 294 questions
  Participating models:
  70
  Rank percentile within the cohort:
  0.0 %
  Retrieved:
  2026-10-19
  Comparison cohort:
  artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:tokens=0:suffix=unspecified
  Documented comparison setup
  Publication cohort:
  ["artificial-analysis-ifbench","AA test set: 294 questions","aa-model-page-2026-10-19-8a022369738c"]
  Dataset type:
  artificial-analysis-evaluation
  Tool policy:
  Single-turn answer evaluated with the official IFBench source code
  Attempts:
  pass@1, 5 repeats per question
  Setup and model parameters
  Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning
  Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/ee708f92-374e-4123-b900-e22d7b2afc19/phi-4/phi-4. Source model name: Phi-4. Provider reported by source: Microsoft. Model snapshot: Phi-4. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Score display: 23.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	2	1	11

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

6 measurements from these sources still lack a verified test-family identity and do not increase the test-series count.

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

IFBench: Overall (Show measurement, test conditions, and source)

Source value: 23.5374149659864
Score: 23.5374149659864
Metric: IFBench prompt-level loose accuracy (pass@1)
Unit: %
Benchmark version: AA test set: 294 questions
Category: general
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:tokens=0:suffix=unspecified
Source type: independent-evaluator
Evaluator: Artificial Analysis
Status: active
Retrieved at: 2026-10-19
Methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/ee708f92-374e-4123-b900-e22d7b2afc19/phi-4/phi-4. Source model name: Phi-4. Provider reported by source: Microsoft. Model snapshot: Phi-4. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Score display: 23.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Comparison methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning
Context: The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.

Source Artificial Analysis

AA-Omniscience Accuracy: Overall (Show measurement, test conditions, and source)

Source value: 14.066666666666666
Score: 14.066666666666666
Metric: AA-Omniscience Accuracy
Unit: %
Benchmark version: AA Omniscience diagnostic
Category: knowledge
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: artificial-analysis-omniscience-accuracy:AA Omniscience diagnostic:standalone:non-reasoning:tokens=0:suffix=unspecified
Source type: independent-evaluator
Evaluator: Artificial Analysis
Status: active
Retrieved at: 2026-10-19
Methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Proportion of all questions answered correctly, regardless of whether the model answers or abstains on other questions.. Source model ID: artificial-analysis/ee708f92-374e-4123-b900-e22d7b2afc19/phi-4/phi-4. Source model name: Phi-4. Provider reported by source: Microsoft. Model snapshot: Phi-4. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Attempts: AA-reported aggregate; diagnostic-specific run count is not stated on the leaderboard. Reasoning: non-reasoning. Score display: 14.1%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Comparison methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Proportion of all questions answered correctly, regardless of whether the model answers or abstains on other questions.. Attempts: AA-reported aggregate; diagnostic-specific run count is not stated on the leaderboard. Reasoning: non-reasoning
Context: This is the separately published accuracy diagnostic, not the AA-Omniscience Index. AA identifies higher values as better.

Source Artificial Analysis

AA-Omniscience Hallucination Rate: Overall (Show measurement, test conditions, and source)

Source value: 81.18696664080683
Score: 81.18696664080683
Metric: AA-Omniscience Hallucination Rate
Unit: %
Benchmark version: AA Omniscience diagnostic
Category: knowledge
Task: overall
Task label: Overall
Direction: Lower is better
Comparison cohort: artificial-analysis-omniscience-hallucination-rate:AA Omniscience diagnostic:standalone:non-reasoning:tokens=0:suffix=unspecified
Source type: independent-evaluator
Evaluator: Artificial Analysis
Status: active
Retrieved at: 2026-10-19
Methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Proportion of incorrect answers among all non-correct responses: incorrect / (incorrect + partial answers + not attempted).. Source model ID: artificial-analysis/ee708f92-374e-4123-b900-e22d7b2afc19/phi-4/phi-4. Source model name: Phi-4. Provider reported by source: Microsoft. Model snapshot: Phi-4. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Attempts: AA-reported aggregate; diagnostic-specific run count is not stated on the leaderboard. Reasoning: non-reasoning. Score display: 81.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
Comparison methodology: Exact AA configuration: Phi-4 (release phi-4, row phi-4, UUID ee708f92-374e-4123-b900-e22d7b2afc19, reasoning=false, source reasoning-token configuration budget=0, suffix=not specified). This configured value is not measured token consumption. Proportion of incorrect answers among all non-correct responses: incorrect / (incorrect + partial answers + not attempted).. Attempts: AA-reported aggregate; diagnostic-specific run count is not stated on the leaderboard. Reasoning: non-reasoning
Context: This is the separately published hallucination-rate diagnostic, not the AA-Omniscience Index. AA identifies lower values as better.

Source Artificial Analysis

MMLU (Show measurement, test conditions, and source)

Source value: 84.8
Score: 84.8
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

GPQA (Show measurement, test conditions, and source)

Source value: 56.1
Score: 56.1
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

MGSM (Show measurement, test conditions, and source)

Source value: 80.6
Score: 80.6
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

MATH (Show measurement, test conditions, and source)

Source value: 80.4
Score: 80.4
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

HumanEval (Show measurement, test conditions, and source)

Source value: 82.6
Score: 82.6
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

SimpleQA (Show measurement, test conditions, and source)

Source value: 3
Score: 3
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

DROP (Show measurement, test conditions, and source)

Source value: 75.5
Score: 75.5
Metric: percent
Unit: %
Category: mixed
Direction: Higher is better
Source type: vendor-reported
Evaluator: Microsoft Phi-4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Vendor card evaluation; card says OpenAI SimpleEval and internal benchmarks, but exact shots are not stated in the visible table.

Source Microsoft Phi-4 model card

Artificial Analysis Intelligence Index (Show measurement, test conditions, and source)

Source value: 5.92257028113751
Score: 5.92257028113751
Metric: points
Unit: points
Direction: Higher is better
Source type: independent-evaluator
Evaluator: Artificial Analysis
Status: active
Retrieved at: 2026-09-08
Methodology: Independent evaluation. unknown

Source Artificial Analysis

AA-Omniscience Index (Show measurement, test conditions, and source)

Source value: -55.7
Score: -55.7
Metric: points
Unit: points
Direction: Higher is better
Source type: independent-evaluator
Evaluator: Artificial Analysis
Status: active
Retrieved at: 2026-09-08
Methodology: Independent evaluation. unknown

Source Artificial Analysis

Page 1 of 2

Predecessor and successor

Sources and data date

Type	Evidence and data date
Speed	Microsoft. Output speed after the first streaming chunk, not total response time. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · Microsoft Phi-4 model card · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · Microsoft Phi-4 model card · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 2 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	active-parameters-billions Original wording The exact Phi-4 model is dense; the card does not publish a separate active-parameter count.
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Artificial Analysis (retrieved October 19, 2026)
Additional source	Phi-4 technical paper · Editorial description reviewed October 2, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0

Phi-4

Specifications and access

Overall and area scores

Overall index

General

Test family: IFBench

Benchmark sources at a glance

Measurements without matching peer values

Our tests

SVG short test

Phi-4

Pelican on a bicycle

Popularity

Published prices

Predecessor and successor

Sources and data date