MicrosoftLanguage model
Phi-4-mini
Phi-4-mini has no overall index value yet because there are too few independent benchmark results.
PopularityRank 57 of 785
55out of 100+13 vs. last month
based on search, usage and attention
Summary
New in Phi-4-mini are a vocabulary enlarged to about 200,000 tokens, grouped-query attention, and shared input and output embeddings. Microsoft lists 23 supported languages. The open 3.8B instruction checkpoint has a 128K-token context and was trained with synthetic and filtered public data, SFT, and DPO.
Microsoft warns that the 3.8B model has limited factual knowledge.
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 59 published benchmark results
Page 1 of 5
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
AIEval DoW Knowledge (Show measurement, test conditions, and source)
- Source value
- 80.2
- Score
- 80.2
- Metric
- share of items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- knowledge
- Direction
- Higher is better
- Sample
- 96
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16, 1,024-token answer cap, 512-token prompt budget, 1,536-token context, offline single run on RTX 4070 Super. No separate thinking cap is reported. Free-form items are re-judged by Granite 4.2 3B under the same-family rule.
- Context
- AI Eval reports its ≥80% human-agreement check as pending. The 88 judge-graded items across this model's three suites were graded by Granite 4.2 3B. The source cautions that differences under about 1 point are not meaningful at these item counts; no CI is published.
AIEval Honest Broker (Show measurement, test conditions, and source)
- Source value
- 16.7
- Score
- 16.7
- Metric
- share of items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- calibration
- Direction
- Higher is better
- Sample
- 48
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16, 1,024-token answer cap, 512-token prompt budget, 1,536-token context, offline single run on RTX 4070 Super. No separate thinking cap is reported. Free-form items are re-judged by Granite 4.2 3B under the same-family rule.
- Context
- AI Eval reports its ≥80% human-agreement check as pending. The 88 judge-graded items across this model's three suites were graded by Granite 4.2 3B. The source cautions that differences under about 1 point are not meaningful at these item counts; no CI is published. Judge strictness differs and the evaluator warns against direct comparison with the other seven models.
AIEval Lawful Order (Show measurement, test conditions, and source)
- Source value
- 80.8
- Score
- 80.8
- Metric
- harmonic mean of unlawful-request refusal and lawful-task helpfulness
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- safety
- Direction
- Higher is better
- Sample
- 40
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16, 1,024-token answer cap, 512-token prompt budget, 1,536-token context, offline single run on RTX 4070 Super. No separate thinking cap is reported. Free-form items are re-judged by Granite 4.2 3B under the same-family rule.
- Context
- AI Eval reports its ≥80% human-agreement check as pending. The 88 judge-graded items across this model's three suites were graded by Granite 4.2 3B. The source cautions that differences under about 1 point are not meaningful at these item counts; no CI is published. The 8 gray-area items are excluded from the headline and its 40-question metric denominator.
AIEval DoW Bench Score (Show measurement, test conditions, and source)
- Source value
- 59.2
- Score
- 59.2
- Metric
- Equal-weight average of the DoW Knowledge, Honest Broker, and Lawful Order headline scores
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013; generation 42532423; re-judged and rescored c52abb57
- Category
- source-specific
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact model Microsoft/Phi-4-mini-instruct, revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; one offline run on one RTX 4070 Super 12 GB Windows workstation; run ID 9bf1f0609013. The evaluator defines this published score as the equal-weight mean of the three suite headline scores, not an item-weighted mean. The run used a 1,024-token answer cap, 512-token prompt budget and 1,536-token context; this model has no separate thinking cap. The 88 judge-graded items were graded by Granite 4.2 3B under the same-family rule. The human-agreement check remains pending.
- Context
- This imports the evaluator's named aggregate, not a locally calculated score or a new comparison cohort. The page prints 59.2% and the equation (80.2% + 16.7% + 80.8%) / 3 = 0.5922; the displayed rounded inputs average to 59.233...%, so the exact unrounded suite values behind the stated aggregate are not disclosed. The published 59.2% is retained without recalculation. The evaluator says differences below about one point are not meaningful at these item counts. No confidence interval or separate aggregate sample count is published. Human agreement for judged items is pending; the result page says free-form results, especially Honest Broker, are not strictly comparable with other models judged by Phi-4-mini-instruct.
AIEval DoW Knowledge submetric: multiple choice (Show measurement, test conditions, and source)
- Source value
- 94.4
- Score
- 94.4
- Metric
- Share of multiple-choice items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 72
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; deterministic multiple-choice submetric within the 96-item DoW Knowledge suite. Offline on one RTX 4070 Super 12 GB Windows workstation; one run, 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run; no confidence interval is published. The 96-item suite headline is already present in the current profile, so this row is only the distinct 72-item deterministic component.
AIEval DoW Knowledge submetric: short answer (Show measurement, test conditions, and source)
- Source value
- 37.5
- Score
- 37.5
- Metric
- Share of judge-graded short-answer items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013; judge rescore c52abb57
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 16
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; judge-graded short-answer submetric within DoW Knowledge. Granite 4.2 3B graded all 88 judge-graded items under the same-family rule. Offline on one RTX 4070 Super 12 GB Windows workstation; one run; 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run, no confidence interval and a pending human-agreement check. Small-judge strictness and the non-comparable free-form evaluation caveat remain applicable.
AIEval DoW Knowledge submetric: name the issuance (Show measurement, test conditions, and source)
- Source value
- 37.5
- Score
- 37.5
- Metric
- Share of issuance-identification items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 8
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; deterministic name-the-issuance submetric within the 96-item DoW Knowledge suite. Offline on one RTX 4070 Super 12 GB Windows workstation; one run, 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run and no confidence interval. This is an eight-item submetric, not another DoW Knowledge headline score.
AIEval Honest Broker submetric: fake-issuance abstention (Show measurement, test conditions, and source)
- Source value
- 0
- Score
- 0
- Metric
- Share of fake-issuance abstention items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 8
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; deterministic fake-issuance-abstention submetric within the 48-item Honest Broker suite. Offline on one RTX 4070 Super 12 GB Windows workstation; one run, 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run and no confidence interval. This deterministic component is part of the existing 48-item Honest Broker headline, not another full-suite result.
AIEval Honest Broker submetric: citation validity (Show measurement, test conditions, and source)
- Source value
- 37.5
- Score
- 37.5
- Metric
- Share of citation-validity items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 8
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; deterministic citation-validity submetric within the 48-item Honest Broker suite. Offline on one RTX 4070 Super 12 GB Windows workstation; one run, 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run and no confidence interval. This deterministic component is part of the existing 48-item Honest Broker headline, not another full-suite result.
AIEval Honest Broker submetric: sycophancy pairs (Show measurement, test conditions, and source)
- Source value
- 12.5
- Score
- 12.5
- Metric
- Share of sycophancy-pair items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 8
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; deterministic sycophancy-pair submetric within the 48-item Honest Broker suite. Offline on one RTX 4070 Super 12 GB Windows workstation; one run, 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run and no confidence interval. This deterministic component is part of the existing 48-item Honest Broker headline, not another full-suite result.
AIEval Honest Broker submetric: rubric (Show measurement, test conditions, and source)
- Source value
- 16.7
- Score
- 16.7
- Metric
- Share of judge-graded rubric items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013; judge rescore c52abb57
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 24
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; judge-graded rubric submetric within the 48-item Honest Broker suite. Granite 4.2 3B graded the 24 rubric items under the same-family rule. Offline on one RTX 4070 Super 12 GB Windows workstation; one run; 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run, no confidence interval and a pending human-agreement check. Small-judge strictness and the non-comparable free-form evaluation caveat remain applicable.
AIEval Lawful Order submetric: refusal correctness (Show measurement, test conditions, and source)
- Source value
- 87.5
- Score
- 87.5
- Metric
- Share of unlawful-request refusal items passed
- Unit
- %
- Benchmark version
- AIEval DoW Bench v1.1.1 stage1; run 9bf1f0609013; judge rescore c52abb57
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 24
- Source type
- independent-evaluator
- Evaluator
- AI Eval Corp, AIEval DoW Bench result page
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Independent evaluation. Exact Microsoft/Phi-4-mini-instruct revision cfbefacb99257ffa30c83adab238a50856ac3083; BF16; run 9bf1f0609013; judge-graded refusal-correctness component within the 48-item Lawful Order suite. Granite 4.2 3B graded the 24 unlawful-request items. Offline on one RTX 4070 Super 12 GB Windows workstation; one run; 1,024-token answer cap, 512-token prompt budget, 1,536-token context and no separate thinking cap.
- Context
- The evaluator reports a single run, no confidence interval and a pending human-agreement check. This component is one input to the already recorded 80.8% harmonic-mean Lawful Order headline; it is not a replacement for that headline.
Page 1 of 5
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 100 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 7, 2026
Phi-4-miniNemotron 3.5 Bolt E4B 55GLM-4.5 Air 55Nemotron 3 Ultra 55Solar Pro 4 55
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as Phi-4-mini, chosen by model line, developer, size, price and release date.
- Phi-4-multimodal-instructMicrosoft · same developer
- Phi-4Microsoft · same developer
- Phi-3 Medium (14B)Microsoft · same developer
- WizardLM-2 8x22BMicrosoft · same developer
- Gemma 3 4BGoogle · similar size
- Llama-3.2-3BMeta · similar size
- Llama 3.2 3BMeta · similar size
- Nemotron Mini 4B InstructNVIDIA · similar size
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
Frequently asked questions about Phi-4-mini
What is the knowledge cutoff of Phi-4-mini?
June 2024, according to Microsoft. That is 8 months before release.
Stated by Microsoft
JunAugOctDecFebKnowledge cutoffJune 2024ReleasedFeb 26, 2025
How large is the context window of Phi-4-mini?
128,000 tokens. That is how much text Phi-4-mini reads at once, about 190 A4 pages.
Page count as a rule of thumb from Artificial Analysis
GLM-5.1200K
DeepSeek-V3 0324164K
GLM-4.5131K
GPT OSS 120B131K
Phi-4-miniRank 98 of 103128K
DeepSeek-V3.2128K
DeepSeek-R1128K
Can I run Phi-4-mini locally?
Yes, the weights are open.
Stated by Microsoft
- Open weights
- yes
- License
- MIT
- Parameters
- 3.8B
- active per token
- n/a
Sources and data date
Show all 15 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.
Without vendor claims and historical or disputed results; those stay labelled in the single values.
| Type | Evidence and data date |
|---|---|
| Input | Documented modalities. Retrieved October 27, 2026. |
| Output | Documented modalities. Retrieved October 27, 2026. |
| Research status | Research date October 15, 2026. 8 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication. |
| Not published | benchmark-score-units-and-library-version Note The card calls this an internal benchmark platform and describes prompting exceptions, but omits numeric score units and platform version; raw values are retained. |
| Unresolved | human-judge-agreement Note The result page marks the human agreement check pending, with an 80% gate; free-form scores, especially Honest Broker, are not strictly comparable with the other seven rows graded by Phi-4-mini-instruct. |
| Not published | confidence-intervals Note The evaluator publishes no confidence interval for the aggregate, suite headlines or submetrics. |
| Not published | checkpoint-revision Original wording The paper identifies Phi-4-mini-instruct but publishes no immutable checkpoint revision or weight digest. |
| Not published | conditional-denominators Original wording Separate denominators for inertia and over-flip subsets are not published; sampleCount is omitted on those rows. |
| Not published | confidence-intervals Original wording No confidence intervals are reported for this row. |
| Not published | reasoning-effort-and-inference-settings Original wording Reasoning effort, quantization, and decoding settings are not specified. |
| Additional source | Hugging Face (retrieved October 15, 2026; October 19, 2026) · Phi-4-mini-instruct model card · Editorial description reviewed October 4, 2026 |
| Additional source | AI Eval Corp, AIEval DoW Bench (retrieved October 17, 2026) |
| Additional source | DeltaLogic: Minimal Premise Edits Reveal Belief-Revision Failures in Logical Reasoning Models (arXiv:2604.02733v1) (retrieved October 17, 2026) |
| Additional source | Truth or Mirage? Towards End-To-End Factuality Evaluation with LLM-OASIS (retrieved October 18, 2026) |
| Additional source | Ardana JevBench (retrieved October 22, 2026) |
| Additional source | FastFlowLM Performance Benchmarks (retrieved October 22, 2026) |
| Additional source | Model metadata (retrieved October 17, 2026) |
| Additional source | Model metadata (retrieved October 18, 2026) |
| Additional source | OpenRouter: Popularity index (derived), CC BY 4.0 |
| Additional source | Vercel AI Gateway: Popularity index (derived), CC BY 4.0 |
| Additional source | Ollama: cumulative downloads for the popularity index |
| Additional source | Replicate: cumulative runs for the popularity index |
| Additional source | Google Trending Now: limited feed selection for Germany and the US |
| Additional source | Creative Commons Attribution 4.0 |