MicrosoftLanguage model
Phi-3 Medium (14B)
Phi-3 Medium (14B) has no overall index value yet because there are too few independent benchmark results.
PopularityRank 572 of 785
28out of 100-2 vs. last month
based on search, usage and attention
Summary
Phi-3 Medium (14B) is Microsoft’s open instruction model for substantial capability under constrained compute and latency budgets. The model card distinguishes 4K and 128K variants. The long-context 128K checkpoint uses synthetic and filtered public data followed by SFT and DPO, described as targeting code, mathematics, logic, and instruction following.
The large window distinguishes the 128K variant from the short Phi-3 configuration. The model card describes this instruction-tuned checkpoint for broad English commercial and research use.
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 39 published benchmark results
Page 1 of 4
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
BigCodeBench Complete (Show measurement, test conditions, and source)
- Source value
- 48.7
- Score
- 48.7
- Metric
- complete pass@1 as published
- Unit
- %
- Benchmark version
- BigCodeBench official result; complete mode; 2024-05-21
- Category
- coding
- Direction
- Higher is better
- Source type
- official-leaderboard
- Evaluator
- BigCodeBench official published results dataset
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Official benchmark leaderboard. Exact Phi-3-Medium-128K-Instruct row; `complete` and `instruct` modes kept separate; published table date 2024-05-21.
- Context
- Published dataset row, not a fresh reproduction; immutable model commit is not specified.
BigCodeBench Instruct (Show measurement, test conditions, and source)
- Source value
- 37.6
- Score
- 37.6
- Metric
- instruct pass@1 as published
- Unit
- %
- Benchmark version
- BigCodeBench official result; instruct mode; 2024-05-21
- Category
- coding
- Direction
- Higher is better
- Source type
- official-leaderboard
- Evaluator
- BigCodeBench official published results dataset
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Official benchmark leaderboard. Exact Phi-3-Medium-128K-Instruct row; `complete` and `instruct` modes kept separate; published table date 2024-05-21.
- Context
- Published dataset row, not a fresh reproduction; immutable model commit is not specified.
BigCodeBench-Hard Complete (Show measurement, test conditions, and source)
- Source value
- 15.5
- Score
- 15.5
- Metric
- complete pass@1 as published
- Unit
- %
- Benchmark version
- BigCodeBench-Hard official result; complete mode; 2024-05-21
- Category
- coding
- Direction
- Higher is better
- Source type
- official-leaderboard
- Evaluator
- BigCodeBench-Hard official published results dataset
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Official benchmark leaderboard. Exact Phi-3-Medium-128K-Instruct BigCodeBench-Hard row; `complete` and `instruct` modes kept separate; published table date 2024-05-21.
- Context
- Published dataset row, not a fresh reproduction; immutable model commit is not specified.
BigCodeBench-Hard Instruct (Show measurement, test conditions, and source)
- Source value
- 10.8
- Score
- 10.8
- Metric
- instruct pass@1 as published
- Unit
- %
- Benchmark version
- BigCodeBench-Hard official result; instruct mode; 2024-05-21
- Category
- coding
- Direction
- Higher is better
- Source type
- official-leaderboard
- Evaluator
- BigCodeBench-Hard official published results dataset
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Official benchmark leaderboard. Exact Phi-3-Medium-128K-Instruct BigCodeBench-Hard row; `complete` and `instruct` modes kept separate; published table date 2024-05-21.
- Context
- Published dataset row, not a fresh reproduction; immutable model commit is not specified.
AGI Eval (Show measurement, test conditions, and source)
- Source value
- 49.7
- Score
- 49.7
- Metric
- 5-shot accuracy
- Unit
- %
- Benchmark version
- AGI Eval, Phi-3 Medium 128K Instruct model-card table
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card at pinned revision
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Microsoft's evaluation table reports 5-shot accuracy; the card omits per-task scored N, confidence intervals, and a pinned benchmark-code revision.
- Context
- Vendor-reported. The card does not publish per-task scored N, confidence intervals, or complete implementation and decoding details.
TriviaQA (Show measurement, test conditions, and source)
- Source value
- 73.9
- Score
- 73.9
- Metric
- 5-shot accuracy
- Unit
- %
- Benchmark version
- TriviaQA, Phi-3 Medium 128K Instruct model-card table
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card at pinned revision
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Microsoft's evaluation table reports 5-shot accuracy; the card omits per-task scored N, confidence intervals, and a pinned benchmark-code revision.
- Context
- Vendor-reported. The card does not publish per-task scored N, confidence intervals, or complete implementation and decoding details.
GSM8K Chain of Thought (Show measurement, test conditions, and source)
- Source value
- 87.5
- Score
- 87.5
- Metric
- 8-shot accuracy
- Unit
- %
- Benchmark version
- GSM8K Chain of Thought, Phi-3 Medium 128K Instruct model-card table
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card at pinned revision
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Microsoft's evaluation table reports 8-shot accuracy; the card omits per-task scored N, confidence intervals, and a pinned benchmark-code revision.
- Context
- Vendor-reported. The card does not publish per-task scored N, confidence intervals, or complete implementation and decoding details.
HumanEval (Show measurement, test conditions, and source)
- Source value
- 58.5
- Score
- 58.5
- Metric
- 0-shot pass rate
- Unit
- %
- Benchmark version
- HumanEval, Phi-3 Medium 128K Instruct model-card table
- Category
- coding
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card at pinned revision
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Microsoft's evaluation table reports 0-shot pass rate; the card omits per-task scored N, confidence intervals, and a pinned benchmark-code revision.
- Context
- Vendor-reported. The card does not publish per-task scored N, confidence intervals, or complete implementation and decoding details.
MBPP (Show measurement, test conditions, and source)
- Source value
- 73.8
- Score
- 73.8
- Metric
- 3-shot pass rate
- Unit
- %
- Benchmark version
- MBPP, Phi-3 Medium 128K Instruct model-card table
- Category
- coding
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card at pinned revision
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Microsoft's evaluation table reports 3-shot pass rate; the card omits per-task scored N, confidence intervals, and a pinned benchmark-code revision.
- Context
- Vendor-reported. The card does not publish per-task scored N, confidence intervals, or complete implementation and decoding details.
MMLU (Show measurement, test conditions, and source)
- Source value
- 76.6
- Score
- 76.6
- Metric
- Reported few-shot benchmark score
- Unit
- %
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card
- Status
- active
- Retrieved at
- 2026-10-22
- Methodology
- Vendor-reported result. Microsoft reports the same internal evaluation pipeline for all compared models. Evaluation uses the per-row few-shot count, temperature 0, and no Phi-3-specific prompt, example, or format optimization.
- Context
- The model card gives neither per-task sample sizes nor confidence intervals or an immutable benchmark-code revision. The read card was on the current main path, not an accessible commit pin.
BigBench Hard (Show measurement, test conditions, and source)
- Source value
- 77.9
- Score
- 77.9
- Metric
- Reported few-shot benchmark score
- Unit
- %
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card
- Status
- active
- Retrieved at
- 2026-10-22
- Methodology
- Vendor-reported result. Microsoft reports the same internal evaluation pipeline for all compared models. Evaluation uses the per-row few-shot count, temperature 0, and no Phi-3-specific prompt, example, or format optimization.
- Context
- The model card gives neither per-task sample sizes nor confidence intervals or an immutable benchmark-code revision. The read card was on the current main path, not an accessible commit pin.
ANLI (Show measurement, test conditions, and source)
- Source value
- 57.3
- Score
- 57.3
- Metric
- Reported few-shot benchmark score
- Unit
- %
- Category
- general
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Microsoft Phi-3 Medium 128K Instruct model card
- Status
- active
- Retrieved at
- 2026-10-22
- Methodology
- Vendor-reported result. Microsoft reports the same internal evaluation pipeline for all compared models. Evaluation uses the per-row few-shot count, temperature 0, and no Phi-3-specific prompt, example, or format optimization.
- Context
- The model card gives neither per-task sample sizes nor confidence intervals or an immutable benchmark-code revision. The read card was on the current main path, not an accessible commit pin.
Page 1 of 4
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 101 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 7, 2026
Phi-3 Medium (14B)Granite 4.0 H Micro 28Llama 3.1 8B 28Trinity Mini 28Qwen 3 235B A22B Thinking 2507 28
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as Phi-3 Medium (14B), chosen by model line, developer, size, price and release date.
- Phi-4Microsoft · same developer
- Phi-4-multimodal-instructMicrosoft · same developer
- Phi-4-miniMicrosoft · same developer
- WizardLM-2 8x22BMicrosoft · same developer
- StarCoder2 15BBigCode · similar size
- Pixtral 12BMistral AI · similar size
- OLMo 2 13BAllen Institute · similar size
- Qwen 3 14BAlibaba · similar size
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
Frequently asked questions about Phi-3 Medium (14B)
How large is the context window of Phi-3 Medium (14B)?
131,072 tokens. That is how much text Phi-3 Medium (14B) reads at once, about 200 A4 pages.
Page count as a rule of thumb from Artificial Analysis
GLM-5.1200K
DeepSeek-V3 0324164K
Phi-3 Medium (14B)Rank 94 of 103131K
GLM-4.5131K
GPT OSS 120B131K
DeepSeek-V3.2128K
DeepSeek-R1128K
Can I run Phi-3 Medium (14B) locally?
Yes, the weights are open.
Stated by Microsoft
- Open weights
- yes
- License
- MIT
- Parameters
- 14B
- active per token
- n/a
Sources and data date
Show all 20 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.
Without vendor claims and historical or disputed results; those stay labelled in the single values.
| Type | Evidence and data date |
|---|---|
| Input | Documented modalities. |
| Output | Documented modalities. |
| Research status | Research date October 15, 2026. 14 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication. |
| Not published | checkpoint-revision Note The Open LLM result JSON identifies the exact repository/model name, but does not pin an immutable model commit; table date is preserved. |
| Not published | task-level-evaluation-details Note The pinned card provides few-shot counts and scores, but not per-task scored N, confidence intervals, or complete implementation and decoding settings. |
| Additional source | Open LLM Leaderboard v1 evaluation result snapshot (retrieved October 15, 2026) |
| Additional source | BigCodeBench official published results dataset (retrieved October 15, 2026) |
| Additional source | BigCodeBench-Hard official published results dataset (retrieved October 15, 2026) |
| Additional source | Microsoft Phi-3 Medium 128K Instruct model card at pinned revision (retrieved October 17, 2026) |
| Additional source | LongBioBench maintainers, results table (retrieved October 19, 2026) |
| Additional source | BigCodeBench official leaderboard (retrieved October 28, 2026)12 |
| Additional source | Model metadata (retrieved October 15, 2026) |
| Additional source | Model metadata (retrieved October 17, 2026) |
| Additional source | Model metadata (retrieved October 28, 2026)1234 |
| Additional source | OpenRouter: Popularity index (derived), CC BY 4.0 |
| Additional source | Vercel AI Gateway: Popularity index (derived), CC BY 4.0 |
| Additional source | Ollama: cumulative downloads for the popularity index |
| Additional source | Replicate: cumulative runs for the popularity index |
| Additional source | Google Trending Now: limited feed selection for Germany and the US |
| Additional source | Creative Commons Attribution 4.0 |