OLMo 2 13B
Unavailable in this round
Language model
OLMo 2 13B
Allen Institute
- Released
- November 2024
- Input
- Text
- Output
- Text
OLMo 2 13B is Allen AI’s fully open research checkpoint, not just a downloadable set of weights. Alongside the models, Allen AI releases training code, checkpoints, logs, and details about data preparation. The model card says the 13B checkpoint was trained on 5 trillion tokens and deliberately keeps a 4,096-token context.
The OLMo 2 release includes base, SFT, DPO, and RLVR variants. OLMo 2 13B belongs to the 1124 release, while OLMo 2 32B belongs to the later 0325 release. Both checkpoints remain limited to a 4,096-token context.
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
OLMES MMLU (Show measurement, test conditions, and source)
- Source value
- 67.5
- Score
- 67.5
- Metric
- accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Test; 14,042 items; 5-shot; char normalization; MCF only.
OLMES ARC-Challenge (Show measurement, test conditions, and source)
- Source value
- 83.5
- Score
- 83.5
- Metric
- accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Test; 1,172 items; 5-shot; PMI normalization.
OLMES HellaSwag (Show measurement, test conditions, and source)
- Source value
- 86.4
- Score
- 86.4
- Metric
- accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Validation; 1,000 sampled from 10,042; 5-shot; char normalization.
OLMES WinoGrande (Show measurement, test conditions, and source)
- Source value
- 81.5
- Score
- 81.5
- Metric
- accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Validation; 1,267 items; 5-shot; no normalization.
OLMES Natural Questions (Show measurement, test conditions, and source)
- Source value
- 46.7
- Score
- 46.7
- Metric
- F1
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Validation; 1,000 sampled from 3,610; 5-shot.
OLMES DROP (Show measurement, test conditions, and source)
- Source value
- 70.7
- Score
- 70.7
- Metric
- F1
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Validation; 1,000 sampled from 9,536; 5-shot.
OLMES AGIEval English (Show measurement, test conditions, and source)
- Source value
- 54.2
- Score
- 54.2
- Metric
- MCF accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Test; 2,646 items; 1-shot; held-out.
OLMES GSM8K (Show measurement, test conditions, and source)
- Source value
- 75.1
- Score
- 75.1
- Metric
- exact match
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Test; 1,319 items; 8-shot CoT; held-out.
OLMES MMLU-Pro (Show measurement, test conditions, and source)
- Source value
- 35.1
- Score
- 35.1
- Metric
- MCF accuracy
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Test; 12,032 items; 5-shot; held-out.
OLMES TriviaQA (Show measurement, test conditions, and source)
- Source value
- 81.9
- Score
- 81.9
- Metric
- F1
- Unit
- %
- Benchmark version
- OLMES Table 6 / Appendix A.1 (report v3 2025-10-08)
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Allen Institute for AI OLMo 2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Independent evaluation. OLMo 2 13B base-model row in Table 6 of report v3 (2025-10-08), evaluated with OLMES. Report defines multiple-choice scores as accuracy and generative scores as F1. Appendix A.1 supplies splits and shots. Exact base checkpoint only; no Instruct results transferred.; Validation; 7,993 items; 5-shot; held-out.
Our tests
SVG short test
One SVG prompt and fixed output settings. Captures are shown only with 0 reported reasoning tokens or documented lack of reasoning controls. Identical test conditions do not make the models' internal compute identical.
Popularity
27.6 / 100
Sources and data date
| Type | Evidence and data date |
|---|---|
| Input | Documented modalities. |
| Output | Documented modalities. |
| Research status | Research date October 15, 2026. 1 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication. |
| Additional source | Allen Institute for AI OLMo 2 technical report (retrieved October 15, 2026) |
| Additional source | OLMo 2 technical report · Editorial description reviewed October 2, 2026 |
| Additional source | OpenRouter: Popularity index (derived), CC BY 4.0 |
| Additional source | Vercel AI Gateway: Popularity index (derived), CC BY 4.0 |
| Additional source | Creative Commons Attribution 4.0 |