Mistral AILanguage model
Devstral Small 2
Devstral Small 2 has no overall index value yet because there are too few independent benchmark results.
PopularityRank 319 of 785
39out of 100-1 vs. last month
based on search, usage and attention
Summary
What is distinctive about Devstral Small 2 is a 24B coding agent with text and image input. It handles 256,000 tokens of context, reads codebases, calls tools, and plans multi-step changes. The release uses Apache 2.0.
Compared with Devstral 2, Small 2 adds screenshots or documentation as image input.
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 27 published benchmark results
Page 1 of 3
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
CritPt v2: Overall (Show measurement, test conditions, and source)
- Source value
- 0
- Score
- 0
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- v2
- Category
- reasoning
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- disputed
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/ce819310-af7c-49d3-9a02-6845111e1788/devstral-small-2/devstral-small-2. Source model name: Devstral Small 2. Provider reported by source: Mistral. Model snapshot: Devstral Small 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none. Score display: 0.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:non-reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none
- Context
- This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 31.156462585034
- Score
- 31.156462585034
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/ce819310-af7c-49d3-9a02-6845111e1788/devstral-small-2/devstral-small-2. Source model name: Devstral Small 2. Provider reported by source: Mistral. Model snapshot: Devstral Small 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 31.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.. This result is retained as historical evidence and is not interchangeable with current results.
IFBench: Overall (Show measurement, test conditions, and source)
- Source value
- 31.156462585034
- Score
- 31.156462585034
- Metric
- IFBench prompt-level loose accuracy (pass@1)
- Unit
- %
- Benchmark version
- AA test set: 294 questions
- Category
- general
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/ce819310-af7c-49d3-9a02-6845111e1788/devstral-small-2/devstral-small-2. Source model name: Devstral Small 2. Provider reported by source: Mistral. Model snapshot: Devstral Small 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 31.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 44.6242774566474
- Score
- 44.6242774566474
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- superseded
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/ce819310-af7c-49d3-9a02-6845111e1788/devstral-small-2/devstral-small-2. Source model name: Devstral Small 2. Provider reported by source: Mistral. Model snapshot: Devstral Small 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 44.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.. This result is retained as historical evidence and is not interchangeable with current results.
MMMU-Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 44.6242774566474
- Score
- 44.6242774566474
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- AA cohort: 1,730 questions
- Category
- multimodal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- active
- Retrieved at
- 2026-11-07
- Methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Source model ID: artificial-analysis/ce819310-af7c-49d3-9a02-6845111e1788/devstral-small-2/devstral-small-2. Source model name: Devstral Small 2. Provider reported by source: Mistral. Model snapshot: Devstral Small 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 44.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-mmmu-pro:AA cohort: 1,730 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: Devstral Small 2 (release devstral-small-2, row devstral-small-2, UUID ce819310-af7c-49d3-9a02-6845111e1788, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. The AA run covers 1,730 questions with ten answer options. Answers are extracted by regex and scored pass@1.. Tool policy: Ten-option multiple choice; regex answer extraction. Attempts: pass@1, one run per question. Reasoning: non-reasoning. Reasoning effort: none
- Context
- The benchmark page describes 3,460 questions in the full MMMU-Pro dataset; AA's methodology specifies a 1,730-question run.
SWE-Bench Verified (Show measurement, test conditions, and source)
- Source value
- 68
- Score
- 68
- Metric
- score as published
- Unit
- %
- Category
- coding
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Mistral AI Devstral 2 model card
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact Devstral Small 2 (24B) row; the card does not publish the benchmark harness, decoding settings or run date.
SWE-Bench Multilingual (Show measurement, test conditions, and source)
- Source value
- 55.7
- Score
- 55.7
- Metric
- score as published
- Unit
- %
- Category
- coding
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Mistral AI Devstral 2 model card
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact Devstral Small 2 (24B) row; the card does not publish the benchmark harness, decoding settings or run date.
Terminal-Bench 2 (Show measurement, test conditions, and source)
- Source value
- 22.5
- Score
- 22.5
- Metric
- score as published
- Unit
- %
- Benchmark version
- 2
- Category
- coding
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Mistral AI Devstral 2 model card
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact Devstral Small 2 (24B) row; the card does not publish the benchmark harness, decoding settings or run date.
SWE-bench Verified (official mini-SWE-agent) (Show measurement, test conditions, and source)
- Source value
- 56.4
- Score
- 56.4
- Metric
- resolved issue rate
- Unit
- %
- Benchmark version
- SWE-bench Verified; mini-SWE-agent 1.17.2; submission 20251209_mini-v1.17.2_devstral-small-2512
- Category
- coding
- Direction
- Higher is better
- Source type
- official-leaderboard
- Evaluator
- SWE-bench
- Status
- active
- Retrieved at
- 2026-10-16
- Methodology
- Official benchmark leaderboard. Official SWE-bench Verified (500 benchmark tasks); mini-SWE-agent 1.17.2; source submission 20251209_mini-v1.17.2_devstral-small-2512; source date 2025-12-09; model label Devstral Small (2512); reasoning_effort=not specified. Source logs: s3://swe-bench-submissions/bash-only/20251209_mini-v1.17.2_devstral-small-2512/logs. The leaderboard marks this submission as independently checked: yes.
- Context
- Historical official leaderboard submission, not a fresh model run or a Vals evaluation. mini-SWE-agent 1.x and 2.x use different action interfaces and are not necessarily comparable. Undisclosed sampling, repetition counts and backend revisions remain unknown.
AutomationBench-AA (Show measurement, test conditions, and source)
- Source value
- 3
- Score
- 3
- Metric
- Objectives completed without guardrail violations
- Unit
- %
- Category
- agentic SaaS workflows
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis model comparison
- Status
- active
- Retrieved at
- 2026-10-18
- Methodology
- Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral Small 2 (24B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
- Context
- Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
Terminal-Bench (Show measurement, test conditions, and source)
- Source value
- 0
- Score
- 0
- Metric
- Score
- Unit
- %
- Benchmark version
- 4.0
- Category
- agentic coding and terminal use
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis model comparison
- Status
- active
- Retrieved at
- 2026-10-18
- Methodology
- Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral Small 2 (24B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
- Context
- Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
SciCode (Show measurement, test conditions, and source)
- Source value
- 32
- Score
- 32
- Metric
- SciCode score
- Unit
- %
- Category
- coding
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis model comparison
- Status
- disputed
- Retrieved at
- 2026-10-18
- Methodology
- Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral Small 2 (24B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
- Context
- Artificial Analysis labels the SciCode evaluation “Under review” in its public evaluation list (retained source-01, L192). The status field is set to “disputed” as the schema’s available non-active flag for this evaluator-marked provisional result; this does not assert that sources report conflicting scores. The displayed value remains provisional, not a finalized active result. Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
Page 1 of 3
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 101 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 8, 2026
Devstral Small 2AliceAI Foundation 80B A3B Base 39Command R7B 39Ling 3.0 Flash 39Ornith 1.0 397B 39
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
- 2November 1, 2026: Method version 4 (common scale, shrinkage)
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as Devstral Small 2, chosen by model line, developer, size, price and release date.
- Magistral Small 1.2Mistral AI · same developer
- Magistral SmallMistral AI · same developer
- Devstral SmallMistral AI · same developer
- Voxtral Small 24BMistral AI · same developer
- Voxtral Small (latest)Mistral AI · same developer
- Ministral 3 14BMistral AI · same developer
- Ministral 3 8BMistral AI · same developer
- Codestral-22B-v0.1Mistral AI · same developer
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
Frequently asked questions about Devstral Small 2
What is the knowledge cutoff of Devstral Small 2?
December 2025, according to Mistral AI. That is the month of release.
Stated by Mistral AI
DecKnowledge cutoffDecember 2025ReleasedDec 9, 2025
How large is the context window of Devstral Small 2?
262,144 tokens. That is how much text Devstral Small 2 reads at once, about 390 A4 pages.
Page count as a rule of thumb from Artificial Analysis
Devstral Small 2Rank 67 of 103262K
Kimi K2.6262K
Qwen 3.8 27B262K
Kimi K2.7 Code262K
Kimi K2.5262K
Nemotron 3 Ultra262K
Kimi K2 Thinking262K
Can I run Devstral Small 2 locally?
Yes, the weights are open.
Stated by Mistral AI
- Open weights
- yes
- License
- Apache 2.0
- Parameters
- 24B
- active per token
- n/a
Sources and data date
Show all 21 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
| Evaluator | Test series | Composite indices | Recorded active measurements |
|---|---|---|---|
| Artificial Analysis | 9 | 1 | 12 |
| SWE-bench | 0 | 0 | 1 |
Without vendor claims and historical or disputed results; those stay labelled in the single values. 2 measurements are not yet assigned to a test series.