Skip to main content

Mistral AILanguage model

Devstral 2

Devstral 2 has no overall index value yet because there are too few independent benchmark results.

PopularityRank 472 of 785

33out of 100same as last month

based on search, usage and attention

Summary

New in Devstral 2 is its focus on agents that operate development environments. The dense checkpoint has 123 billion parameters, 256,000 tokens of context, tool calls, repository navigation, and long code changes.

Devstral Small 2 is the 24B variant in the same product line. Devstral 2 uses a Modified MIT license.

See for yourself: one prompt, one attempt

SVG short test

One SVG task with the same fixed settings for every model, without hidden thinking.

Devstral 2

Devstral 2: SVG short test

Prompt

Return only one standalone SVG of a pelican riding a bicycle. Use viewBox="0 0 800 600", simple shapes, two clearly visible wheels and a recognizable pelican with a long beak. At most 30 SVG elements and 1800 characters. No Markdown, explanations, comments, scripts, animation, CSS or external resources.

Oct 14, 2026 · $0.0010

Test details
Test run
Oct 14, 2026
Provider
mistral/eu
API model
mistralai/devstral-2512
Output
461 tokens
Reasoning
API reports 0 reasoning tokens
Cost of the attempt
$0.0010
Test round
43b8c5c46830
Earlier tests
Devstral 2: Pelican on a bicycle
A community classic for free-form SVG drawing.

Pelican on a bicycle

Prompt

Generate an SVG of a pelican riding a bicycle

Oct 2, 2026 · 7.1 s · $0.0020

Test details
Run date
Oct 2, 2026
API provider
openrouter
Requested API model
mistralai/devstral-2512
Reasoning setting
not specified
Token limit including reasoning
8,192 tokens
Output tokens
1,002 tokens
Cost of the attempt
$0.0020
Response time
7.1 s
API model reported in response
mistralai/devstral-2512
Pinned endpoint
Mistral | mistralai/devstral-2512
Temperature
Provider default
Test protocol
community-visual-tests-v1
Technical checks
Passed, not a quality rating

Visually reviewed. This individual test does not establish overall model quality.

Earlier attempts (1)

  • Oct 1, 2026API request failedReasoning setting: not specified
Task origin: Simon Willison

All benchmark results

Every published result with its source, test setup and comparison models.

Show all 44 published benchmark results

Page 1 of 4

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

CritPt v2: Overall (Show measurement, test conditions, and source)
Source value
0
Score
0
Metric
pass@1 accuracy
Unit
%
Benchmark version
v2
Category
reasoning
Task
overall
Task label
Overall
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis
Status
disputed
Retrieved at
2026-11-02
Methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/09f43999-b67b-4c1b-b050-44df41ed7e62/devstral-2/devstral-2. Source model name: Devstral 2. Provider reported by source: Mistral. Model snapshot: Devstral 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none. Score display: 0.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:non-reasoning:effort=unspecified:suffix=unspecified
Comparison methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: non-reasoning. Reasoning effort: none
Context
This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.
Source Artificial Analysis
IFBench: Overall (Show measurement, test conditions, and source)
Source value
38.0952380952381
Score
38.0952380952381
Metric
IFBench prompt-level loose accuracy (pass@1)
Unit
%
Benchmark version
AA test set: 294 questions
Category
general
Task
overall
Task label
Overall
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis
Status
superseded
Retrieved at
2026-11-02
Methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/09f43999-b67b-4c1b-b050-44df41ed7e62/devstral-2/devstral-2. Source model name: Devstral 2. Provider reported by source: Mistral. Model snapshot: Devstral 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 38.1%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
Comparison methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). AA marks this model variant as non-reasoning and reports 0 reasoning tokens. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
Context
The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.. This result is retained as historical evidence and is not interchangeable with current results.
Source Artificial Analysis
IFBench: Overall (Show measurement, test conditions, and source)
Source value
38.0952380952381
Score
38.0952380952381
Metric
IFBench prompt-level loose accuracy (pass@1)
Unit
%
Benchmark version
AA test set: 294 questions
Category
general
Task
overall
Task label
Overall
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis
Status
active
Retrieved at
2026-11-07
Methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Source model ID: artificial-analysis/09f43999-b67b-4c1b-b050-44df41ed7e62/devstral-2/devstral-2. Source model name: Devstral 2. Provider reported by source: Mistral. Model snapshot: Devstral 2. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none. Score display: 38.1%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-07-b96e3ffc8c5b. Source payload hash: b96e3ffc8c5b052aa56f96d05e89d02b8cc234a75085820fca0ca2a39e498274. Parser version: artificial-analysis-rsc-models-v4. Imported at: 2026-11-07. Source cohort: artificial-analysis-ifbench:AA test set: 294 questions:standalone:non-reasoning:effort=unspecified:suffix=unspecified
Comparison methodology
Exact AA configuration: Devstral 2 (release devstral-2, row devstral-2, UUID 09f43999-b67b-4c1b-b050-44df41ed7e62, reasoning=false, effort=not specified, suffix=not specified). This source capture does not provide an average reasoning-token count. Artificial Analysis runs 294 questions from the IFBench test set with five repeats per question. The official evaluator scores prompt-level accuracy in loose mode as pass@1.. Tool policy: Single-turn answer evaluated with the official IFBench source code. Attempts: pass@1, 5 repeats per question. Reasoning: non-reasoning. Reasoning effort: none
Context
The often cited 58 refers to verifiable out-of-domain constraint types, not the number of questions in AA's test run.
Source Artificial Analysis
SWE-Bench Verified (Show measurement, test conditions, and source)
Source value
72.2
Score
72.2
Metric
score as published
Unit
%
Category
coding
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Devstral 2 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Devstral 2 (123B) row; the card does not publish the benchmark harness, decoding settings or run date.
Source Mistral AI Devstral 2 model card
SWE-Bench Multilingual (Show measurement, test conditions, and source)
Source value
61.3
Score
61.3
Metric
score as published
Unit
%
Category
coding
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Devstral 2 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Devstral 2 (123B) row; the card does not publish the benchmark harness, decoding settings or run date.
Source Mistral AI Devstral 2 model card
Terminal-Bench 2 (Show measurement, test conditions, and source)
Source value
32.6
Score
32.6
Metric
score as published
Unit
%
Benchmark version
2
Category
coding
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Devstral 2 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Devstral 2 (123B) row; the card does not publish the benchmark harness, decoding settings or run date.
Source Mistral AI Devstral 2 model card
SWE-bench Verified (official mini-SWE-agent) (Show measurement, test conditions, and source)
Source value
53.8
Score
53.8
Metric
resolved issue rate
Unit
%
Benchmark version
SWE-bench Verified; mini-SWE-agent 1.17.2; submission 20251209_mini-v1.17.2_devstral-2512
Category
coding
Direction
Higher is better
Source type
official-leaderboard
Evaluator
SWE-bench
Status
active
Retrieved at
2026-10-16
Methodology
Official benchmark leaderboard. Official SWE-bench Verified (500 benchmark tasks); mini-SWE-agent 1.17.2; source submission 20251209_mini-v1.17.2_devstral-2512; source date 2025-12-09; model label Devstral (2512); reasoning_effort=not specified. Source logs: s3://swe-bench-submissions/bash-only/20251209_mini-v1.17.2_devstral-2512/logs. The leaderboard marks this submission as independently checked: yes.
Context
Historical official leaderboard submission, not a fresh model run or a Vals evaluation. mini-SWE-agent 1.x and 2.x use different action interfaces and are not necessarily comparable. Undisclosed sampling, repetition counts and backend revisions remain unknown.
Source SWE-bench
Cline human evaluation versus DeepSeek V3.2, win rate (Show measurement, test conditions, and source)
Source value
42.8
Score
42.8
Metric
Win rate
Unit
%
Benchmark version
Mistral Devstral 2 release evaluation; version not stated
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI, Devstral 2 and Mistral Vibe release
Status
active
Retrieved at
2026-10-17
Methodology
Vendor-reported result. Devstral 2 compared with DeepSeek V3.2 in human evaluations conducted by an independent annotation provider, with tasks scaffolded through Cline. The source does not publish the task count, annotation protocol, or confidence interval.
Context
Vendor-reported comparison; the human evaluations were conducted by an independent annotation provider, but the task count, annotation protocol, and confidence interval are not published. No tie rate is inferred.
Source Mistral AI, Devstral 2 and Mistral Vibe release
Cline human evaluation versus DeepSeek V3.2, loss rate (Show measurement, test conditions, and source)
Source value
28.6
Score
28.6
Metric
Loss rate
Unit
%
Benchmark version
Mistral Devstral 2 release evaluation; version not stated
Direction
Lower is better
Source type
vendor-reported
Evaluator
Mistral AI, Devstral 2 and Mistral Vibe release
Status
active
Retrieved at
2026-10-17
Methodology
Vendor-reported result. Devstral 2 compared with DeepSeek V3.2 in human evaluations conducted by an independent annotation provider, with tasks scaffolded through Cline. The source does not publish the task count, annotation protocol, or confidence interval. No tie rate is inferred from the two reported rates.
Context
Vendor-reported comparison; the human evaluations were conducted by an independent annotation provider, but the task count, annotation protocol, and confidence interval are not published. No tie rate is inferred.
Source Mistral AI, Devstral 2 and Mistral Vibe release
AutomationBench-AA (Show measurement, test conditions, and source)
Source value
3
Score
3
Metric
Objectives completed without guardrail violations
Unit
%
Category
agentic SaaS workflows
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis model comparison
Status
active
Retrieved at
2026-10-18
Methodology
Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral 2 (125B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
Context
Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
Source Artificial Analysis model comparison
Terminal-Bench (Show measurement, test conditions, and source)
Source value
0
Score
0
Metric
Score
Unit
%
Benchmark version
4.0
Category
agentic coding and terminal use
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis model comparison
Status
active
Retrieved at
2026-10-18
Methodology
Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral 2 (125B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
Context
Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
Source Artificial Analysis model comparison
SciCode (Show measurement, test conditions, and source)
Source value
33
Score
33
Metric
SciCode score
Unit
%
Category
coding
Direction
Higher is better
Source type
independent-evaluator
Evaluator
Artificial Analysis model comparison
Status
disputed
Retrieved at
2026-10-18
Methodology
Independent evaluation. Artificial Analysis comparison entry for Mistral Devstral 2 (125B as listed; reasoning: No). The comparison reports 13k output tokens per task, 0 reasoning tokens per task, and 18M output tokens for the Intelligence Index. The exact checkpoint revision, serving or agent route, prompt, decoding settings, and per-evaluation inference configuration are not disclosed.
Context
Artificial Analysis labels the SciCode evaluation “Under review” in its public evaluation list (retained source-01, L192). The status field is set to “disputed” as the schema’s available non-active flag for this evaluator-marked provisional result; this does not assert that sources report conflicting scores. The displayed value remains provisional, not a finalized active result. Artificial Analysis marks the model as deprecated and states that only performance benchmarking for the default 10k-input-token workload continues; results for other workloads are historical and no longer updated. The comparison does not report an evaluation date, per-cell sample count, or confidence interval. Retrieval date is 2026-10-18, not a claim of a fresh run.
Source Artificial Analysis model comparison

Page 1 of 4

Popularity compared

How often the model is searched for, used and mentioned, compared with every model of its category.

Popularity index

The 100 most popular models of this category, 0 to 100, higher is more popular

20 of 101 models

GPT-6 Astra83
Claude Haiku 5.577
GPT-6 Sol73
GPT-573
DeepSeek V4.1 Flash 101769
Claude Opus 5.568
Qwen 3.8 27B68
Grok 4.767
Kimi K367
Qwen 3.8 Max 090266
Claude Sonnet 564
Qwen3.5 4B64
DeepSeek-V4-Flash64
Claude Fable 5.164
GPT-5.6 Terra64
MiniCPM-V 4.7 35B-A3B63
GPT-6.1 Sol63
Claude Opus 563
Qwen 3.8 Flash62
Devstral 233

As of November 2026 · Source: gradually.ai popularity index

Over time

Weekly values from October 18, 2026 to November 7, 2026

Devstral 2Llama 3.3 Nemotron Super 49B v1 33Holo4-27B 33Index-Translate-35B-A3B Preview 32Mistral Large 3 32

Method changes in the timeline, a jump at these points is no market event:

  1. 1November 1, 2026: Value only with search or usage data, 30-day attention

What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.

Similar models

More models of the same kind as Devstral 2, chosen by model line, developer, size, price and release date.

All specifications

Architecture

Parameters
123 billion
Source: Parameters
Knowledge cutoff
December 2025
Source: Knowledge cutoff

Input and output

Context window
262,144 tokens
Source: Context window
Maximum output
262,144 tokens
Source: Maximum output
Inputs
Text
Source: Inputs
Outputs
Text
Source: Outputs
Tool calling
Yes
Source: Tool calling

Watermarking and provenance

AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.

AI watermarking
Unknown
no source
Watermarking technique
Not known
no source

Licence and access

Weights
Open weights
Source: Weights
Access
API
Source: Access
License
Modified MIT (Mistral)
Source: License

Prices

Not included in any subscription in Germany.

All plans in the AI subscription comparison

Frequently asked questions about Devstral 2

What is the knowledge cutoff of Devstral 2?

December 2025, according to Mistral AI. That is the month of release.

Stated by Mistral AI

How large is the context window of Devstral 2?

262,144 tokens. That is how much text Devstral 2 reads at once, about 390 A4 pages.

Page count as a rule of thumb from Artificial Analysis

Context windowTokens the model reads at once · higher is better
Devstral 2Rank 67 of 103262K
Kimi K2.6262K
Qwen 3.8 27B262K
Kimi K2.7 Code262K
Kimi K2.5262K
Nemotron 3 Ultra262K
Kimi K2 Thinking262K

Can I run Devstral 2 locally?

Yes, the weights are open. With 123B parameters it needs a lot of memory, more a server than a laptop.

Stated by Mistral AI

Open weights
yes
License
Modified MIT (Mistral)
Parameters
123B
active per token
n/a

Sources and data date

Show all 23 sources

Benchmark sources at a glance

Which independent evaluators tested this model. This is coverage, not a quality ranking.

EvaluatorTest seriesComposite indicesRecorded active measurements
Artificial Analysis8111
SWE-bench001

Without vendor claims and historical or disputed results; those stay labelled in the single values. 2 measurements are not yet assigned to a test series.

TypeEvidence and data date
Input

Documented modalities. Retrieved October 27, 2026.

Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026)
Output

Documented modalities. Retrieved October 27, 2026.

Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026)
Research statusResearch date October 15, 2026. 12 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not published

benchmark-configuration

Note

The card gives three scores but not harness, agent scaffold, task dates or inference settings.

Not published

human-evaluation-sample-count-and-confidence-interval

Original wording

The article does not publish the number of human-evaluated tasks, annotation details, or confidence intervals. The unreported remainder is not interpreted as ties.

Not published

95% confidence interval

Original wording

The paper reports SD, CV, and an observed range for Total but no confidence interval; confidence-interval fields are omitted.

Not published

sampleCount

Original wording

The paper describes five evaluation rounds and 107 tasks per round but does not state sampleCount for the aggregate result; sampleCount is omitted.

Not published

Provider/API route and immutable serving revision

Original wording

The paper does not identify the provider/API route or an immutable serving revision.

Additional sourceHugging Face (retrieved October 15, 2026; October 18, 2026) · Devstral 2 model card · Editorial description reviewed October 2, 2026
Additional sourceArtificial Analysis (retrieved November 2, 2026)
Additional sourceArtificial Analysis (retrieved October 29, 2026; November 2, 2026; November 7, 2026)
Additional sourceArtificial Analysis (retrieved November 2, 2026; November 7, 2026)
Additional sourceSWE-bench (retrieved October 16, 2026)
Additional sourceMistral AI, Devstral 2 and Mistral Vibe release (retrieved October 17, 2026) · Devstral 2 and Vibe CLI announcement · Editorial description reviewed October 2, 2026
Additional sourceDataLLM Lab (retrieved October 18, 2026)
Additional sourceArtificial Analysis model comparison (retrieved October 18, 2026)
Additional sourceEVM-QuestBench, ACL 2026 (retrieved October 18, 2026)
Additional sourceContextBench authors (arXiv paper) (retrieved October 19, 2026)
Additional sourceOfficial n8n AI Benchmark (retrieved October 23, 2026) (unavailable on November 2, 2026) https://n8n.io/ai-benchmark/devstral-2512/
Additional sourceObviousBench Mistral model reliability (retrieved October 23, 2026)
Additional sourceModel metadata (retrieved October 16, 2026)12
Additional sourceModel metadata (retrieved October 18, 2026)
Additional sourceMistral model overview · Editorial description reviewed October 2, 2026
Additional sourceOpenRouter: Popularity index (derived), CC BY 4.0
Additional sourceVercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional sourceOllama: cumulative downloads for the popularity index
Additional sourceReplicate: cumulative runs for the popularity index
Additional sourceGoogle Trending Now: limited feed selection for Germany and the US
Additional sourceCreative Commons Attribution 4.0