Skip to main content

Mistral AILanguage model

Mistral Small 3.1 24B

Mistral Small 3.1 24B has no overall index value yet because there are too few independent benchmark results. From the vendor-reported results we estimate about 35.4 points (95% range 15.8 to 55.0). This is not an independent measurement.

PopularityRank 613 of 875

16out of 100same as last month

Summary

Mistral Small 3.1 24B is the dated instruction-tuned checkpoint released in March 2025. The multimodal model accepts text and images with 128,000 tokens of context, supports function calls, and uses Apache 2.0.

The family entry Mistral Small 3.1 covers the same release.

See for yourself: one prompt, one attempt

We show the latest attempt without fixes; it does not count toward the overall index.

Output withheld for safety

Pelican on a bicycle

Prompt

Generate an SVG of a pelican riding a bicycle

Oct 2, 2026 · 191 s · $0.0046

Not yet visually reviewed. A successful recording does not confirm correct physics or full compliance with the prompt.

Test details
Run date
Oct 2, 2026
API provider
openrouter
Requested API model
mistralai/mistral-small-3.1-24b-instruct
Reasoning setting
not specified
Token limit including reasoning
8,192 tokens
Output tokens
8,192 tokens
Cost of the attempt
$0.0046
Response time
191.5 s
API model reported in response
mistralai/mistral-small-3.1-24b-instruct
Pinned endpoint
Cloudflare | mistralai/mistral-small-3.1-24b-instruct-2503
Temperature
Provider default
Test protocol
community-visual-tests-v1
Technical checks
Failed
Task origin: Simon Willison
SVG short test

SVG short test

One SVG task with the same fixed settings for every model, without hidden thinking.

Mistral Small 3.1 24B

No independent test. This API route aliases mistral-small-3.1.

All benchmark results

Every published result with its source, test setup and comparison models.

Show all 43 published benchmark results

Page 1 of 4

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

MMLU (Show measurement, test conditions, and source)
Source value
80.62
Score
80.62
Metric
score as published
Unit
%
Category
knowledge
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Instruction Evals; card does not state shot count for MMLU.
Source Mistral AI Mistral Small 3.1 model card
MMLU-Pro (Show measurement, test conditions, and source)
Source value
66.76
Score
66.76
Metric
5-shot CoT
Unit
%
Category
knowledge
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row; the column explicitly states 5-shot CoT.
Source Mistral AI Mistral Small 3.1 model card
MATH (Show measurement, test conditions, and source)
Source value
69.3
Score
69.3
Metric
score as published
Unit
%
Category
reasoning
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Instruction Evals; task setup not specified for this column.
Source Mistral AI Mistral Small 3.1 model card
GPQA Main (Show measurement, test conditions, and source)
Source value
44.42
Score
44.42
Metric
5-shot CoT
Unit
%
Category
reasoning
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row; the column explicitly states 5-shot CoT.
Source Mistral AI Mistral Small 3.1 model card
GPQA Diamond (Show measurement, test conditions, and source)
Source value
45.96
Score
45.96
Metric
5-shot CoT
Unit
%
Category
reasoning
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row; the column explicitly states 5-shot CoT.
Source Mistral AI Mistral Small 3.1 model card
MBPP (Show measurement, test conditions, and source)
Source value
74.71
Score
74.71
Metric
score as published
Unit
%
Category
coding
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Instruction Evals; task setup not specified for this column.
Source Mistral AI Mistral Small 3.1 model card
HumanEval (Show measurement, test conditions, and source)
Source value
88.41
Score
88.41
Metric
score as published
Unit
%
Category
coding
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Instruction Evals; task setup not specified for this column.
Source Mistral AI Mistral Small 3.1 model card
SimpleQA (Show measurement, test conditions, and source)
Source value
10.43
Score
10.43
Metric
TotalAcc
Unit
%
Category
knowledge
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row; table labels this metric TotalAcc.
Source Mistral AI Mistral Small 3.1 model card
MMMU (Show measurement, test conditions, and source)
Source value
64
Score
64
Metric
score as published
Unit
%
Category
vision
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Vision Evals; the card does not state additional prompt settings.
Source Mistral AI Mistral Small 3.1 model card
MMMU Pro (Show measurement, test conditions, and source)
Source value
49.25
Score
49.25
Metric
score as published
Unit
%
Category
vision
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Vision Evals; the card does not state additional prompt settings.
Source Mistral AI Mistral Small 3.1 model card
MathVista (Show measurement, test conditions, and source)
Source value
68.91
Score
68.91
Metric
score as published
Unit
%
Category
vision
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Vision Evals; the card does not state additional prompt settings.
Source Mistral AI Mistral Small 3.1 model card
LongBench v2 (Show measurement, test conditions, and source)
Source value
37.18
Score
37.18
Metric
score as published
Unit
%
Category
long-context
Direction
Higher is better
Source type
vendor-reported
Evaluator
Mistral AI Mistral Small 3.1 model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Exact Small 3.1 24B Instruct row in Long Context Evals; the card does not state context setup or prompt details.
Source Mistral AI Mistral Small 3.1 model card

Page 1 of 4

Popularity compared

How often the model is searched for, used and mentioned, compared with every model of its category.

Popularity index

The 100 most popular models of this category, 0 to 100, higher is more popular

20 of 101 models

Gemini 4 Argon92
GPT-6 Astra78
Claude Haiku 5.577
Qwen3.5 4B76
GPT-573
GPT-6 Sol72
Qwen3.5 0.8B72
Qwen3.5 2B71
Qwen 3.8 27B70
Claude Opus 5.570
DeepSeek V4.1 Flash 101768
Qwen 3.8 Max 090268
Kimi K368
DeepSeek-V4-Flash65
GPT-5.6 Terra64
Claude Sonnet 563
DiffusionGemma 26B A4B IT63
Claude Opus 562
Qwen 3.8 Flash62
Mistral Small 3.1 24B16

As of November 2026 · Source: gradually.ai popularity index

Over time

Weekly values from October 18, 2026 to November 5, 2026

Mistral Small 3.1 24BJamba LargeJamba Large 1.7Sonar ProMiMo V2.6 Pro

All specifications

Architecture

Parameters
24 billion
Source: Parameters
Knowledge Cutoff
June 2024
Source: Knowledge Cutoff

Input and output

Context window
128,000 tokens
Source: Context window
Maximum output
16,384 tokens
Source: Maximum output
Inputs
Text, Image
Source: Inputs
Outputs
Text
Source: Outputs
Tool calling
Yes
Source: Tool calling

Watermarking and provenance

AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.

AI watermarking
Unknown
no source
Watermarking technique
Not known
no source

Licence and access

Weights
Open weights
Source: Weights
Access
API
Source: Access
License
Apache 2.0
Source: License

Prices

Not included in any subscription in Germany.

All plans in the AI subscription comparison

Frequently asked questions about Mistral Small 3.1 24B

How good is Mistral Small 3.1 24B in benchmarks?

Independent results are not yet enough for the overall index. An estimate from 4 vendor-reported results (GPQA Diamond 46%, MMLU-Pro 66.8%, MMMU-Pro 49.3%, MMMU (val) 64%) gives about 35.4 points, with a 95% range of 15.8 to 55.0. The estimate is calibrated on models with both vendor-reported and independent results. Vendor numbers run clearly above independent measurements on some tests; the calibration offsets that on average.

Vendor estimate, not an overall index value

What is the knowledge cutoff of Mistral Small 3.1 24B?

June 2024, according to Mistral AI. That is 9 months before release.

Stated by Mistral AI

How large is the context window of Mistral Small 3.1 24B?

128,000 tokens. That is how much text Mistral Small 3.1 24B reads at once, about 190 A4 pages.

Page count as a rule of thumb from Artificial Analysis

Context windowTokens the model reads at once · higher is better
GLM-5.1200K
DeepSeek-V3 0324164K
GLM-4.5131K
GPT OSS 120B131K
Mistral Small 3.1 24BRank 97 of 102128K
DeepSeek-V3.2128K
DeepSeek-R1128K

Can I run Mistral Small 3.1 24B locally?

Yes, the weights are open.

Stated by Mistral AI

Open weights
yes
License
Apache 2.0
Parameters
24B
active per token
n/a

Sources and data date

Show all 15 sources

Benchmark sources at a glance

Which independent evaluators tested this model. This is coverage, not a quality ranking.

No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.

Without vendor claims and historical or disputed results; those stay labelled in the single values.

TypeEvidence and data date
Input

Documented modalities. Retrieved October 27, 2026.

Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026)
Output

Documented modalities. Retrieved October 27, 2026.

Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026)
Research statusResearch date October 15, 2026. 6 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not published

benchmark-run-details

Note

The card gives exact task rows and selected few-shot settings but omits task dates, most prompting details, and model inference budgets.

Additional sourceHugging Face (retrieved October 15, 2026; October 19, 2026) · Mistral Small 3.1 model card · Editorial description reviewed October 2, 2026
Additional sourceTALES, UC San Diego, Microsoft Research Montréal and Johns Hopkins University (retrieved October 17, 2026)
Additional sourceEvaLooop paper, arXiv:2505.12185v5 (retrieved October 17, 2026; October 18, 2026)
Additional sourceLongTableBench, Findings of the Association for Computational Linguistics: EMNLP 2025 (retrieved October 18, 2026)
Additional sourceWu et al., Why Chain of Thought Fails in Clinical Text Understanding (retrieved October 22, 2026)
Additional sourceModel metadata (retrieved October 17, 2026)
Additional sourceMistral Small 3.1 announcement · Editorial description reviewed October 2, 2026
Additional sourceMistral model overview · Editorial description reviewed October 2, 2026
Additional sourceOpenRouter: Popularity index (derived), CC BY 4.0
Additional sourceVercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional sourceOllama: cumulative downloads for the popularity index
Additional sourceReplicate: cumulative runs for the popularity index
Additional sourceGoogle Trending Now: limited feed selection for Germany and the US
Additional sourceCreative Commons Attribution 4.0