MetaLanguage model

Llama 4 Scout 17B Instruct

Llama 4 Scout 17B Instruct has no overall index value yet because there are too few independent benchmark results. From the vendor-reported results we estimate about 43.0 points (95% range 22.9 to 63.1). This is not an independent measurement.

PopularityRank 333 of 785

38out of 100+2 vs. last month

based on search, usage and attention

Summary

Llama 4 Scout 17B Instruct is the cataloged Instruct identity for Meta’s open Scout weights. As with Maverick, 17B describes active rather than all MoE parameters. Its defining property is a context window of about ten million tokens for text and images.

The weights use the Llama 4 Community License. Llama 4 Scout leads to the family checkpoint, while Llama 4 Maverick 17B Instruct is the Maverick variant with a smaller context window. For Scout, the model card lists vLLM and SGLang, along with quantizations for llama.cpp, Ollama, and LM Studio.

See for yourself: one prompt, one attempt

We show the latest attempt without fixes; it does not count toward the overall index.

Pelican on a bicycle

Prompt

Generate an SVG of a pelican riding a bicycle

Oct 2, 2026 · 15 s · $0.0002

Test details

Run date: Oct 2, 2026
API provider: openrouter
Requested API model: meta-llama/llama-4-scout
Reasoning setting: not specified
Token limit including reasoning: 8,192 tokens
Output tokens: 598 tokens
Cost of the attempt: $0.0002
Response time: 14.6 s
API model reported in response: meta-llama/llama-4-scout
Pinned endpoint: DeepInfra | meta-llama/llama-4-scout-17b-16e-instruct
Temperature: Provider default
Test protocol: community-visual-tests-v1
Technical checks: Passed, not a quality rating

Visually reviewed. This individual test does not establish overall model quality.

Task origin: Simon Willison

SVG short test

One SVG task with the same fixed settings for every model, without hidden thinking.

Llama 4 Scout 17B Instruct

No independent test. This API route aliases llama-4-scout.

All benchmark results

Every published result with its source, test setup and comparison models.

Show all 59 published benchmark results

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

SWE-bench Verified (official mini-SWE-agent) (Show measurement, test conditions, and source)

Source value: 9.06
Score: 9.06
Metric: resolved issue rate
Unit: %
Benchmark version: SWE-bench Verified; mini-SWE-agent 0.0.0; submission 20250720_mini-v0.0.0-Llama-4-Scout-17B-Instruct
Category: coding
Direction: Higher is better
Source type: official-leaderboard
Evaluator: SWE-bench
Status: active
Retrieved at: 2026-10-16
Methodology: Official benchmark leaderboard. Official SWE-bench Verified (500 benchmark tasks); mini-SWE-agent 0.0.0; source submission 20250720_mini-v0.0.0-Llama-4-Scout-17B-Instruct; source date 2025-07-20; model label Llama 4 Scout Instruct; reasoning_effort=not specified. Source logs: s3://swe-bench-submissions/bash-only/20250720_mini-v0.0.0-Llama-4-Scout-17B-Instruct/logs. The leaderboard marks this submission as independently checked: yes.
Context: Historical official leaderboard submission, not a fresh model run or a Vals evaluation. mini-SWE-agent 1.x and 2.x use different action interfaces and are not necessarily comparable. Undisclosed sampling, repetition counts and backend revisions remain unknown.

Source SWE-bench

MMMU (Show measurement, test conditions, and source)

Source value: 69.4
Score: 69.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

MMMU Pro (Show measurement, test conditions, and source)

Source value: 52.2
Score: 52.2
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

MathVista (Show measurement, test conditions, and source)

Source value: 70.7
Score: 70.7
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

ChartQA (Show measurement, test conditions, and source)

Source value: 88.8
Score: 88.8
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

DocVQA (Show measurement, test conditions, and source)

Source value: 94.4
Score: 94.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

LiveCodeBench (Show measurement, test conditions, and source)

Source value: 32.8
Score: 32.8
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

MMLU Pro instruct (Show measurement, test conditions, and source)

Source value: 74.3
Score: 74.3
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

GPQA (Show measurement, test conditions, and source)

Source value: 57.2
Score: 57.2
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

MGSM (Show measurement, test conditions, and source)

Source value: 90.6
Score: 90.6
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned table; BF16

Source Meta Llama 4 model card

MTOB half (Show measurement, test conditions, and source)

Source value: 54
Score: 54
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: disputed
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned tool-use table; BF16
Context: The official Llama 4 model card assigns this value to Maverick in the English-to-Kalamang translation direction. This former Scout attribution is not supported. Separate direction-specific Scout results are retained.

Source Meta Llama 4 model card

MTOB full (Show measurement, test conditions, and source)

Source value: 50.8
Score: 50.8
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 4 model card
Status: disputed
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruction-tuned tool-use table; BF16
Context: The official Llama 4 model card assigns this value to Maverick in the English-to-Kalamang translation direction. This former Scout attribution is not supported. Separate direction-specific Scout results are retained.

Source Meta Llama 4 model card

Page 1 of 5

Popularity compared

How often the model is searched for, used and mentioned, compared with every model of its category.

Popularity index

The 100 most popular models of this category, 0 to 100, higher is more popular

20 of 101 models

GPT-6 Astra83

Claude Haiku 5.577

GPT-6 Sol73

GPT-573

DeepSeek V4.1 Flash 101769

Kimi K367

MiniCPM-V 4.7 35B-A3B63

GPT-6.1 Sol63

Claude Opus 563

Qwen 3.8 Flash62

Llama 4 Scout 17B Instruct38

DeepSeek V4.1 Flash 1017

MiniCPM-V 4.7 35B-A3B

GPT-6.1 Sol

Claude Opus 5

Qwen 3.8 Flash

Llama 4 Scout 17B Instruct

As of November 2026 · Source: gradually.ai popularity index

Over time

Weekly values from October 18, 2026 to November 7, 2026

Llama 4 Scout 17B InstructFugu Ultra v2 38Kimi K2.8 Preview 38DeepSeek-V3.1 Terminus 38MiniMax M2 38

Method changes in the timeline, a jump at these points is no market event:

1November 1, 2026: Value only with search or usage data, 30-day attention

What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.

Similar models

More models of the same kind as Llama 4 Scout 17B Instruct, chosen by model line, developer, size, price and release date.

All specifications

Architecture

Parameters: 109 billion; Source: Parameters
Active parameters: 17 billion; Source: Active parameters
Knowledge cutoff: August 2024; Source: Knowledge cutoff

Input and output

Context window: 10,000,000 tokens; Source: Context window
Maximum output: 16,384 tokens; Source: Maximum output
Inputs: Text, Image; Source: Inputs
Outputs: Text; Source: Outputs
Tool calling: Yes; Source: Tool calling

Watermarking and provenance

AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.

AI watermarking: Unknown; no source
Watermarking technique: Not known; no source

Licence and access

Weights: Open weights; Source: Weights
Access: API; Source: Access
License: Llama 4 Community; Source: License

Prices

Not included in any subscription in Germany.

All plans in the AI subscription comparison

Frequently asked questions about Llama 4 Scout 17B Instruct

How good is Llama 4 Scout 17B Instruct in benchmarks?

Independent results are not yet enough for the overall index. An estimate from 3 vendor-reported results (MMLU-Pro 74.3%, MMMU-Pro 52.2%, MMMU (val) 69.4%) gives about 43.0 points, with a 95% range of 22.9 to 63.1. The estimate is calibrated on models with both vendor-reported and independent results. Vendor numbers run clearly above independent measurements on some tests; the calibration offsets that on average.

Vendor estimate, not an overall index value

What is the knowledge cutoff of Llama 4 Scout 17B Instruct?

August 2024, according to Meta. That is 8 months before release.

Stated by Meta

AugOctDecFebAprKnowledge cutoffAugust 2024ReleasedApr 5, 2025

How large is the context window of Llama 4 Scout 17B Instruct?

10 M tokens. That is how much text Llama 4 Scout 17B Instruct reads at once, about 15,000 A4 pages.

Page count as a rule of thumb from Artificial Analysis

Context windowTokens the model reads at once · higher is better

Llama 4 Scout 17B InstructRank 2 of 10310M

GPT-6 Astra1.05M

GPT-6.1 Sol1.05M

GPT-6 Sol1.05M

GPT-5.6 Sol1.05M

GPT-6.1 Terra1.05M

GPT-5.51.05M

Can I run Llama 4 Scout 17B Instruct locally?

Yes, the weights are open. With 109B parameters it needs a lot of memory, more a server than a laptop.

Stated by Meta

Open weights: yes
License: Llama 4 Community
Parameters: 109B
active per token: 17B

Sources and data date

Show all 29 sources

Benchmark sources at a glance

Which independent evaluators tested this model. This is coverage, not a quality ranking.

Evaluator	Test series	Composite indices	Recorded active measurements
LMArena	0	0	1
SWE-bench	0	0	1

Without vendor claims and historical or disputed results; those stay labelled in the single values. 2 measurements are not yet assigned to a test series.

Type	Evidence and data date
Input	Documented modalities. Retrieved October 27, 2026. Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026) · Meta model catalog snapshot · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved October 27, 2026. Source: models.dev catalog · models.dev catalog (retrieved October 27, 2026) · Meta model catalog snapshot · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 22 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not published	per-row evaluation N and confidence interval Note The paper describes dataset sizes but reports no model-row sample size or confidence interval.
Unresolved	row-specific prompt, route, and checkpoint revision Note Study-level protocol is described, but no prompt variant, serving route, or checkpoint revision is bound to this row.
Not published	evaluated N Original wording The leaderboard reports preference-vote totals but no fixed task-case N; Votes are not copied to sampleCount.
Unresolved	confidence interval Original wording The Score cell displays ±, but the official policy does not define that notation as a confidence interval; no CI field is populated.
Unresolved	score direction Original wording The captured source does not explicitly state higher-is-better or lower-is-better; higherIsBetter remains null.
Not published	per-model prompts and generation settings Original wording No exact prompt, decoding settings, or serving configuration is published in the captured row.
Additional source	Hugging Face (retrieved October 15, 2026; October 28, 2026) · Hugging Face, Llama 4 Scout 17B Instruct · Editorial description reviewed October 2, 2026
Additional source	SWE-bench (retrieved October 16, 2026)
Additional source	Meta Llama 4 model card (retrieved September 8, 2026; October 28, 2026)
Additional source	Meta Llama-verifications benchmark report, pinned commit 9a53f29 (retrieved October 17, 2026)
Additional source	Robustness as an Emergent Property of Task Performance (retrieved October 17, 2026; October 24, 2026)
Additional source	EFFIBENCH-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code (retrieved October 17, 2026)
Additional source	SpatialViz-Bench, anonymous ICLR 2026 submission (retrieved October 18, 2026)
Additional source	ACL Anthology, Findings of EMNLP 2025, From KMMLU-Redux to Pro (retrieved October 18, 2026)
Additional source	Arena leaderboard (retrieved October 18, 2026)
Additional source	Meta Llama Verifications Benchmark Evaluation Report (retrieved October 19, 2026)
Additional source	NVIDIA NIM for Vision Language Models 2.0.0, Llama 4 Scout benchmarking results (retrieved October 20, 2026)
Additional source	BigCodeBench official leaderboard (retrieved October 28, 2026)1 2
Additional source	Model metadata (retrieved October 16, 2026)1 2
Additional source	Model metadata (retrieved September 8, 2026)
Additional source	Model metadata (retrieved October 17, 2026)
Additional source	Model metadata (retrieved October 18, 2026)
Additional source	Model metadata (retrieved October 28, 2026)1 2 3 4
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0