Llama 3.1 8B: Specifications & Sources

Specifications and access

Specification	Value and source
Weights	Open weightsSource not specified
Access	API, LocalSource
Parameters	8BSource not specified
Context window	131,072 tokensSource
Knowledge cutoff	December 2023Source
License	Llama 3.1 Community LicenseSource
Release	2024-07-23Source
Parameters	8 BSource
Architecture	dense transformer with GQASource
Input	TextSource
Output	Text, codeSource
Training tokens	15T+Source
Knowledge cutoff	2023-12Source

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

MMLU instruct (Show measurement, test conditions, and source)

Source value: 69.4
Score: 69.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table; 5-shot

Source Meta Llama 3.1 model card

MMLU CoT (Show measurement, test conditions, and source)

Source value: 73
Score: 73
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table; CoT

Source Meta Llama 3.1 model card

MMLU Pro instruct (Show measurement, test conditions, and source)

Source value: 48.3
Score: 48.3
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

IFEval (Show measurement, test conditions, and source)

Source value: 80.4
Score: 80.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

ARC instruct (Show measurement, test conditions, and source)

Source value: 83.4
Score: 83.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

GPQA (Show measurement, test conditions, and source)

Source value: 30.4
Score: 30.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

HumanEval (Show measurement, test conditions, and source)

Source value: 72.6
Score: 72.6
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

MBPP (Show measurement, test conditions, and source)

Source value: 72.8
Score: 72.8
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

MultiE HumanEval (Show measurement, test conditions, and source)

Source value: 50.8
Score: 50.8
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct multilingual code table

Source Meta Llama 3.1 model card

MultiE MBPP (Show measurement, test conditions, and source)

Source value: 52.4
Score: 52.4
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct multilingual code table

Source Meta Llama 3.1 model card

GSM8K (Show measurement, test conditions, and source)

Source value: 84.5
Score: 84.5
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table; CoT

Source Meta Llama 3.1 model card

MATH (Show measurement, test conditions, and source)

Source value: 51.9
Score: 51.9
Metric: percent
Unit: %
Direction: Higher is better
Source type: vendor-reported
Evaluator: Meta Llama 3.1 model card
Status: active
Retrieved at: 2026-09-08
Methodology: Vendor-reported result. Instruct table

Source Meta Llama 3.1 model card

Page 1 of 3

Our tests

Each test shows the latest attempt, with no subsequent fixes to the generated code. Earlier runs remain available in the test details. These results do not contribute to an overall score.

Pelican on a bicycle

A community classic for free-form SVG drawing.

Llama 3.1 8B

Reasoning (requested): not specified

Current attempt

Run date: 1 Oct 2026
API provider: openrouter
Requested API model: meta-llama/llama-3.1-8b-instruct
API model reported in response: meta-llama/llama-3.1-8b-instruct
Reasoning setting: not specified
Pinned endpoint: DeepInfra | meta-llama/llama-3.1-8b-instruct
Temperature: Provider default
Token limit including reasoning: 8,192 tokens
Test protocol: community-visual-tests-v1
Technical checks: Passed, not a quality rating
Response time: 33.6 s

Visually reviewed. This individual test does not establish overall model quality.

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

Popularity

27.6 / 100

Published prices

Hosting-market price: the median of 5 OpenRouter endpoints from 5 providers (representative: Groq), observed on October 12, 2026. Cache prices differ by provider and are not shown.

API input: $0.05 per 1M tokensSource
API output: $0.08 per 1M tokensSource

Sources and data date

Type	Evidence and data date
Speed	Median across serving providers. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Meta Llama 3.1 model card (retrieved September 8, 2026) · Meta Llama 3.1 model card · Editorial description reviewed October 4, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Meta Llama 3.1 model card (retrieved September 8, 2026) · Meta Llama 3.1 model card · Editorial description reviewed October 4, 2026
Research status	Research date September 8, 2026. 2 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	max-output-tokens Original wording No model-specific maximum output in card.
Additional source	Context window data
Additional source	Knowledge cutoff data · Meta Llama model documentation · Editorial description reviewed October 4, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Creative Commons Attribution 4.0