Language modelActive

DeepSeek-R1

DeepSeek

Released: January 2025
Data date: September 8, 2026

Intelligence: Not ranked
Overall score
Price: $0.7 · $2.5
Input · Output / 1M tokens

No published category-wide ranking is available for this model.

Entries 1-12 of 91. Page 1 of 8.

Published benchmarks

Each chart compares models in the same documented test and highlights the current model.

Coding

3 measurements

Aider Polyglot

Overall

What this benchmark measures: 225 code-editing tasks from Exercism across C++, Go, Java, JavaScript, Python, and Rust.

Context: Aider, the edit format, reasoning budget, and two attempts are part of the result.

Higher is better4 modelsDeepSeek-R1: Rank 4 of 4Data date: September 2, 2026

1.o3-pro84.9%

2.Gemini 2.5 Pro83.1%

3.o381.3%

4.DeepSeek-R171.4%

Scale: 0% - 100%

Source: Aider

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: aider-polyglot-pass2-diff
Metric: percent correct after two attempts
Scale: 0-100
Participants: 4
Methodology and context: 225 Exercism tasks across six languages using Aider and a documented edit format.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-0528. Evaluator: Aider. Dataset type: public. Leaderboard date: 2025-06-06. Attempts: pass 2. Latency in seconds: 716.6. Cost per task in USD: 0.021340444444444443. Source ID: aider. Evidence capture ID: curated-aider-2026-09-02-83be0654f25d. Source payload hash: 83be0654f25dcee536c478783a0547fbe586876fabb5c3f977893c0de30a9531. Parser version: curated-v1. Imported at: 2026-09-04

Source: Aider

ScienceAgentBench

Accuracy

What this benchmark measures: 102 expert-validated tasks for data-driven scientific work.

Context: The scaffold, tools, and code execution are part of the result. HAL reports both 42 and 44 underlying publications in different places.

Higher is better5 modelsDeepSeek-R1: Rank 4 of 5Data date: August 6, 2026

1.o333.33%

2.GPT-530.39%

3.GPT-4.124.51%

4.DeepSeek-R123.53%

5.DeepSeek-V315.69%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-scienceagentbench-sab-self-debug-overall
Metric: accuracy
Scale: 0-100
Participants: 5
Methodology and context: SAB Self-Debug on 102 scientific tasks. Total cost is divided by 102 and excludes caching benefits according to HAL.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: SAB Self-Debug. Agent: SAB Self-Debug. Cost per task in USD: 0.1788235294117647. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-6fc49efd47b7. Source payload hash: 6fc49efd47b70abe914ca04758101a25125a9e8a20c5a6c90019906650d6beee. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

USACO

Accuracy

What this benchmark measures: 307 algorithmic programming tasks from Bronze through Platinum with exhaustive tests.

Context: The benchmark measures competitive programming, not repository navigation, maintainability, or product experience.

Higher is better5 modelsDeepSeek-R1: Rank 5 of 5Data date: August 6, 2026

1.GPT-569.06%

2.o346.25%

3.GPT-4.144.95%

4.DeepSeek-V339.09%

5.DeepSeek-R138.11%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-usaco-usaco-episodic-semantic-overall
Metric: accuracy
Scale: 0-100
Participants: 5
Methodology and context: USACO Episodic + Semantic on the same 307-problem split. Total cost is divided by 307 and excludes caching benefits.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: USACO Episodic + Semantic. Agent: USACO Episodic + Semantic. Cost per task in USD: 0.2607166123778502. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-b17824331ef0. Source payload hash: b17824331ef027e156721f99110a57b278ee78be01b8959a892bacfb3502ff67. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

Online Mind2Web

Accuracy

What this benchmark measures: 300 verified tasks across 136 dynamic live websites.

Context: SeeAct and Browser-Use are different agent scaffolds and therefore remain separate comparison cohorts.

Higher is better5 modelsDeepSeek-R1: Rank 5 of 5Data date: August 5, 2026

1.GPT-4.136.33%

2.DeepSeek-V332.33%

3.GPT-532%

4.o329%

5.DeepSeek-R125.33%

Scale: 0% - 100%

Source: Princeton HAL

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: hal-online-mind2web-browser-use-overall
Metric: accuracy
Scale: 0-100
Participants: 5
Methodology and context: The Browser-Use scaffold handles the same 300 tasks across 136 live websites. Total cost is divided by 300.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: Browser-Use. Agent: Browser-Use. Cost per task in USD: 0.9364333333333332. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-664f99d510f5. Source payload hash: 664f99d510f5321c20560c70c760680b5d40263821b082cc494f3bdfae6cc0cb. Parser version: curated-v1. Imported at: 2026-09-04

Source: Princeton HAL

Reasoning

6 measurements

LongBench v2

Easy

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 4 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro75%

2.Gemini 2.5 Flash72.3%

3.o166.8%

4.DeepSeek-R166.1%

5.GPT-4o54.2%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-easy
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-0a5a05f9e399. Source payload hash: 0a5a05f9e3998728b5a5b827562b52b9ca09a1367d7119c88a6029d109ea6ba1. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Hard

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 3 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro56.1%

2.Gemini 2.5 Flash55.8%

3.DeepSeek-R153.4%

4.o152.1%

5.GPT-4o49.7%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-hard
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-4bbc7fc01da6. Source payload hash: 4bbc7fc01da6ef1b5e372f2f17fe0849fdde69eabd7caecb2a887466f25e1590. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Long, 128,000 to 2 million words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 2 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro71%

2.DeepSeek-R159.3%

3.o158.1%

4.Gemini 2.5 Flash55.7%

5.GPT-4o43.5%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-long
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-82dc59c97071. Source payload hash: 82dc59c970714785cd17275ff52c361f3d2b8a5993aa36a66c75fdeeb5ae73f7. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Medium, 32,000 to 128,000 words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 3 of 5Data date: September 2, 2026

1.Gemini 2.5 Flash60%

2.Gemini 2.5 Pro56.3%

3.DeepSeek-R154.4%

4.o153.5%

5.GPT-4o48.6%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-medium
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-fb7d385c6728. Source payload hash: fb7d385c672839bb9d1ed3dc00b8c30983a1ce5c55a3eb9d36472f825cd289e0. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Overall

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 3 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro63.3%

2.Gemini 2.5 Flash62.1%

3.DeepSeek-R158.3%

4.o157.7%

5.GPT-4o51.4%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-overall
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-6ffc97a26a0d. Source payload hash: 6ffc97a26a0d168714d6ad2c17b12e71f02f861a991e218b4e26bc83961982ac. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Short, up to 32,000 words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsDeepSeek-R1: Rank 4 of 5Data date: September 2, 2026

1.Gemini 2.5 Flash68.3%

2.Gemini 2.5 Pro67.2%

3.o162.6%

4.DeepSeek-R162.2%

5.GPT-4o59.6%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-short
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-95504bd97ff3. Source payload hash: 95504bd97ff39c57c15186af68e6f7d6c9ec26f875b04b56acd8b82b88cb1b28. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

Knowledge

2 measurements

MMLU-ProX

Average across 29 languages

What this benchmark measures: 11,829 parallel questions per language compare knowledge and reasoning across 29 languages.

Context: Results use five-shot chain of thought and are not interchangeable with MMLU-Pro.

Higher is better4 modelsDeepSeek-R1: Rank 1 of 4Data date: August 9, 2026

1.DeepSeek-R175.5%

2.GPT-4.172.7%

3.DeepSeek-V370.5%

4.o4-mini69.3%

Scale: 0% - 100%

Source: MMLU-ProX

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: mmlu-prox:official:avg29
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: MMLU-ProX. Dataset type: public. Source ID: mmlu-prox. Evidence capture ID: curated-mmlu-prox-2026-08-09-8f9f86134c2e. Source payload hash: 8f9f86134c2ebc8925a6d4ff3dbb89da6047f972657ea0db5d5844f2ec5914a1. Parser version: curated-v1. Imported at: 2026-09-04

Source: MMLU-ProX

MMLU-ProX

German

What this benchmark measures: 11,829 parallel questions per language compare knowledge and reasoning across 29 languages.

Context: Results use five-shot chain of thought and are not interchangeable with MMLU-Pro.

Higher is better4 modelsDeepSeek-R1: Rank 1 of 4Data date: August 9, 2026

1.DeepSeek-R176.7%

2.GPT-4.176.4%

3.DeepSeek-V375.1%

4.o4-mini73.5%

Scale: 0% - 100%

Source: MMLU-ProX

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: mmlu-prox:official:de
Metric: accuracy
Scale: 0-100
Participants: 4
Methodology and context: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: MMLU-ProX. Dataset type: public. Source ID: mmlu-prox. Evidence capture ID: curated-mmlu-prox-2026-08-09-e0ead6d94ec9. Source payload hash: e0ead6d94ec90867a9587837a0fe3c52850e102d710bae5ca6e97bf6b86470e1. Parser version: curated-v1. Imported at: 2026-09-04

Source: MMLU-ProX

Recorded measurement details

Every observation retains its source value and published test conditions.

Aider Polyglot: Overall (Show measurement, test conditions, and source)

Source value: 71.4
Score: 71.4
Metric: percent correct after two attempts
Unit: %
Benchmark version: 225-case polyglot suite
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: aider-polyglot-pass2-diff
Source type: official-leaderboard
Evaluator: Aider
Status: active
Retrieved at: 2026-09-02
Methodology: 225 Exercism tasks across six languages using Aider and a documented edit format.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-0528. Evaluator: Aider. Dataset type: public. Leaderboard date: 2025-06-06. Attempts: pass 2. Latency in seconds: 716.6. Cost per task in USD: 0.021340444444444443. Source ID: aider. Evidence capture ID: curated-aider-2026-09-02-83be0654f25d. Source payload hash: 83be0654f25dcee536c478783a0547fbe586876fabb5c3f977893c0de30a9531. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 225 Exercism tasks across six languages using Aider and a documented edit format.

Source Aider

Online Mind2Web: Accuracy (Show measurement, test conditions, and source)

Source value: 25.33
Score: 25.33
Metric: accuracy
Unit: %
Benchmark version: 300-task live set
Category: agentic
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-online-mind2web-browser-use-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-05
Methodology: The Browser-Use scaffold handles the same 300 tasks across 136 live websites. Total cost is divided by 300.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: Browser-Use. Agent: Browser-Use. Cost per task in USD: 0.9364333333333332. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-05-664f99d510f5. Source payload hash: 664f99d510f5321c20560c70c760680b5d40263821b082cc494f3bdfae6cc0cb. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: The Browser-Use scaffold handles the same 300 tasks across 136 live websites. Total cost is divided by 300.

Source Princeton HAL

ScienceAgentBench: Accuracy (Show measurement, test conditions, and source)

Source value: 23.53
Score: 23.53
Metric: accuracy
Unit: %
Benchmark version: verified 102-task set
Category: coding
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-scienceagentbench-sab-self-debug-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: SAB Self-Debug on 102 scientific tasks. Total cost is divided by 102 and excludes caching benefits according to HAL.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: SAB Self-Debug. Agent: SAB Self-Debug. Cost per task in USD: 0.1788235294117647. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-6fc49efd47b7. Source payload hash: 6fc49efd47b70abe914ca04758101a25125a9e8a20c5a6c90019906650d6beee. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: SAB Self-Debug on 102 scientific tasks. Total cost is divided by 102 and excludes caching benefits according to HAL.

Source Princeton HAL

USACO: Accuracy (Show measurement, test conditions, and source)

Source value: 38.11
Score: 38.11
Metric: accuracy
Unit: %
Benchmark version: HAL 307-problem set
Category: coding
Task: overall
Task label: Accuracy
Direction: Higher is better
Comparison cohort: hal-usaco-usaco-episodic-semantic-overall
Source type: independent-evaluator
Evaluator: Princeton HAL
Status: active
Retrieved at: 2026-08-06
Methodology: USACO Episodic + Semantic on the same 307-problem split. Total cost is divided by 307 and excludes caching benefits.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: Princeton HAL. Dataset type: public. Harness: USACO Episodic + Semantic. Agent: USACO Episodic + Semantic. Cost per task in USD: 0.2607166123778502. Source ID: princeton-hal. Evidence capture ID: curated-princeton-hal-2026-08-06-b17824331ef0. Source payload hash: b17824331ef027e156721f99110a57b278ee78be01b8959a892bacfb3502ff67. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: USACO Episodic + Semantic on the same 307-problem split. Total cost is divided by 307 and excludes caching benefits.

Source Princeton HAL

LongBench v2: Easy (Show measurement, test conditions, and source)

Source value: 66.1
Score: 66.1
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: easy
Task label: Easy
Direction: Higher is better
Comparison cohort: longbench-v2-cot-easy
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-0a5a05f9e399. Source payload hash: 0a5a05f9e3998728b5a5b827562b52b9ca09a1367d7119c88a6029d109ea6ba1. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Hard (Show measurement, test conditions, and source)

Source value: 53.4
Score: 53.4
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: hard
Task label: Hard
Direction: Higher is better
Comparison cohort: longbench-v2-cot-hard
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-4bbc7fc01da6. Source payload hash: 4bbc7fc01da6ef1b5e372f2f17fe0849fdde69eabd7caecb2a887466f25e1590. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Long, 128,000 to 2 million words (Show measurement, test conditions, and source)

Source value: 59.3
Score: 59.3
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: long
Task label: Long, 128,000 to 2 million words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-long
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-82dc59c97071. Source payload hash: 82dc59c970714785cd17275ff52c361f3d2b8a5993aa36a66c75fdeeb5ae73f7. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Medium, 32,000 to 128,000 words (Show measurement, test conditions, and source)

Source value: 54.4
Score: 54.4
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: medium
Task label: Medium, 32,000 to 128,000 words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-medium
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-fb7d385c6728. Source payload hash: fb7d385c672839bb9d1ed3dc00b8c30983a1ce5c55a3eb9d36472f825cd289e0. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Overall (Show measurement, test conditions, and source)

Source value: 58.3
Score: 58.3
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: longbench-v2-cot-overall
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-6ffc97a26a0d. Source payload hash: 6ffc97a26a0d168714d6ad2c17b12e71f02f861a991e218b4e26bc83961982ac. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Short, up to 32,000 words (Show measurement, test conditions, and source)

Source value: 62.2
Score: 62.2
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: short
Task label: Short, up to 32,000 words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-short
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Model snapshot: deepseek-r1-2025-01-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2025-01-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-95504bd97ff3. Source payload hash: 95504bd97ff39c57c15186af68e6f7d6c9ec26f875b04b56acd8b82b88cb1b28. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

MMLU-ProX: Average across 29 languages (Show measurement, test conditions, and source)

Source value: 75.5
Score: 75.5
Metric: accuracy
Unit: %
Benchmark version: EMNLP 2025 full release
Category: knowledge
Task: avg29
Task label: Average across 29 languages
Direction: Higher is better
Comparison cohort: mmlu-prox:official:avg29
Source type: official-leaderboard
Evaluator: MMLU-ProX
Status: active
Retrieved at: 2026-08-09
Methodology: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: MMLU-ProX. Dataset type: public. Source ID: mmlu-prox. Evidence capture ID: curated-mmlu-prox-2026-08-09-8f9f86134c2e. Source payload hash: 8f9f86134c2ebc8925a6d4ff3dbb89da6047f972657ea0db5d5844f2ec5914a1. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.

Source MMLU-ProX

MMLU-ProX: German (Show measurement, test conditions, and source)

Source value: 76.7
Score: 76.7
Metric: accuracy
Unit: %
Benchmark version: EMNLP 2025 full release
Category: knowledge
Task: de
Task label: German
Direction: Higher is better
Comparison cohort: mmlu-prox:official:de
Source type: official-leaderboard
Evaluator: MMLU-ProX
Status: active
Retrieved at: 2026-08-09
Methodology: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.. Source model ID: curated/deepseek-r1. Source model name: DeepSeek-R1. Provider reported by source: DeepSeek. Evaluator: MMLU-ProX. Dataset type: public. Source ID: mmlu-prox. Evidence capture ID: curated-mmlu-prox-2026-08-09-e0ead6d94ec9. Source payload hash: e0ead6d94ec90867a9587837a0fe3c52850e102d710bae5ca6e97bf6b86470e1. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: Full 29-language dataset with five-shot chain of thought. The source states August 2025 without a verifiable publication day.

Source MMLU-ProX

Our model tests

Community tests you can inspect

Compare how the models respond to the same prompt. Each test shows the first attempt, with no subsequent fixes to the generated code. These results do not contribute to an overall score.

Pelican on a bicycle

A community classic for free-form SVG drawing.

DeepSeek-R1

Not tested yet

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

DeepSeek-R1

Published benchmarks

Coding

Aider Polyglot

ScienceAgentBench

USACO

Online Mind2Web

Reasoning

LongBench v2

LongBench v2

LongBench v2

LongBench v2

LongBench v2

LongBench v2

Knowledge

MMLU-ProX

MMLU-ProX

Recorded measurement details

Community tests you can inspect

Pelican on a bicycle

Specifications and access

Additional source observations

Published prices

100 API requests

Compare this model

Sources and data date