Language modelActive

GPT-4o

OpenAI

Released: May 13, 2024
Data date: September 8, 2026

Intelligence: Not ranked
Overall score
Speed: 154 tokens/s
Output throughput
Price: $2.5 · $10
Input · Output / 1M tokens
Input: Text, Image
Output: Text

No published category-wide ranking is available for this model.

Entries 1-12 of 166. Page 1 of 14.

Published benchmarks

Each chart compares models in the same documented test and highlights the current model.

Reasoning

8 measurements

LongBench v2

Easy

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro75%

2.Gemini 2.5 Flash72.3%

3.o166.8%

4.DeepSeek-R166.1%

5.GPT-4o54.2%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-easy
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-bfa6246ba625. Source payload hash: bfa6246ba62584289e0204aefc1865fb285a328eef654dcce4382a41b68a3804. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Hard

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro56.1%

2.Gemini 2.5 Flash55.8%

3.DeepSeek-R153.4%

4.o152.1%

5.GPT-4o49.7%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-hard
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-185c2fd32a7b. Source payload hash: 185c2fd32a7bedc4dcf0643d2e27095af0b394e9b8a14d95e0c6b0feb36245ac. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Long, 128,000 to 2 million words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro71%

2.DeepSeek-R159.3%

3.o158.1%

4.Gemini 2.5 Flash55.7%

5.GPT-4o43.5%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-long
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-221d7b98957d. Source payload hash: 221d7b98957db163a9e154e9a456439363305e450a9acf020a33ad2fce54bf28. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Medium, 32,000 to 128,000 words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Flash60%

2.Gemini 2.5 Pro56.3%

3.DeepSeek-R154.4%

4.o153.5%

5.GPT-4o48.6%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-medium
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-d97a751d30a9. Source payload hash: d97a751d30a9e32384a7ce4bdbc341c9dd8f97266490b537b19b7b3026c0fb43. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Overall

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Pro63.3%

2.Gemini 2.5 Flash62.1%

3.DeepSeek-R158.3%

4.o157.7%

5.GPT-4o51.4%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-overall
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-69320d71d1be. Source payload hash: 69320d71d1be0f7687a4c525303c3abb0ca327d9b7fd521d884acd561a44656d. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

LongBench v2

Short, up to 32,000 words

What this benchmark measures: 503 tasks with contexts from 8,000 to 2 million words cover documents, dialogue, code repositories, and structured data.

Context: Scores with chain of thought are not interchangeable with direct-answer results without reasoning.

Higher is better5 modelsGPT-4o: Rank 5 of 5Data date: September 2, 2026

1.Gemini 2.5 Flash68.3%

2.Gemini 2.5 Pro67.2%

3.o162.6%

4.DeepSeek-R162.2%

5.GPT-4o59.6%

Scale: 0% - 100%

Source: LongBench

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: longbench-v2-cot-short
Metric: accuracy with chain of thought
Scale: 0-100
Participants: 5
Methodology and context: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-7555ce91dc0b. Source payload hash: 7555ce91dc0bbf8b98caf76a0a7c5981a69b3ee6a8ecc938a1915be2133d61cd. Parser version: curated-v1. Imported at: 2026-09-04

Source: LongBench

GPQA Diamond

Few-Shot CoT

Higher is better2 modelsGPT-4o: Rank 1 of 2Data date: August 11, 2026

1.GPT-4o54.04%

2.GPT-4o mini44.95%

Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-gpqa:1:diamond_few_shot_cot
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 15.717. Standard error: 3.542. Cost per task in USD: 0.011827. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-08-11-d3325290a404. Source payload hash: d3325290a40411054e05ecb86812143465b4d8eee24a0f71f6cf5dfee5b232a1. Parser version: vals-astro-v2. Imported at: 2026-08-11

Source: Vals AI

GPQA Diamond

Zero-Shot CoT

Higher is better2 modelsGPT-4o: Rank 1 of 2Data date: August 11, 2026

1.GPT-4o53.54%

2.GPT-4o mini43.43%

Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-gpqa:1:diamond_zero_shot_cot
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 14.131. Standard error: 3.55. Cost per task in USD: 0.007103. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-08-11-d3325290a404. Source payload hash: d3325290a40411054e05ecb86812143465b4d8eee24a0f71f6cf5dfee5b232a1. Parser version: vals-astro-v2. Imported at: 2026-08-11

Source: Vals AI

TutorBench

Overall

What this benchmark measures: 1,490 tasks test adaptive explanations, feedback, and hints across six STEM subjects.

Context: A Claude 4 Sonnet judge scores weighted rubrics, so strong results do not prove safe tutoring of real learners.

Higher is better8 modelsGPT-4o: Rank 8 of 8Data date: August 26, 2026

1.GPT-5.4 Pro56.62%

2.Gemini 2.5 Pro55.65%

3.GPT-555.33%

4.o3-pro54.62%

5.Gemini 3.1 Pro Preview52.99%

6.Gemini 3.1 Flash-Lite51.5%

7.Llama 4 Maverick40.2%

8.GPT-4o36.12%

Scale: 0% - 100%

Source: Scale AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: scale-tutorbench-overall
Metric: weighted rubric score
Scale: 0-100
Participants: 8
Methodology and context: 1,490 tasks with weighted rubrics and Claude 4 Sonnet as judge.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Evaluator: Scale AI. Dataset type: public. Attempts: pass@1. Score display: 36.12% ± 0.96. Source ID: scale-ai. Evidence capture ID: curated-scale-ai-2026-08-26-2d47f3f45c11. Source payload hash: 2d47f3f45c11d5dfb09f242e535afc28a9fd65f5672d1a6a41964d0c2ec8c719. Parser version: curated-v1. Imported at: 2026-09-04

Source: Scale AI

Coding

3 measurements

LiveCodeBench v6

Easy

Higher is better2 modelsGPT-4o: Rank 1 of 2Data date: August 11, 2026

1.GPT-4o86.03%

2.GPT-4o mini60.87%

Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-lcb:1:easy
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 2.741. Standard error: 1.932. Cost per task in USD: 0.00269. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11

Source: Vals AI

LiveCodeBench v6

Hard

Higher is better2 modelsGPT-4o: Rank 1 of 2Data date: August 11, 2026

1.GPT-4o5.14%

2.GPT-4o mini1.43%

Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-lcb:1:hard
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 5.656. Standard error: 1.181. Cost per task in USD: 0.005244. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11

Source: Vals AI

LiveCodeBench v6

Medium

Higher is better2 modelsGPT-4o: Rank 1 of 2Data date: August 11, 2026

1.GPT-4o39.16%

2.GPT-4o mini16.97%

Scale: 0% - 100%

Source: Vals AI

CC BY 4.0

gradually.ai

Methodology and context

Comparison cohort: vals-lcb:1:medium
Metric: accuracy
Scale: 0-100
Participants: 2
Methodology and context: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 4.643. Standard error: 2.494. Cost per task in USD: 0.004063. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11

Source: Vals AI

Recorded measurement details

Every observation retains its source value and published test conditions.

LongBench v2: Easy (Show measurement, test conditions, and source)

Source value: 54.2
Score: 54.2
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: easy
Task label: Easy
Direction: Higher is better
Comparison cohort: longbench-v2-cot-easy
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-bfa6246ba625. Source payload hash: bfa6246ba62584289e0204aefc1865fb285a328eef654dcce4382a41b68a3804. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Hard (Show measurement, test conditions, and source)

Source value: 49.7
Score: 49.7
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: hard
Task label: Hard
Direction: Higher is better
Comparison cohort: longbench-v2-cot-hard
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-185c2fd32a7b. Source payload hash: 185c2fd32a7bedc4dcf0643d2e27095af0b394e9b8a14d95e0c6b0feb36245ac. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Long, 128,000 to 2 million words (Show measurement, test conditions, and source)

Source value: 43.5
Score: 43.5
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: long
Task label: Long, 128,000 to 2 million words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-long
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-221d7b98957d. Source payload hash: 221d7b98957db163a9e154e9a456439363305e450a9acf020a33ad2fce54bf28. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Medium, 32,000 to 128,000 words (Show measurement, test conditions, and source)

Source value: 48.6
Score: 48.6
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: medium
Task label: Medium, 32,000 to 128,000 words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-medium
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-d97a751d30a9. Source payload hash: d97a751d30a9e32384a7ce4bdbc341c9dd8f97266490b537b19b7b3026c0fb43. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Overall (Show measurement, test conditions, and source)

Source value: 51.4
Score: 51.4
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: longbench-v2-cot-overall
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-69320d71d1be. Source payload hash: 69320d71d1be0f7687a4c525303c3abb0ca327d9b7fd521d884acd561a44656d. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

LongBench v2: Short, up to 32,000 words (Show measurement, test conditions, and source)

Source value: 59.6
Score: 59.6
Metric: accuracy with chain of thought
Unit: %
Benchmark version: v2
Category: reasoning
Task: short
Task label: Short, up to 32,000 words
Direction: Higher is better
Comparison cohort: longbench-v2-cot-short
Source type: official-leaderboard
Evaluator: LongBench
Status: active
Retrieved at: 2026-09-02
Methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: LongBench. Dataset type: public. Leaderboard date: 2024-11-20. Attempts: pass@1. Reasoning effort: chain of thought. Source ID: longbench. Evidence capture ID: curated-longbench-2026-09-02-7555ce91dc0b. Source payload hash: 7555ce91dc0bbf8b98caf76a0a7c5981a69b3ee6a8ecc938a1915be2133d61cd. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 503 multiple-choice tasks with the prescribed reasoning prompt.

Source LongBench

TutorBench: Overall (Show measurement, test conditions, and source)

Source value: 36.12
Score: 36.12
Metric: weighted rubric score
Unit: %
Benchmark version: 1,490-task release
Category: education
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: scale-tutorbench-overall
Confidence interval: 0.96
Source type: official-leaderboard
Evaluator: Scale AI
Status: active
Retrieved at: 2026-08-26
Methodology: 1,490 tasks with weighted rubrics and Claude 4 Sonnet as judge.. Source model ID: curated/gpt-4o. Source model name: GPT-4o. Provider reported by source: OpenAI. Evaluator: Scale AI. Dataset type: public. Attempts: pass@1. Score display: 36.12% ± 0.96. Source ID: scale-ai. Evidence capture ID: curated-scale-ai-2026-08-26-2d47f3f45c11. Source payload hash: 2d47f3f45c11d5dfb09f242e535afc28a9fd65f5672d1a6a41964d0c2ec8c719. Parser version: curated-v1. Imported at: 2026-09-04
Comparison methodology: 1,490 tasks with weighted rubrics and Claude 4 Sonnet as judge.

Source Scale AI

GPQA Diamond: Few-Shot CoT (Show measurement, test conditions, and source)

Source value: 54.04
Score: 54.04
Metric: accuracy
Unit: %
Benchmark version: 1
Category: reasoning
Task: diamond_few_shot_cot
Task label: Few-Shot CoT
Direction: Higher is better
Comparison cohort: vals-gpqa:1:diamond_few_shot_cot
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 15.717. Standard error: 3.542. Cost per task in USD: 0.011827. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-08-11-d3325290a404. Source payload hash: d3325290a40411054e05ecb86812143465b4d8eee24a0f71f6cf5dfee5b232a1. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384

Source Vals AI

GPQA Diamond: Zero-Shot CoT (Show measurement, test conditions, and source)

Source value: 53.535
Score: 53.535
Metric: accuracy
Unit: %
Benchmark version: 1
Category: reasoning
Task: diamond_zero_shot_cot
Task label: Zero-Shot CoT
Direction: Higher is better
Comparison cohort: vals-gpqa:1:diamond_zero_shot_cot
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 14.131. Standard error: 3.55. Cost per task in USD: 0.007103. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-08-11-d3325290a404. Source payload hash: d3325290a40411054e05ecb86812143465b4d8eee24a0f71f6cf5dfee5b232a1. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384

Source Vals AI

LiveCodeBench v6: Easy (Show measurement, test conditions, and source)

Source value: 86.025
Score: 86.025
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: easy
Task label: Easy
Direction: Higher is better
Comparison cohort: vals-lcb:1:easy
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 2.741. Standard error: 1.932. Cost per task in USD: 0.00269. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384

Source Vals AI

LiveCodeBench v6: Hard (Show measurement, test conditions, and source)

Source value: 5.143
Score: 5.143
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: hard
Task label: Hard
Direction: Higher is better
Comparison cohort: vals-lcb:1:hard
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 5.656. Standard error: 1.181. Cost per task in USD: 0.005244. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384

Source Vals AI

LiveCodeBench v6: Medium (Show measurement, test conditions, and source)

Source value: 39.164
Score: 39.164
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: medium
Task label: Medium
Direction: Higher is better
Comparison cohort: vals-lcb:1:medium
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-08-11
Methodology: Source model ID: openai/gpt-4o-2024-11-20. Source model name: GPT-4o. Provider reported by source: OpenAI. Model snapshot: gpt-4o-2024-11-20. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-08-09. Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384. Latency in seconds: 4.643. Standard error: 2.494. Cost per task in USD: 0.004063. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-08-11-0342bcc364e4. Source payload hash: 0342bcc364e4426ffb24589c2752c9225be61fbd26db8deba5bdab9775ce5a75. Parser version: vals-astro-v2. Imported at: 2026-08-11
Comparison methodology: Reasoning effort: high. Temperature: 1. Maximum output tokens: 16384

Source Vals AI

Our model tests

Community tests you can inspect

Compare how the models respond to the same prompt. Each test shows the first attempt, with no subsequent fixes to the generated code. These results do not contribute to an overall score.

Pelican on a bicycle

A community classic for free-form SVG drawing.

GPT-4o

Not tested yet

Prompt and test conditions

Generate an SVG of a pelican riding a bicycle

Token limit including reasoning: 8,192 tokens

Task origin (Simon Willison)

Evidence

Sources and data date

Every statement links to its underlying documentation or leaderboard.

Speed: OpenAI, Nov. Output speed after the first streaming chunk, not total response time. Classified as deprecated by Artificial Analysis. Artificial Analysis maintains only the default 10K-input workload there. Retrieved September 8, 2026.
Artificial Analysis Methodology
Price: Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs.
Pricing
Input: Documented modalities. Retrieved September 8, 2026.
Model documentation
Output: Documented modalities. Retrieved September 8, 2026.
Model documentation

Research date September 8, 2026. 6 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.

Open evidence questions and research notes (original wording)

Not found in the inspected sources

independent-benchmarks. The recorded GPT-4o benchmark rows are official vendor evaluations; no independent evaluator row was merged.

Not found in the inspected sources

parameters-billions. OpenAI does not publish parameter counts.

Model metadataSource
Context window dataSource
API pricing (retrieved September 8, 2026)Source
LongBench (retrieved September 2, 2026)Source
Scale AI (retrieved August 26, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
Vals AI (retrieved August 11, 2026; September 4, 2026)Source
OpenAI (retrieved September 8, 2026)Source
OpenAI (retrieved September 8, 2026)Source
Artificial Analysis (retrieved September 8, 2026)Source
Speed measurement methodologySource

GPT-4o

Published benchmarks

Reasoning

LongBench v2

LongBench v2

LongBench v2

LongBench v2

LongBench v2

LongBench v2

GPQA Diamond

GPQA Diamond

TutorBench

Coding

LiveCodeBench v6

LiveCodeBench v6

LiveCodeBench v6

Recorded measurement details

Community tests you can inspect

Pelican on a bicycle

Specifications and access

Additional source observations

Published prices

100 API requests

Compare this model

Sources and data date