GPT-6 Astra
Unavailable in this round
Language modelLimited access
GPT-6 Astra
OpenAI
- Released
- September 3, 2026
- Data date
- September 24, 2026
- Predecessor
- GPT-5.6 Sol
- Benchmark-Meta-Score
- Provisional, 58.4 % panel coverage
- Speed
- 63.4 tokens/sOutput throughput
- Price
- Input · Output / 1M tokens
- Input
- Text, Image
- Output
- Text
With GPT-6 Astra, OpenAI introduced an experimental approach to long-term context in Codex. The model can keep notes across context windows and search earlier messages or tool results. During lengthy debugging work, fewer details need to be repeatedly compressed into a single summary.
OpenAI classifies Astra’s cybersecurity capability as Critical under its Preparedness Framework. The release therefore includes additional safeguards and restrictions on certain security tasks. GPT-6.1 Sol approaches Astra on several coding and professional-work evaluations, while OpenAI still reports an advantage for Astra on its hardest scientific tasks.
- Position
- Rank 2 of 24
- Index score
- 85.6 / 100
Methodology and source
The LLM comparison hub's selected-panel index averages rank percentiles from 10 documented comparison cohorts. Benchmark meta-scores are shown in the benchmark section.
Benchmark meta-analysis
Overall and area scores
These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.
Retrieval dates of the sources used 2026-10-15 - 2026-10-19
Overall index
Index 71.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 58 %
- Test families
- 41 / 71
- Sources
- 4
- Mathematical bounds with missing values
- 42.0-83.6 / 100
Coding
Index 88.6 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 66 %
- Test families
- 10 / 15
- Sources
- 3
- Mathematical bounds with missing values
- 58.1-92.5 / 100
Reasoning
Index 91.8 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 57 %
- Test families
- 5 / 7
- Sources
- 2
- Mathematical bounds with missing values
- 52.4-95.3 / 100
Mathematics
Index 93.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 60 %
- Test families
- 3 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 55.9-95.9 / 100
Knowledge
Index 97.2 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 67 %
- Test families
- 2 / 3
- Sources
- 2
- Mathematical bounds with missing values
- 64.8-98.1 / 100
Agents
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 42 %
- Test families
- 5 / 12
- Sources
- 1
- Mathematical bounds with missing values
- 36.3-94.6 / 100
General
Index 85.6 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 80 %
- Test families
- 4 / 5
- Sources
- 2
- Mathematical bounds with missing values
- 68.5-88.5 / 100
Finance
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 43 %
- Test families
- 3 / 7
- Sources
- 1
- Mathematical bounds with missing values
- 27.4-84.5 / 100
Legal
Index 64.9 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 2 / 4
- Sources
- 1
- Mathematical bounds with missing values
- 32.4-82.4 / 100
Health
Index 77.0 / 100
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 67 %
- Test families
- 2 / 3
- Sources
- 1
- Mathematical bounds with missing values
- 51.3-84.7 / 100
Cybersecurity
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 1 / 1
- Sources
- 1
- Mathematical bounds with missing values
- 9.1-9.1 / 100
Multimodal
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 38 %
- Test families
- 2 / 4
- Sources
- 2
- Mathematical bounds with missing values
- 30.0-92.5 / 100
Search
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 100 %
- Test families
- 1 / 1
- Sources
- 1
- Mathematical bounds with missing values
- 76.9-76.9 / 100
Education
Not enough coverage for a score yet
provisional
Published configurations differ within this comparison cohort. Treat the index as provisional.
- Panel-weight coverage
- 50 %
- Test families
- 1 / 2
- Sources
- 1
- Mathematical bounds with missing values
- 25.0-75.0 / 100
Documented method · Included test families and measurements
Method version: benchmark-meta-score-v1
The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture; when dates match, the wider slice is used. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. When selecting between multiple results, unknown or ambiguous tiers exclude the model from that series. Unambiguous single configurations do not require a tier ordering. The best individual result for each model is not selected. Equal raw values share the same rank percentile. Multiple series from the same evaluator within a family receive equal weight; evaluators within a family, families within an area, and areas in the overall index then receive equal weight. Unmeasured series remain visible in coverage and bounds. An overall score requires at least four areas, six test families, and two sources. An area score requires at least two test families. Every overall and area score also requires at least 50% of its panel weight.
Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.
Unmeasured portions are not entered as zero. They reduce coverage and contribute only to the displayed bounds. Scores with different coverage are therefore not equally complete.
The bounds hold observed percentiles fixed and allow missing contributions anywhere from 0 to 100. They are not confidence intervals or predictions of future recalculations.
The source count is not a count of independent experiments.
OECD handbook on composite indicatorsAll 41 test families and 12 of 45 raw measurements are listed here. The complete calculation for every model is available in the JSON export.
Test family: AIME
Mathematics · Family index: 90.9 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-otis-mock-aime-2024-2025
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.43 %
- Raw value:
- 1 ratio
- Metric:
- model_extracted_exact_match
- Version:
- 1.1.6
- Participating models:
- 23
- Rank percentile within the cohort:
- 90.9 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["OTIS Mock AIME 2024-2025","OTIS Mock AIME 2024-2025","1.1.6","model_extracted_exact_match"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gpt-6-astra_max; Epoch model label: GPT-6 Astra; Display label field: GPT-6 Astra (max); Reasoning setting encoded in the alias: max. Task ID: OTIS Mock AIME 2024-2025. Task version: 1.1.6. Run ID: L7kW8yxFKCjVXhxNnt3VTq. Started at 2026-08-30T14:57:43.000Z. Original task label: OTIS Mock AIME 2024-2025. The published row reports mean_score=1.0 and stderr=0.0. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: AA-AnalystAgent
Agents · Family index: 81.6 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-analyst-agent
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 51.24999999999999 %
- Metric:
- pass^5 reliability
- Version:
- AA cohort: 80 questions
- Participating models:
- 39
- Rank percentile within the cohort:
- 81.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-analyst-agent:AA cohort: 80 questions:standalone:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-analyst-agent","AA cohort: 80 questions","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup analyst agent
- Tool policy:
- 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission
- Attempts:
- pass^5, 5 independent attempts per question
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 80 quantitative questions across 14 domains are run five independent times. pass^5 counts only questions answered correctly in all five attempts.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup analyst agent. Tool policy: 100-turn cap; Python sandbox with source files, URL fetch, image viewing, and answer submission. Attempts: pass^5, 5 independent attempts per question. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 51.2%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Briefcase
Agents · Family index: 91.5 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-briefcase
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,568.99 Elo
- Metric:
- AA-Briefcase Elo
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 91.5 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-briefcase:v1.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-briefcase","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-LCR
Reasoning · Family index: 72.6 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-aa-lcr
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 80.6666666666667 %
- Metric:
- Pass rate
- Version:
- v1.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 72.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-aa-lcr:v1.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-aa-lcr","v1.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 80.7%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AutomationBench-AA
Agents · Family index: 97.6 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-automationbench-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 68.49174806688876 %
- Metric:
- Objectives completed without guardrail violations
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 97.6 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-automationbench-aa:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-automationbench-aa","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 68.5%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: FrontierCode 1.1 Main
Coding · Family index: 86.7 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-frontiercode
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.48 %
- Raw value:
- 46.9827666666667 %
- Metric:
- Mean score over three attempts per task
- Version:
- 1.1 Main
- Participating models:
- 16
- Rank percentile within the cohort:
- 86.7 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-frontiercode:1.1 Main:standalone:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-frontiercode","1.1 Main","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup coding agent
- Tool policy:
- 250-turn cap; repository worktree graded as left by the agent
- Attempts:
- Mean score, 3 attempts per task
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. The primary 100-task FrontierCode 1.1 set is run three times per task, with equal task weights. Failing any blocking check or being flagged by the trajectory scanner scores zero; runs passing every blocking check score 70 to 100 percent.. Harness: Artificial Analysis Stirrup. Agent: Stirrup coding agent. Tool policy: 250-turn cap; repository worktree graded as left by the agent. Attempts: Mean score, 3 attempts per task. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. The primary 100-task FrontierCode 1.1 set is run three times per task, with equal task weights. Failing any blocking check or being flagged by the trajectory scanner scores zero; runs passing every blocking check score 70 to 100 percent.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup coding agent. Tool policy: 250-turn cap; repository worktree graded as left by the agent. Attempts: Mean score, 3 attempts per task. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 47.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDP.pdf
Reasoning · Family index: 98.2 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdp-pdf
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 31 %
- Metric:
- All-pass rate
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 98.2 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdp-pdf:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdp-pdf","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 31.0%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: GDPval-AA
Agents · Family index: 81.7 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-gdpval-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 1,572.84 Elo
- Metric:
- GDPval-AA Elo
- Version:
- v2.1
- Participating models:
- 83
- Rank percentile within the cohort:
- 81.7 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-gdpval-aa:v2.1:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-gdpval-aa","v2.1","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: ITBench-AA (SRE)
Agents · Family index: 82.9 / 100Nominal family weight in the overall panel: 0.6 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-itbench-aa
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 0.6 %
- Raw value:
- 48.5875706214689 %
- Metric:
- Average precision at full recall
- Version:
- AA cohort: 59 tasks
- Participating models:
- 36
- Rank percentile within the cohort:
- 82.9 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-itbench-aa:AA cohort: 59 tasks:standalone:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-itbench-aa","AA cohort: 59 tasks","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Harness:
- Artificial Analysis Stirrup
- Agent:
- Stirrup agent
- Tool policy:
- Shell access to an offline Kubernetes incident snapshot
- Attempts:
- Average precision at full recall, 3 repeats per task
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption. 59 Kubernetes incident tasks, 40 public and 19 private, with three repeats per task. A run scores zero if it misses any true root cause; otherwise it receives its precision at full recall. This is the SRE slice of ITBench.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Harness: Artificial Analysis Stirrup. Agent: Stirrup agent. Tool policy: Shell access to an offline Kubernetes incident snapshot. Attempts: Average precision at full recall, 3 repeats per task. Reasoning: reasoning enabled. Reasoning effort: Max. Score display: 48.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: AA-Omniscience
Knowledge · Family index: 100.0 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Artificial Analysis
Published configurations differ within this cohort.
- Measurement:
- artificial-analysis-omniscience
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 2.38 %
- Raw value:
- 43.4 points
- Metric:
- AA-Omniscience Index
- Version:
- unspecified
- Participating models:
- 83
- Rank percentile within the cohort:
- 100.0 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- artificial-analysis-omniscience:unspecified:v4.3.2:reasoning:tokens=0:suffix=Max
- Reasoning tier:
- Max
- Documented comparison setup
- Publication cohort:
- ["artificial-analysis-omniscience","unspecified","aa-model-page-2026-10-19-8a022369738c"]
- Dataset type:
- artificial-analysis-evaluation
- Setup and model parameters
- Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Reasoning: reasoning enabled. Reasoning effort: Max
Exact AA configuration: GPT-6 Astra (Max) (release gpt-6-astra, row gpt-6-astra, UUID 2f339a97-9a0d-499a-9cb5-e0db665bfa25, reasoning=true, source reasoning-token configuration budget=0, suffix=Max). This configured value is not measured token consumption.. Source model ID: artificial-analysis/2f339a97-9a0d-499a-9cb5-e0db665bfa25/gpt-6-astra/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Model snapshot: GPT-6 Astra (Max). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Reasoning: reasoning enabled. Reasoning effort: Max. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-10-19-8a022369738c. Source payload hash: 8a022369738cbe2a0e2f9bf486a2882865dbb0c6f1396dfec84059a802560e3c. Parser version: artificial-analysis-rsc-models-v2. Imported at: 2026-10-19
- Artificial Analysis
Test family: Chess Puzzles
Reasoning · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-chess-puzzles
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 1.02 %
- Raw value:
- 0.74 ratio
- Metric:
- exact
- Version:
- 2.0.0
- Participating models:
- 22
- Rank percentile within the cohort:
- 100.0 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["Chess Puzzles","Chess Puzzles","2.0.0","exact"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gpt-6-astra_max; Epoch model label: GPT-6 Astra; Display label field: GPT-6 Astra (max); Reasoning setting encoded in the alias: max. Task ID: Chess Puzzles. Task version: 2.0.0. Run ID: KaEir82ukWPm2Fq4FanY2M. Started at 2026-10-13T13:42:38.000Z. Original task label: Chess Puzzles. The published row reports mean_score=0.74 and stderr=0.0440844002276808. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: SimpleQA Verified
Knowledge · Family index: 94.4 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
- Epoch AI benchmark run dataset snapshot
Published configurations differ within this cohort.
- Measurement:
- epoch-simpleqa-verified
- Series weight within the test family:
- 100 %
- Nominal contribution to the overall panel:
- 2.38 %
- Raw value:
- 0.756 ratio
- Metric:
- simpleqa_scorer
- Version:
- 1.2.0
- Participating models:
- 19
- Rank percentile within the cohort:
- 94.4 %
- Retrieved:
- 2026-10-19
- Comparison cohort:
- epoch-score-design-v1:["SimpleQA Verified","SimpleQA Verified (anti-abstention)","1.2.0","simpleqa_scorer"]
- Reasoning tier:
- max
- Documented comparison setup
- Publication cohort:
- epoch-score-design-v1:["SimpleQA Verified","SimpleQA Verified (anti-abstention)","1.2.0","simpleqa_scorer"]
- Dataset type:
- not-reported-in-captured-run-csv
- Setup and model parameters
- Published reasoning tier: max. The capture does not report other inference parameters or a compute budget.
Independent evaluation. Epoch run alias: gpt-6-astra_max; Epoch model label: GPT-6 Astra; Display label field: GPT-6 Astra (max); Reasoning setting encoded in the alias: max. Task ID: SimpleQA Verified. Task version: 1.2.0. Run ID: PppwnX7EjXx2SeTcFN8Ujn. Started at 2026-08-30T14:57:43.000Z. Original task label: SimpleQA Verified (anti-abstention). The published row reports mean_score=0.756 and stderr=0.013588548437881386. The CSV has no sample-count column, so none is inferred. Standard error is not presented as a confidence interval. Meta-score selection: Per model, the highest documented reasoning tier is selected within the same Epoch source test design, original task, version, and scorer, independently of the measured value. This derived score copy does not claim matched compute budgets; the capture publishes neither a common budget nor all inference parameters. The raw model-profile observation remains unchanged. Selected reasoning tier: max.
- Epoch AI benchmark run dataset snapshot
Test family: FrontierMath
Mathematics · Family index: 98.9 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: GPQA Diamond
Reasoning · Family index: 95.2 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
Test family: Humanity’s Last Exam
Reasoning · Family index: 97.6 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 50 %
Test family: LMArena Code Arena WebDev
Coding · Family index: 97.2 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: LMArena Search
Search · Family index: 76.9 / 100Nominal family weight in the overall panel: 7.14 % · Coverage of this test family: 100 %
Test family: LMArena Text, Style Control
General · Family index: 58.3 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: LMArena Vision, Style Control
Multimodal · Family index: 70.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MMMU-Pro
Multimodal · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 50 %
Test family: Terminal-Bench
Coding · Family index: 98.6 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 83 %
Test family: Terminal-Bench Science
Coding · Family index: 100.0 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: BioMysteryBench
General · Family index: 84.2 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: Code Migration
Coding · Family index: 98.4 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: CyberBench v1.1
Cybersecurity · Family index: 9.1 / 100Nominal family weight in the overall panel: 7.14 % · Coverage of this test family: 100 %
Test family: EMB
Finance · Family index: 86.4 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Finance Agent (v2)
Finance · Family index: 63.3 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: IOI
Coding · Family index: 100.0 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Harvey's Legal Agent Benchmark
Legal · Family index: 64.8 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: Legal Research Bench
Legal · Family index: 65.0 / 100Nominal family weight in the overall panel: 1.79 % · Coverage of this test family: 100 %
Test family: MedCode
Health · Family index: 70.1 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MedScribe
Health · Family index: 83.8 / 100Nominal family weight in the overall panel: 2.38 % · Coverage of this test family: 100 %
Test family: MysteryMechanism
General · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: ProgramBench
Coding · Family index: 93.9 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: ProofBench
Mathematics · Family index: 89.7 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: SAGE
Education · Family index: 50.0 / 100Nominal family weight in the overall panel: 3.57 % · Coverage of this test family: 100 %
Test family: SRE Bench
General · Family index: 100.0 / 100Nominal family weight in the overall panel: 1.43 % · Coverage of this test family: 100 %
Test family: SWE-bench Multimodal
Coding · Family index: 38.5 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Tax Agent Bench
Finance · Family index: 41.8 / 100Nominal family weight in the overall panel: 1.02 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench 1-100
Coding · Family index: 81.0 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Family index: 93.1 / 100Nominal family weight in the overall panel: 0.48 % · Coverage of this test family: 100 %
Benchmark sources at a glance
Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.
| Evaluator | Test series | Composite indices | Recorded active measurements |
|---|---|---|---|
| Artificial Analysis | 13 | 1 | 64 |
| Vals AI | 25 | 1 | 189 |
| LMArena | 4 | 0 | 42 |
| Epoch AI | 6 | 0 | 11 |
Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.
Page 1 of 30
Published benchmarks
Coding
SWE-Marathon v1.1
Overall
Long-horizon software engineering tasks with executable project tests. The model, reasoning effort and agent jointly define a result. The vendor table reports different values.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Claude Opus 5.546.3%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Claude Fable 5.145.6%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-swe-marathon-v1-1-max
- Metric
- pass@1
- Scale
- 0-100
- Participants
- 3
- Methodology and context
- Comparator score in Meta's published table, max reasoning. The agent is not documented for the scorecard cell.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-c06f95889965. Source payload hash: c06f9588996525fb84e3530a61520976ecb3d64f429cfb2c81a440add1a1553b. Parser version: curated-v1. Imported at: 2026-10-15
DeepSWE
Overall
113 tasks across 91 repositories and five languages. Max reasoning. Meta attributes these results to Datacurve; the Muse scores could not be found on its public leaderboard at retrieval. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Muse Wonder Full79%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Muse Wonder78.8%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Claude Opus 5.574.2%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
5.Claude Fable 5.167.4%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-deepswe
- Metric
- pass@1
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- 113 tasks across 91 repositories and five languages. Max reasoning. Meta attributes these results to Datacurve; the Muse scores could not be found on its public leaderboard at retrieval.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-141926d17f5e. Source payload hash: 141926d17f5e3bc3e9786428cbf648053ee3b4b2d5aeb1720bb45cd13413ea47. Parser version: curated-v1. Imported at: 2026-10-15
ProgramBench
Overall
Reconstruct programs from a specification. Common mini-swe-agent, clean environment without internet after setup. Share of rollouts passing at least 95% of tests, max reasoning. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Claude Opus 5.570.5%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Muse Wonder Full59%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Claude Fable 5.155.5%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
5.Muse Wonder50%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-programbench
- Metric
- rollouts passing at least 95% of tests
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- Reconstruct programs from a specification. Common mini-swe-agent, clean environment without internet after setup. Share of rollouts passing at least 95% of tests, max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Harness: mini-swe-agent. Tool policy: no internet after setup. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-942a47cb4260. Source payload hash: 942a47cb42603fecb6f8952847377ac6f1d06ee18b9997495755358980465131. Parser version: curated-v1. Imported at: 2026-10-15
Agent tasks
AutomationBench-AA (strict pass@1)
Overall
Business workflows in a simulated application environment. Every end-state assertion must pass; max reasoning. Do not merge with other AutomationBench-AA metrics. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Muse Wonder Full45.4%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Claude Opus 5.541.2%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
4.Muse Wonder39.1%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
5.Claude Fable 5.132.1%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-automationbench-aa
- Metric
- strict pass@1; every end-state assertion passed
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- Business workflows in a simulated application environment. Every end-state assertion must pass; max reasoning. Do not merge with other AutomationBench-AA metrics.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-f66314f94623. Source payload hash: f66314f94623a3c173b1379afec7d9b15f7ca83ef47e4f8114afb50a6f4c624d. Parser version: curated-v1. Imported at: 2026-10-15
OSWorld V2.1 (binary completion)
Overall
108 Ubuntu desktop workflows. Full task completion without partial credit, max reasoning. No binary Opus 5.5 score is published. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Claude Fable 5.143.1%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Muse Wonder Full39.4%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Muse Wonder34.3%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-osworld-binary
- Metric
- strict binary task completion
- Scale
- 0-100
- Participants
- 4
- Methodology and context
- 108 Ubuntu desktop workflows. Full task completion without partial credit, max reasoning. No binary Opus 5.5 score is published.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-c6588e11ac3d. Source payload hash: c6588e11ac3db4227286f2e8b5654dd03a3288c0f5a0064ebcbbf4c26b3e92ec. Parser version: curated-v1. Imported at: 2026-10-15
OSWorld V2.1 (partial reward)
Overall
108 Ubuntu desktop workflows. Partial reward, max reasoning. Meta attributes the Opus 5.5 score to Anthropic. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Claude Opus 5.581.8%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Claude Fable 5.180.7%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Muse Wonder Full78.4%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
5.Muse Wonder72.6%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-osworld-partial
- Metric
- mean per-task partial reward
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- 108 Ubuntu desktop workflows. Partial reward, max reasoning. Meta attributes the Opus 5.5 score to Anthropic.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-a3ae060e2e18. Source payload hash: a3ae060e2e18724040a1a36cf644512042bcabb3c5293186b1305ca125cb812a. Parser version: curated-v1. Imported at: 2026-10-15
Vals Finance Agent V2
Overall
Financial research using filings and market data. Private test set, max reasoning. Vals results republished by Meta. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Muse Wonder Full59.1%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Claude Fable 5.158.9%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
3.Claude Opus 5.558.6%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
4.Muse Wonder56.6%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-finance-agent-v2
- Metric
- private test accuracy
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- Financial research using filings and market data. Private test set, max reasoning. Vals results republished by Meta.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-ac916679d1f7. Source payload hash: ac916679d1f7509d34c3eb423338f90de9176538c76366a16021f7069cf491ba. Parser version: curated-v1. Imported at: 2026-10-15
Productivity
GDP.pdf
Overall
Professional work grounded in supplied PDFs. Max reasoning. Meta names Surge as the original source. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
2.Muse Wonder Full33.2 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
3.Muse Wonder30.8 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
4.Claude Opus 5.530.6 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
5.Claude Fable 5.127.6 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
higher is betterAs of October 15, 2026Scale: 0 points - 34.2 points
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-gdp-pdf
- Metric
- published GDP.pdf score
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 5
- Methodology and context
- Professional work grounded in supplied PDFs. Max reasoning. Meta names Surge as the original source.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-30ab1b5141ae. Source payload hash: 30ab1b5141ae6c43de38335484211998a2c591e1a56a2b015d0447137e1ce291. Parser version: curated-v1. Imported at: 2026-10-15
JobBench
Overall
65 professional tasks across 35 occupations. Mean task-specific reward, max reasoning. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
1.Claude Opus 5.575.4 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
2.Muse Wonder Full75.3 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
3.Claude Fable 5.173.1 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
4.Muse Wonder70.5 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
higher is betterAs of October 15, 2026Scale: 0 points - 75.4 points
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-jobbench
- Metric
- mean task reward
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 5
- Methodology and context
- 65 professional tasks across 35 occupations. Mean task-specific reward, max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-d6668b8e3bfa. Source payload hash: d6668b8e3bfa2e4bc3cdc16ba69e44f67b9b85396d40a45c97423c52a7ca4a28. Parser version: curated-v1. Imported at: 2026-10-15
Harvey LAB-AA
Overall
Legal deliverables must pass every criterion including hallucination checks. Max reasoning. Meta attributes the original evaluation to Artificial Analysis. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
1.Muse Wonder Full13.1%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
2.Muse Wonder10%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
4.Claude Fable 5.16.4%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
5.Claude Opus 5.54.2%
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
higher is betterAs of October 15, 2026Scale: 0% - 100%
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-harvey-lab-aa
- Metric
- all criteria passed with hallucination checks
- Scale
- 0-100
- Participants
- 5
- Methodology and context
- Legal deliverables must pass every criterion including hallucination checks. Max reasoning. Meta attributes the original evaluation to Artificial Analysis.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-414ee216934a. Source payload hash: 414ee216934a26a8d39821dcfe2d0697efd535462773e42ab7b5d31baecff4a3. Parser version: curated-v1. Imported at: 2026-10-15
HealthBench Pro
Overall
525 health questions without tools; physician rubrics graded by GPT-5.4. Max reasoning. Meta attributes the Claude scores to Anthropic system cards. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
1.Claude Opus 5.565.6 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
3.Muse Wonder Full62.3 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
4.Claude Fable 5.162.1 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
5.Muse Wonder60.5 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
higher is betterAs of October 15, 2026Scale: 0 points - 65.6 points
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-healthbench-pro
- Metric
- length-normalized weighted rubric score
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 5
- Methodology and context
- 525 health questions without tools; physician rubrics graded by GPT-5.4. Max reasoning. Meta attributes the Claude scores to Anthropic system cards.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-4347037f8760. Source payload hash: 4347037f87605c49e7270880ec584c6bb2a26458f112422f95989ea8f6904c75. Parser version: curated-v1. Imported at: 2026-10-15
MemoryBench
Overall
Internal Meta evaluation with 64K-100K-token conversations, distractors and conflicting or updated turns. Max reasoning. Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Note (GPT-6 Astra): Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
1.Muse Wonder85.6 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
2.Muse Wonder Full85.5 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
3.Claude Opus 5.581.5 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
5.Claude Fable 5.179 points
Note: Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
higher is betterAs of October 15, 2026Scale: 0 points - 85.6 points
Methodology and context
- Comparison cohort
- meta-wonder-launch-2026-10-12-memorybench
- Metric
- LLM-judge weighted rubric score
- Scale
- No upper bound is documented. The chart spans the recorded values and zero.
- Participants
- 5
- Methodology and context
- Internal Meta evaluation with 64K-100K-token conversations, distractors and conflicting or updated turns. Max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-0d4ffeace00d. Source payload hash: 0d4ffeace00d7c07b96781e8da9e1cb1c87f652b8cfcca9de07cec52de19f8ce. Parser version: curated-v1. Imported at: 2026-10-15
Recorded measurement details
SWE-Marathon v1.1: Overall (Show measurement, test conditions, and source)
- Source value
- 42.5
- Score
- 42.5
- Metric
- pass@1
- Unit
- %
- Benchmark version
- v1.1
- Category
- coding
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-swe-marathon-v1-1-max
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Comparator score in Meta's published table, max reasoning. The agent is not documented for the scorecard cell.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-c06f95889965. Source payload hash: c06f9588996525fb84e3530a61520976ecb3d64f429cfb2c81a440add1a1553b. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Comparator score in Meta's published table, max reasoning. The agent is not documented for the scorecard cell.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
AutomationBench-AA (strict pass@1): Overall (Show measurement, test conditions, and source)
- Source value
- 41.6
- Score
- 41.6
- Metric
- strict pass@1; every end-state assertion passed
- Unit
- %
- Benchmark version
- Meta Wonder scorecard; version unspecified
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-automationbench-aa
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Business workflows in a simulated application environment. Every end-state assertion must pass; max reasoning. Do not merge with other AutomationBench-AA metrics.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-f66314f94623. Source payload hash: f66314f94623a3c173b1379afec7d9b15f7ca83ef47e4f8114afb50a6f4c624d. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Business workflows in a simulated application environment. Every end-state assertion must pass; max reasoning. Do not merge with other AutomationBench-AA metrics.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
DeepSWE: Overall (Show measurement, test conditions, and source)
- Source value
- 74.1
- Score
- 74.1
- Metric
- pass@1
- Unit
- %
- Benchmark version
- v1.1
- Category
- coding
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-deepswe
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 113 tasks across 91 repositories and five languages. Max reasoning. Meta attributes these results to Datacurve; the Muse scores could not be found on its public leaderboard at retrieval.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-141926d17f5e. Source payload hash: 141926d17f5e3bc3e9786428cbf648053ee3b4b2d5aeb1720bb45cd13413ea47. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 113 tasks across 91 repositories and five languages. Max reasoning. Meta attributes these results to Datacurve; the Muse scores could not be found on its public leaderboard at retrieval.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Vals Finance Agent V2: Overall (Show measurement, test conditions, and source)
- Source value
- 53.5
- Score
- 53.5
- Metric
- private test accuracy
- Unit
- %
- Benchmark version
- V2
- Category
- finance
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-finance-agent-v2
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Financial research using filings and market data. Private test set, max reasoning. Vals results republished by Meta.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-ac916679d1f7. Source payload hash: ac916679d1f7509d34c3eb423338f90de9176538c76366a16021f7069cf491ba. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Financial research using filings and market data. Private test set, max reasoning. Vals results republished by Meta.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
GDP.pdf: Overall (Show measurement, test conditions, and source)
- Source value
- 34.2
- Score
- 34.2
- Metric
- published GDP.pdf score
- Unit
- points
- Benchmark version
- Meta Wonder scorecard; version unspecified
- Category
- productivity
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-gdp-pdf
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Professional work grounded in supplied PDFs. Max reasoning. Meta names Surge as the original source.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-30ab1b5141ae. Source payload hash: 30ab1b5141ae6c43de38335484211998a2c591e1a56a2b015d0447137e1ce291. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Professional work grounded in supplied PDFs. Max reasoning. Meta names Surge as the original source.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
Harvey LAB-AA: Overall (Show measurement, test conditions, and source)
- Source value
- 8.6
- Score
- 8.6
- Metric
- all criteria passed with hallucination checks
- Unit
- %
- Benchmark version
- 1.1
- Category
- legal
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-harvey-lab-aa
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Legal deliverables must pass every criterion including hallucination checks. Max reasoning. Meta attributes the original evaluation to Artificial Analysis.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-414ee216934a. Source payload hash: 414ee216934a26a8d39821dcfe2d0697efd535462773e42ab7b5d31baecff4a3. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Legal deliverables must pass every criterion including hallucination checks. Max reasoning. Meta attributes the original evaluation to Artificial Analysis.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
HealthBench Pro: Overall (Show measurement, test conditions, and source)
- Source value
- 65.3
- Score
- 65.3
- Metric
- length-normalized weighted rubric score
- Unit
- points
- Benchmark version
- Meta Wonder scorecard; version unspecified
- Category
- health
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-healthbench-pro
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 525 health questions without tools; physician rubrics graded by GPT-5.4. Max reasoning. Meta attributes the Claude scores to Anthropic system cards.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-4347037f8760. Source payload hash: 4347037f87605c49e7270880ec584c6bb2a26458f112422f95989ea8f6904c75. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 525 health questions without tools; physician rubrics graded by GPT-5.4. Max reasoning. Meta attributes the Claude scores to Anthropic system cards.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
JobBench: Overall (Show measurement, test conditions, and source)
- Source value
- 66.1
- Score
- 66.1
- Metric
- mean task reward
- Unit
- points
- Benchmark version
- Meta Wonder scorecard; version unspecified
- Category
- productivity
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-jobbench
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 65 professional tasks across 35 occupations. Mean task-specific reward, max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-d6668b8e3bfa. Source payload hash: d6668b8e3bfa2e4bc3cdc16ba69e44f67b9b85396d40a45c97423c52a7ca4a28. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 65 professional tasks across 35 occupations. Mean task-specific reward, max reasoning.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
MemoryBench: Overall (Show measurement, test conditions, and source)
- Source value
- 81.4
- Score
- 81.4
- Metric
- LLM-judge weighted rubric score
- Unit
- points
- Benchmark version
- Meta internal Wonder evaluation
- Category
- knowledge
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-memorybench
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Internal Meta evaluation with 64K-100K-token conversations, distractors and conflicting or updated turns. Max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-0d4ffeace00d. Source payload hash: 0d4ffeace00d7c07b96781e8da9e1cb1c87f652b8cfcca9de07cec52de19f8ce. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Internal Meta evaluation with 64K-100K-token conversations, distractors and conflicting or updated turns. Max reasoning.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort. The source specifies neither a denominator nor a normalized scale. This is the published raw score, not a percentage.
OSWorld V2.1 (binary completion): Overall (Show measurement, test conditions, and source)
- Source value
- 34.3
- Score
- 34.3
- Metric
- strict binary task completion
- Unit
- %
- Benchmark version
- V2.1
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-osworld-binary
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 108 Ubuntu desktop workflows. Full task completion without partial credit, max reasoning. No binary Opus 5.5 score is published.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-c6588e11ac3d. Source payload hash: c6588e11ac3db4227286f2e8b5654dd03a3288c0f5a0064ebcbbf4c26b3e92ec. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 108 Ubuntu desktop workflows. Full task completion without partial credit, max reasoning. No binary Opus 5.5 score is published.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
OSWorld V2.1 (partial reward): Overall (Show measurement, test conditions, and source)
- Source value
- 75.8
- Score
- 75.8
- Metric
- mean per-task partial reward
- Unit
- %
- Benchmark version
- V2.1
- Category
- agentic
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-osworld-partial
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- 108 Ubuntu desktop workflows. Partial reward, max reasoning. Meta attributes the Opus 5.5 score to Anthropic.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-a3ae060e2e18. Source payload hash: a3ae060e2e18724040a1a36cf644512042bcabb3c5293186b1305ca125cb812a. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- 108 Ubuntu desktop workflows. Partial reward, max reasoning. Meta attributes the Opus 5.5 score to Anthropic.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
ProgramBench: Overall (Show measurement, test conditions, and source)
- Source value
- 52.5
- Score
- 52.5
- Metric
- rollouts passing at least 95% of tests
- Unit
- %
- Benchmark version
- Meta Wonder scorecard; version unspecified
- Category
- coding
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Comparison cohort
- meta-wonder-launch-2026-10-12-programbench
- Source type
- vendor-reported
- Evaluator
- Meta
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Reconstruct programs from a specification. Common mini-swe-agent, clean environment without internet after setup. Share of rollouts passing at least 95% of tests, max reasoning.. Source model ID: curated/gpt-6-astra. Source model name: GPT-6 Astra. Provider reported by source: OpenAI. Evaluator: Meta. Dataset type: public. Harness: mini-swe-agent. Tool policy: no internet after setup. Reasoning effort: max. Source ID: meta. Evidence capture ID: curated-meta-2026-10-15-942a47cb4260. Source payload hash: 942a47cb42603fecb6f8952847377ac6f1d06ee18b9997495755358980465131. Parser version: curated-v1. Imported at: 2026-10-15
- Comparison methodology
- Reconstruct programs from a specification. Common mini-swe-agent, clean environment without internet after setup. Share of rollouts passing at least 95% of tests, max reasoning.
- Context
- Comparison table published by Meta. Depending on the benchmark, Meta uses internal runs, provider reports or third-party results and selects the highest available validated score. This is not a uniformly independently measured cohort.
Page 1 of 30
Our tests
SVG short test
One SVG prompt and fixed output settings. Captures are shown only with 0 reported reasoning tokens or documented lack of reasoning controls. Identical test conditions do not make the models' internal compute identical.
Earlier tests
Pelican on a bicycle
A community classic for free-form SVG drawing.
GPT-6 Astra
Reasoning (requested): high
Current attempt
- Run date
- Sep 7, 2026
- API provider
- openrouter
- Requested API model
- openai/gpt-6-astra
- API model reported in response
- openai/gpt-6-astra
- Reasoning setting
- high
- Pinned endpoint
- OpenAI | openai/gpt-6-astra-20260903
- Temperature
- Provider default
- Token limit including reasoning
- 8,192 tokens
- Test protocol
- community-visual-tests-v1
- Technical checks
- Passed, not a quality rating
- Response time
- 141.7 s
Visually reviewed. This individual test does not establish overall model quality.
Prompt and test conditions
Generate an SVG of a pelican riding a bicycle
Token limit including reasoning: 8,192 tokens
Task origin (Simon Willison)Popularity
76.6 / 100
Specifications and access
| Specification | Value and source |
|---|---|
| Weights | ProprietarySource |
| Access | APISource |
| Context window | 1,050,000 tokensSource |
| Knowledge cutoff | April 30, 2026Source |
| Notes | New OpenAI flagship with a 1.05 million token context window, 128,000 maximum output tokens, and an April 30, 2026 knowledge cutoff. Access has been rolling out in phases since September 3, 2026. Since September 22, 2026, GPT-6 Sol and GPT-6 Luna complete the family as lower-cost tiers.Source |
| API model ID | gpt-6-astraSource |
| Release | 2026-09-03Source |
| Maximum output | 128,000 tokensSource |
| Input | Text, ImageSource |
| Output | TextSource |
| Reasoning mode | low, medium, high, xhigh, maxSource |
| API interfaces | Responses API, Chat Completions APISource |
| Tool calling | YesSource |
| Structured output | YesSource |
Published prices
- API input
- $10 per 1M tokensSource
- API input
- $20 per 1M tokens (above 272,000 context tokens)Source
- API output
- $50 per 1M tokensSource
- API output
- $75 per 1M tokens (above 272,000 context tokens)Source
- Cache write
- $12.5 per 1M tokensSource
- Cache write
- $25 per 1M tokens (above 272,000 context tokens)Source
- Cache read
- $1 per 1M tokensSource
- Cache read
- $2 per 1M tokens (above 272,000 context tokens)Source