- Source value
- 4.57142857142857
- Score
- 4.57142857142857
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- v2
- Category
- reasoning
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- disputed
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: Hy3-preview (Reasoning) (release hy3-preview, row hy3-preview, UUID 8ca48626-ff5e-48b3-8401-38081376d706, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap.
The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/8ca48626-ff5e-48b3-8401-38081376d706/hy3-preview/hy3-preview. Source model name: Hy3-preview. Provider reported by source: Tencent. Model snapshot: Hy3-preview (Reasoning). Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled. Score display: 4.6%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:reasoning:effort=unspecified:suffix=Reasoning
- Comparison methodology
- Exact AA configuration: Hy3-preview (Reasoning) (release hy3-preview, row hy3-preview, UUID 8ca48626-ff5e-48b3-8401-38081376d706, reasoning=true, effort=not specified, suffix=Reasoning). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap.
The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled
- Context
- This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.