- Source value
- 0.28571428571428603
- Score
- 0.28571428571428603
- Metric
- pass@1 accuracy
- Unit
- %
- Benchmark version
- v2
- Category
- reasoning
- Task
- overall
- Task label
- Overall
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- Artificial Analysis
- Status
- disputed
- Retrieved at
- 2026-11-02
- Methodology
- Exact AA configuration: North Mini Code (release north-mini-code, row north-mini-code, UUID 1e9907e0-ffac-4595-b006-962e4f1da7cf, reasoning=true, effort=not specified, suffix=not specified). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap.
The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Source model ID: artificial-analysis/1e9907e0-ffac-4595-b006-962e4f1da7cf/north-mini-code/north-mini-code. Source model name: North Mini Code. Provider reported by source: Cohere. Model snapshot: North Mini Code. Evaluator: Artificial Analysis. Dataset type: artificial-analysis-evaluation. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled. Score display: 0.3%. Source ID: artificial-analysis. Evidence capture ID: aa-model-page-2026-11-02-67d46be20a5b. Source payload hash: 67d46be20a5b412db6cfb60ad190441fcaa5513500b8fd47d033168d0e97b6b2. Parser version: artificial-analysis-rsc-models-v3. Imported at: 2026-11-02. Source cohort: artificial-analysis-critpt-v2:v2:standalone:reasoning:effort=unspecified:suffix=unspecified
- Comparison methodology
- Exact AA configuration: North Mini Code (release north-mini-code, row north-mini-code, UUID 1e9907e0-ffac-4595-b006-962e4f1da7cf, reasoning=true, effort=not specified, suffix=not specified). AA source value for average reasoning tokens before the answer: 2000. Its methodology averages 60 varied prompts and assumes 2,000 when no average is available. This is not a token cap.
The full CritPt v2 run covers 70 questions with five repeats. The official grading server reports pass@1.. Tool policy: Python functions, symbolic expressions, and numerical answers; official grading server. Attempts: 70 questions, five repeats per question, pass@1. Reasoning: reasoning enabled
- Context
- This AA source field contains the original CritPt score. The earlier CritPt v2 attribution was incorrect; the value and former label remain as disputed historical evidence.. The source dataset marks this result as disputed. It remains visible as documented evidence, but is not reliable comparative proof.