Code MigrationOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 17.313 % | Vals AISource detailsData as of: 09/27/2026 |
CorpFin v2OverallTest detailsNo documented difference in the test setupVersion: 2Metric: accuracyScoring: Higher is better | | 64.491 % | Vals AISource detailsData as of: 08/12/2026 |
EMBOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 52.616 % | Vals AISource detailsData as of: 09/27/2026 |
Finance Agent (v2)OverallTest detailsNo documented difference in the test setupVersion: 2Metric: accuracyScoring: Higher is better | | 42.982 % | Vals AISource detailsData as of: 09/27/2026 |
GAIA ReliabilityAccuracyTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: exact-match accuracyScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | 73.3 % | | Princeton HALSource detailsData as of: 08/06/2026 |
GPQA DiamondOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 90.152 % | | Vals AISource detailsData as of: 09/01/2026 |
Harvey's Legal Agent BenchmarkOverall · Task fully resolvedTest detailsNo documented difference in the test setupVersion: 1Metric: task resolution rateScoring: Higher is better | | 0 % | Vals AISource detailsData as of: 09/27/2026 |
Humanity's Last ExamOverallTest detailsNo documented difference in the test setupVersion: final 2,500-question public setMetric: accuracyScoring: Higher is betterClaude Opus 4.7: Attempts: pass@1 Gemini 3.1 Pro Preview: Snapshot: gemini-3.1-pro-preview-thinking-high, Attempts: pass@1All 2,500 public questions, temperature 0, and an automatic answer extractor. | 36.20% ± 1.88 95% confidence interval: ±1.88 | 95% confidence interval: ±1.96 | Scale AISource detailsData as of: 09/02/2026 |
Legal Research BenchOverall · All-passTest detailsNo documented difference in the test setupVersion: 1Metric: all-pass rateScoring: Higher is better | | 20.673 % | Vals AISource detailsData as of: 09/27/2026 |
LegalBenchOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 85.251 % | | Vals AISource detailsData as of: 09/27/2026 |
LiveCodeBench v6OverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 85.073 % | | Vals AISource detailsData as of: 09/01/2026 |
MedCodeOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 54.858 % | | Vals AISource detailsData as of: 09/26/2026 |
MedScribeOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 76.114 % | Vals AISource detailsData as of: 09/26/2026 |
MMLU-ProOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 89.871 % | | Vals AISource detailsData as of: 09/01/2026 |
MMMU-ProOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | 85.549 % | | Vals AISource detailsData as of: 09/01/2026 |
MortgageTaxOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 69.396 % | Vals AISource detailsData as of: 09/01/2026 |
ProgramBenchOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | | Vals AISource detailsData as of: 09/27/2026 |
SAGEOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 48.677 % | Vals AISource detailsData as of: 09/26/2026 |
SWE-bench VerifiedOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 78.8 % | Vals AISource detailsData as of: 09/01/2026 |
Tax Agent BenchOverallTest detailsNo documented difference in the test setupVersion: 1Metric: accuracyScoring: Higher is better | | 37.842 % | Vals AISource detailsData as of: 09/27/2026 |
TaxEval v2OverallTest detailsNo documented difference in the test setupVersion: 2Metric: accuracyScoring: Higher is better | | 72.882 % | Vals AISource detailsData as of: 09/01/2026 |
Terminal-Bench 2.0OverallTest detailsNo documented difference in the test setupVersion: 2.0Metric: accuracyScoring: Higher is better | | 67.416 % | Vals AISource detailsData as of: 06/04/2026 |
Terminal-Bench 2.1OverallTest detailsNo documented difference in the test setupVersion: 2.1Metric: accuracyScoring: Higher is better | 68.539 % | | Vals AISource detailsData as of: 09/27/2026 |
Vals IndexOverallTest detailsNo documented difference in the test setupVersion: 2Metric: weighted index scoreScoring: Higher is better | | 41.903 % | Vals AISource detailsData as of: 09/27/2026 |
Vals Multimodal IndexOverallTest detailsNo documented difference in the test setupVersion: 1.2Metric: weighted index scoreScoring: Higher is better | | 56.068 % | Vals AISource detailsData as of: 08/11/2026 |
Vibe Code Bench v1.1OverallTest detailsNo documented difference in the test setupVersion: 1.1Metric: accuracyScoring: Higher is betterSettings: Harness: OpenHands | | 32.034 % | Vals AISource detailsData as of: 09/27/2026 |
τ-bench Airline CleanAccuracyTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: task success rateScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | | 82.1 % | Princeton HALSource detailsData as of: 08/05/2026 |
GAIA ReliabilityConsistencyTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: aggregate consistencyScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | | 0.76 rating points | Princeton HALSource detailsData as of: 08/06/2026 |
τ-bench Airline CleanConsistencyTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: aggregate consistencyScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | | 0.81 rating points | Princeton HALSource detailsData as of: 08/05/2026 |
GAIA ReliabilityPredictabilityTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: aggregate predictabilityScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | 0.73 rating points | | Princeton HALSource detailsData as of: 08/06/2026 |
τ-bench Airline CleanPredictabilityTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: aggregate predictabilityScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | | 0.83 rating points | Princeton HALSource detailsData as of: 08/05/2026 |
GAIA ReliabilityReliabilityTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: aggregate reliabilityScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | | 0.82 rating points | Princeton HALSource detailsData as of: 08/06/2026 |
τ-bench Airline CleanReliabilityTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: aggregate reliabilityScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | | 0.86 rating points | Princeton HALSource detailsData as of: 08/05/2026 |
GAIA ReliabilityRobustnessTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: aggregate robustnessScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | | 0.94 rating points | Princeton HALSource detailsData as of: 08/06/2026 |
τ-bench Airline CleanRobustnessTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: aggregate robustnessScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | 0.91 rating points | | Princeton HALSource detailsData as of: 08/05/2026 |
GAIA ReliabilitySafetyTest detailsNo documented difference in the test setupVersion: final leak-cleaned public validation setMetric: aggregate safetyScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability165 GAIA tasks with repeated runs and perturbations. HAL does not publish cost for this evaluation. | | | Princeton HALSource detailsData as of: 08/06/2026 |
τ-bench Airline CleanSafetyTest detailsNo documented difference in the test setupVersion: 26-task cleaned subsetMetric: aggregate safetyScoring: Higher is betterSettings: Agent: HAL Reliability, Harness: HAL Reliability26 cleaned airline tasks. Princeton HAL recomputed every metric from scratch on this subset. | 0.94 rating points | | Princeton HALSource detailsData as of: 08/05/2026 |