Language modelActive

Nemotron 3 Ultra

NVIDIA

Released: June 4, 2026
Data date: September 8, 2026

Benchmark-Meta-Score: Unavailable
Results from one source only
Speed: 162.9 tokens/s
Output throughput
Price: $0.6 · $2.4
Input · Output / 1M tokens
Input: Text
Output: Text

Nemotron 3 Ultra combines 550 billion total parameters with 55 billion active parameters in NVIDIA's LatentMoE architecture. Mamba2, MoE, attention, and multi-token prediction form the technical basis. The checkpoint supports ten languages and returns text only.

The 1M-token window covers long software and document tasks. NVIDIA's technical report describes the efficiency approach without making a blanket quality promise. For memory and parallelism, the full 550-billion-parameter footprint still matters.

Specifications and access

Specification	Value and source
Weights	Open weightsSource
Access	API, LocalSource
Parameters	550BSource
Active parameters	55BSource
Context window	1,000,000 tokensSource
Release	2026-06-04Source
Parameters	550 BSource
Active parameters	55 BSource
License	OpenMDW-1.1Source
Architecture	LatentMoE, Mamba-2 + MoE + Attention hybrid with MTPSource
Languages	English, French, Spanish, Italian, German, Japanese, Hindi, Korean, Brazilian Portuguese, ChineseSource
Input	TextSource
Output	TextSource

Benchmark meta-analysis

Overall and area scores

These Gradually indexes place selected published measurements on a 0 to 100 scale. They are not accuracy percentages and do not guarantee results in your application.

Retrieval dates of the sources used 2026-10-15 - 2026-10-15

Overall index

Not enough independent results for a score yet

An overall score needs results from at least two sources.

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Test series: 16
Test families: 16 / 65
Sources: 1
Share of the fixed test panel (informative): 21 %

Coding

Index 1.6 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.3
95% interval: 0.3 to 6.5
Test series: 6
Test families: 6 / 15
Sources: 1
Share of the fixed test panel (informative): 31 %

Reasoning

Index 3.1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.1
95% interval: 1.4 to 6.1
Test series: 1
Test families: 1 / 7
Sources: 1
Share of the fixed test panel (informative): 7 %

Knowledge

Index 3.2 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.2
95% interval: 1.5 to 6.3
Test series: 1
Test families: 1 / 3
Sources: 1
Share of the fixed test panel (informative): 33 %

Finance

Index 3.1 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 2.0
95% interval: 0.8 to 9.6
Test series: 4
Test families: 4 / 7
Sources: 1
Share of the fixed test panel (informative): 57 %

Legal

Index 1.3 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 1.3
95% interval: 0.1 to 7.1
Test series: 3
Test families: 3 / 4
Sources: 1
Share of the fixed test panel (informative): 75 %

Health

Index 1.5 / 100

provisional

Published configurations differ within this comparison cohort. Treat the index as provisional.

Standard error: ± 0.6
95% interval: 0.6 to 3.2
Test series: 1
Test families: 1 / 3
Sources: 1
Share of the fixed test panel (informative): 33 %

Documented method · Included test families and measurements

Method version: benchmark-meta-score-v3

The index includes active measurements from reviewed primary sources when at least five models are compared in the same documented comparison slice across dataset, harness, agent, tools, and trial count. Each benchmark series and evaluator contributes one shared setup slice from the newest capture. When one source and series include several explicitly documented reasoning tiers, the highest documented tier (none through max) is selected, never the best result value. The explicit source label “Non-reasoning” is treated as none. Unknown or ambiguous tiers exclude the model from that series. Within each series, percentages and rates are logit-transformed, other units stay unchanged, and all values are then standardized. A one-factor model then estimates one shared strength per model and, per series, how sharply it separates the models. Area scores move away from that strength only as far as the model's own results in the area support. The overall score weights the core areas at 75% and finance, law and health at 25%. It requires at least eight test series from two sources and three areas. The 0 to 100 scale refers to a fixed reference group of 29 language models whose average sits at 50.

Published model-specific parameters can differ. This is not a controlled experiment; comparisons with such differences are marked provisional.

Unmeasured series are not entered as zero. Where a model has no own results in a core area, its overall strength stands in, and the note “estimated without own results in …” names the area. Fewer test series lead to a larger standard error.

The 95% interval follows from the standard error of 200 recalculations with randomly redrawn test families. It describes how precisely the recorded test series determine the value. It is no prediction for future tasks or for later recalculations with other participants.

The source count is not a count of independent experiments.

OECD handbook on composite indicators

All 16 test families and 12 of 16 raw measurements are listed here. The complete calculation for every model is available in the JSON export.

Test family: GPQA Diamond
Reasoning · Mean rank percentile of the family: 57.9 / 100
Family weight in the fixed test panel (informative): 1.53 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-gpqa
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.77 %
  Raw value:
  86.112 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  77
  Rank percentile within the cohort:
  57.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-gpqa:1:overall
  Documented comparison setup
  Publication cohort:
  vals-gpqa:1:overall
  Dataset type:
  public
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 158.471. Standard error: 1.738. Source ID: vals-ai. Evidence capture ID: vals-gpqa-2026-10-15-951eb5224097. Source payload hash: 951eb5224097733a9d09d06c14908ae2980e2e640e539f2c284877c5092d7077. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: SWE-bench Verified
Coding · Mean rank percentile of the family: 19.5 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 50 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-swebench
  Series weight within the test family:
  50 %
  Nominal contribution to the overall panel:
  0.36 %
  Raw value:
  69 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  60
  Rank percentile within the cohort:
  19.5 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-swebench:1:overall
  Documented comparison setup
  Publication cohort:
  vals-swebench:1:overall
  Dataset type:
  public
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 283.089. Standard error: 2.07. Source ID: vals-ai. Evidence capture ID: vals-swebench-2026-10-15-7a4e2e848528. Source payload hash: 7a4e2e8485283caac500127419f68ec4bd5414c837ade114fc6afa5b2d34ccaa. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Terminal-Bench
Coding · Mean rank percentile of the family: 23.2 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 17 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-terminal-bench-2-1
  Series weight within the test family:
  16.67 %
  Nominal contribution to the overall panel:
  0.12 %
  Raw value:
  50.936 %
  Metric:
  accuracy
  Version:
  2.1
  Participating models:
  57
  Rank percentile within the cohort:
  23.2 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-terminal-bench-2-1:2.1:overall
  Documented comparison setup
  Publication cohort:
  vals-terminal-bench-2-1:2.1:overall
  Dataset type:
  public
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-28. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 584.116. Standard error: 2.925. Source ID: vals-ai. Evidence capture ID: vals-terminal-bench-2-1-2026-10-15-a5306805cdf8. Source payload hash: a5306805cdf86495bdcceff6b2c8534a3768b9853f5077895dab975b809b054f. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Code Migration
Coding · Mean rank percentile of the family: 8.1 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-code-migration
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.71 %
  Raw value:
  4.905 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  63
  Rank percentile within the cohort:
  8.1 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-code-migration:1:overall
  Documented comparison setup
  Publication cohort:
  vals-code-migration:1:overall
  Dataset type:
  private
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 5470.513. Standard error: 1.589. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: CorpFin v2
Finance · Mean rank percentile of the family: 75.7 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-corp-fin-v2
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  65.462 %
  Metric:
  accuracy
  Version:
  2
  Participating models:
  75
  Rank percentile within the cohort:
  75.7 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-corp-fin-v2:2:overall
  Documented comparison setup
  Publication cohort:
  vals-corp-fin-v2:2:overall
  Dataset type:
  private
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 15.015. Standard error: 0.937. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: EMB
Finance · Mean rank percentile of the family: 16.9 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-emb
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  31.735 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  60
  Rank percentile within the cohort:
  16.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-emb:1:overall
  Documented comparison setup
  Publication cohort:
  vals-emb:1:overall
  Dataset type:
  private
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 2382.355. Standard error: 2.805. Source ID: vals-ai. Evidence capture ID: vals-emb-2026-10-15-e1170aaade72. Source payload hash: e1170aaade72d6b81079f9cb90244c41f028a12e29d982c3ec1d6e9913a9872d. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Finance Agent (v2)
Finance · Mean rank percentile of the family: 15.0 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-finance-agent-v2
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  1.19 %
  Raw value:
  37.667 %
  Metric:
  accuracy
  Version:
  2
  Participating models:
  61
  Rank percentile within the cohort:
  15.0 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-finance-agent-v2:2:overall
  Documented comparison setup
  Publication cohort:
  vals-finance-agent-v2:2:overall
  Dataset type:
  private
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 253.882. Standard error: 0.149. Source ID: vals-ai. Evidence capture ID: vals-fabv2-2026-10-15-3780be1a23c9. Source payload hash: 3780be1a23c9133a3c34a6095024af162b35a8ef50963a62fc67a6049880426c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: LiveCodeBench v6
Coding · Mean rank percentile of the family: 76.6 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-lcb
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  0.71 %
  Raw value:
  85.976 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  78
  Rank percentile within the cohort:
  76.6 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-lcb:1:overall
  Documented comparison setup
  Publication cohort:
  vals-lcb:1:overall
  Dataset type:
  public
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-09-01. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 161.2. Standard error: 0.992. Source ID: vals-ai. Evidence capture ID: vals-lcb-2026-10-15-1a7c7dabc9d9. Source payload hash: 1a7c7dabc9d95a0182886b1edc42623987cac2095a0d407a5e334f37acdfa2f5. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Harvey's Legal Agent Benchmark
Legal · Mean rank percentile of the family: 24.6 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-legal-agent-benchmark
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  0.417 %
  Metric:
  task resolution rate
  Version:
  1
  Participating models:
  62
  Rank percentile within the cohort:
  24.6 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-legal-agent-benchmark:1:overall
  Documented comparison setup
  Publication cohort:
  vals-legal-agent-benchmark:1:overall
  Dataset type:
  public
  Setup and model parameters
  Temperature: 1. Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-12. Temperature: 1. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 2048.539. Standard error: 0. Source ID: vals-ai. Evidence capture ID: vals-hlab-2026-10-15-7587eca48586. Source payload hash: 7587eca48586be017c9db8c1a11e6f6797b1eb62b7627cbf198f35f592f4fc69. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: LegalBench
Legal · Mean rank percentile of the family: 41.0 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-legal-bench
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  82.066 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  79
  Rank percentile within the cohort:
  41.0 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-legal-bench:1:overall
  Documented comparison setup
  Publication cohort:
  vals-legal-bench:1:overall
  Dataset type:
  public
  Setup and model parameters
  Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: public. Leaderboard date: 2026-10-13. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 5.179. Standard error: 0.46. Source ID: vals-ai. Evidence capture ID: vals-legal_bench-2026-10-15-c8486ba756ba. Source payload hash: c8486ba756baa5cd3955d035edff67440e8db23e69d62ad1a9e18d15b8c4bc52. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: Legal Research Bench
Legal · Mean rank percentile of the family: 19.2 / 100
Family weight in the fixed test panel (informative): 2.08 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-legal-research
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.08 %
  Raw value:
  15.385 %
  Metric:
  all-pass rate
  Version:
  1
  Participating models:
  61
  Rank percentile within the cohort:
  19.2 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-legal-research:1:overall
  Documented comparison setup
  Publication cohort:
  vals-legal-research:1:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 1. Top P: 0.95. Maximum output tokens: 128000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Temperature: 1. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 611.366. Standard error: 2.508. Source ID: vals-ai. Evidence capture ID: vals-legal_research-2026-10-15-f16d6b6678c0. Source payload hash: f16d6b6678c06111a5c628248d14f0eb5c918491d50288c7f649729fcd5dcfaf. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MedCode
Health · Mean rank percentile of the family: 23.9 / 100
Family weight in the fixed test panel (informative): 2.78 % · Coverage of this test family: 100 %
- Vals AI
  Published configurations differ within this cohort.
  Measurement:
  vals-medcode
  Series weight within the test family:
  100 %
  Nominal contribution to the overall panel:
  2.78 %
  Raw value:
  38.621 %
  Metric:
  accuracy
  Version:
  1
  Participating models:
  68
  Rank percentile within the cohort:
  23.9 %
  Retrieved:
  2026-10-15
  Comparison cohort:
  vals-medcode:1:overall
  Documented comparison setup
  Publication cohort:
  vals-medcode:1:overall
  Dataset type:
  private
  Setup and model parameters
  Temperature: 1. Top P: 0.95. Maximum output tokens: 30000
  Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-13. Temperature: 1. Top P: 0.95. Maximum output tokens: 30000. Latency in seconds: 18.675. Standard error: 2.001. Source ID: vals-ai. Evidence capture ID: vals-medcode-2026-10-15-988a6ae1e07e. Source payload hash: 988a6ae1e07e33ce031572ed4a804be11c36ee97a1dd75cbe4740fff3b534f8c. Parser version: vals-astro-v3. Imported at: 2026-10-15
Test family: MMLU-Pro
Knowledge · Mean rank percentile of the family: 50.6 / 100
Family weight in the fixed test panel (informative): 3.57 % · Coverage of this test family: 100 %
Test family: ProgramBench
Coding · Mean rank percentile of the family: 24.5 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %
Test family: TaxEval v2
Finance · Mean rank percentile of the family: 53.9 / 100
Family weight in the fixed test panel (informative): 1.19 % · Coverage of this test family: 100 %
Test family: Vibe Code Bench v1.1
Coding · Mean rank percentile of the family: 12.5 / 100
Family weight in the fixed test panel (informative): 0.71 % · Coverage of this test family: 100 %

Benchmark sources at a glance

Each evaluator measures different capabilities under its own conditions. Test series, composite indices and their components stay separate. The detailed results below retain their scales, configurations and source dates.

Evaluator	Test series	Composite indices	Recorded active measurements
Artificial Analysis	2	0	9
Vals AI	16	0	143
Epoch AI	2	0	2

Coverage is not a quality ranking. Overlapping tests and multiple settings do not count as independent experiments. Vendor claims, historical and disputed results are excluded from this overview and remain labelled in the detailed evidence.

7 measurements from these sources still lack a verified test-family identity and do not increase the test-series count.

Published benchmarks

Coding

Code Migration

Sub-task	Nemotron 3 Ultra (This model)	Kimi K2.6	Kimi K2.5
CLI	4.64%	17.98% (best value in this row)	8.03%
COBOL	5.71%	57.14% (best value in this row)	3.77%
Overall	4.91%	27.77% (best value in this row)	6.96%

higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

Code Migration

Code Quality

1.Kimi K2.53.27 rating

2.Nemotron 3 Ultra3.25 rating

3.Kimi K2.63.15 rating

higher is betterAs of October 15, 2026Scale: 0 rating - 3.27 rating

Source: Vals AI

|

gradually.ai

Methodology and context

Comparison cohort: vals-code-migration:1:code_quality
Metric: mean reviewer code-quality rating
Scale: No upper bound is documented. The chart spans the recorded values and zero.
Participants: 3
Methodology and context: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 5470.513. Standard error: 0.045. Score display: 3.245 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15

Source: Vals AI

Finance

CorpFin v2

Sub-task	Nemotron 3 Ultra (This model)	Kimi K2.5
Exact Pages	64.34%	66.2% (best value in this row)
Max Fitting Context	67.13%	68.42% (best value in this row)
Overall	65.46%	68.26% (best value in this row)
Shared Max Context	64.92%	70.16% (best value in this row)

higher is betterAs of October 15, 2026Source: Vals AIbars scaled within each row, not across rows

Methodology and context

Metric: accuracy

Recorded measurement details

Code Migration: CLI (Show measurement, test conditions, and source)

Source value: 4.636
Score: 4.636
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: cli
Task label: CLI
Direction: Higher is better
Comparison cohort: vals-code-migration:1:cli
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 6272.162. Standard error: 0.929. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

Code Migration: COBOL (Show measurement, test conditions, and source)

Source value: 5.714
Score: 5.714
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: cobol
Task label: COBOL
Direction: Higher is better
Comparison cohort: vals-code-migration:1:cobol
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 3065.567. Standard error: 5.714. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

Code Migration: Overall (Show measurement, test conditions, and source)

Source value: 4.905
Score: 4.905
Metric: accuracy
Unit: %
Benchmark version: 1
Category: coding
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-code-migration:1:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 5470.513. Standard error: 1.589. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

Code Migration: Code Quality (Show measurement, test conditions, and source)

Source value: 3.245
Score: 3.245
Metric: mean reviewer code-quality rating
Unit: rating
Benchmark version: 1
Category: coding
Task: code_quality
Task label: Code Quality
Direction: Higher is better
Comparison cohort: vals-code-migration:1:code_quality
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-10-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 5470.513. Standard error: 0.045. Score display: 3.245 / 5. Source ID: vals-ai. Evidence capture ID: vals-code-migration-2026-10-15-3fa9c10568ec. Source payload hash: 3fa9c10568ecc117fabec81767c405b49a25d5d2ded308ee435fffb197c53474. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

CorpFin v2: Exact Pages (Show measurement, test conditions, and source)

Source value: 64.336
Score: 64.336
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: exact_pages
Task label: Exact Pages
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:exact_pages
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 6.738. Standard error: 1.635. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

CorpFin v2: Max Fitting Context (Show measurement, test conditions, and source)

Source value: 67.133
Score: 67.133
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: max_fitting_context
Task label: Max Fitting Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:max_fitting_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 22.607. Standard error: 1.604. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

CorpFin v2: Overall (Show measurement, test conditions, and source)

Source value: 65.462
Score: 65.462
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: overall
Task label: Overall
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:overall
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 15.015. Standard error: 0.937. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

CorpFin v2: Shared Max Context (Show measurement, test conditions, and source)

Source value: 64.918
Score: 64.918
Metric: accuracy
Unit: %
Benchmark version: 2
Category: finance
Task: shared_max_context
Task label: Shared Max Context
Direction: Higher is better
Comparison cohort: vals-corp-fin-v2:2:shared_max_context
Source type: independent-evaluator
Evaluator: Vals AI
Status: active
Retrieved at: 2026-10-15
Methodology: Source model ID: nvidia/nemotron-3-ultra-550b-a55b. Source model name: Nemotron 3 Ultra. Provider reported by source: Nvidia. Evaluator: Vals AI. Dataset type: private. Leaderboard date: 2026-08-12. Top P: 0.95. Maximum output tokens: 128000. Latency in seconds: 15.701. Standard error: 1.629. Source ID: vals-ai. Evidence capture ID: vals-corp_fin_v2-2026-10-15-7cf2217ab372. Source payload hash: 7cf2217ab3728d5a84bffce82aab9fc52d7f810b0cbe8c65b387df47f3d31c9b. Parser version: vals-astro-v3. Imported at: 2026-10-15
Comparison methodology: Top P: 0.95. Maximum output tokens: 128000

Page 1 of 12

Sources and data date

Type	Evidence and data date
Speed	Median across serving providers. Output speed after the first streaming chunk, not total response time. Retrieved September 8, 2026. Source: Artificial Analysis · Artificial Analysis (retrieved September 8, 2026)Speed measurement methodology
Price	Base API prices without caching or batch discounts. Higher context tiers and other rates are listed under Costs. Source: Pricing · API pricing
Input	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · NVIDIA Nemotron 3 Ultra model card · Editorial description reviewed October 2, 2026
Output	Documented modalities. Retrieved September 8, 2026. Source: Model documentation · Model metadata (retrieved September 8, 2026) · NVIDIA Nemotron 3 Ultra model card · Editorial description reviewed October 2, 2026
Research status	Research date September 8, 2026. 3 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	knowledge-cutoff Original wording No exact value located in the checked primary model card/API documentation.
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Vals AI (retrieved October 15, 2026)
Additional source	Epoch AI benchmark run dataset snapshot (retrieved October 19, 2026)
Additional source	Nemotron 3 Ultra technical report · Editorial description reviewed October 2, 2026
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0