BigCodeLanguage model
StarCoder2 15B
StarCoder2 15B has no overall index value yet because there are too few independent benchmark results.
35out of 100same as last month
based on search, usage and attention
Summary
StarCoder2 15B is BigCode’s code-completion model, not a general chat assistant. It was trained on more than 600 programming languages from The Stack v2 and more than 4 trillion tokens overall. With a 16,384-token context, 4,096-token sliding attention, and fill-in-the-middle support, it fits code completion, gap filling, and editor work across files.
The model card explicitly says direct instructions do not work well because the checkpoint is not instruction tuned. StarCoder2 15B is therefore built for completion and fill-in-the-middle rather than chat. BigCode also warns that generated code can reproduce material from the training data, and the OpenRAIL-M license is part of the published model identity.
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 100 published benchmark results
Page 1 of 9
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
CRUXEval-I pass@1 (Show measurement, test conditions, and source)
- Source value
- 48.1
- Score
- 48.1
- Metric
- pass@1
- Unit
- %
- Benchmark version
- CRUXEval Table 15 (StarCoder2 report, 2024)
- Direction
- Higher is better
- Sample
- 800
- Source type
- vendor-reported
- Evaluator
- BigCode StarCoder2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact StarCoder2-15B row in CRUXEval-I/O Table 15. Benchmark contains 800 examples; pass@1 uses temperature 0.2 and pass@5 uses 0.8; both use 10 generated samples. Report benchmark snapshot is the CRUXEval release used in the 2024 report.; Temperature 0.2; 10 generations; 800-example task.
- Context
- Source reports the pass rate on a 0-100 score scale.
CRUXEval-I pass@5 (Show measurement, test conditions, and source)
- Source value
- 66.9
- Score
- 66.9
- Metric
- pass@5
- Unit
- %
- Benchmark version
- CRUXEval Table 15 (StarCoder2 report, 2024)
- Direction
- Higher is better
- Sample
- 800
- Source type
- vendor-reported
- Evaluator
- BigCode StarCoder2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact StarCoder2-15B row in CRUXEval-I/O Table 15. Benchmark contains 800 examples; pass@1 uses temperature 0.2 and pass@5 uses 0.8; both use 10 generated samples. Report benchmark snapshot is the CRUXEval release used in the 2024 report.; Temperature 0.8; 10 generations; 800-example task.
- Context
- Source reports the pass rate on a 0-100 score scale.
CRUXEval-O pass@1 (Show measurement, test conditions, and source)
- Source value
- 47.1
- Score
- 47.1
- Metric
- pass@1
- Unit
- %
- Benchmark version
- CRUXEval Table 15 (StarCoder2 report, 2024)
- Direction
- Higher is better
- Sample
- 800
- Source type
- vendor-reported
- Evaluator
- BigCode StarCoder2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact StarCoder2-15B row in CRUXEval-I/O Table 15. Benchmark contains 800 examples; pass@1 uses temperature 0.2 and pass@5 uses 0.8; both use 10 generated samples. Report benchmark snapshot is the CRUXEval release used in the 2024 report.; Temperature 0.2; 10 generations; 800-example task.
- Context
- Source reports the pass rate on a 0-100 score scale.
CRUXEval-O pass@5 (Show measurement, test conditions, and source)
- Source value
- 59.5
- Score
- 59.5
- Metric
- pass@5
- Unit
- %
- Benchmark version
- CRUXEval Table 15 (StarCoder2 report, 2024)
- Direction
- Higher is better
- Sample
- 800
- Source type
- vendor-reported
- Evaluator
- BigCode StarCoder2 technical report
- Status
- active
- Retrieved at
- 2026-10-15
- Methodology
- Vendor-reported result. Exact StarCoder2-15B row in CRUXEval-I/O Table 15. Benchmark contains 800 examples; pass@1 uses temperature 0.2 and pass@5 uses 0.8; both use 10 generated samples. Report benchmark snapshot is the CRUXEval release used in the 2024 report.; Temperature 0.8; 10 generations; 800-example task.
- Context
- Source reports the pass rate on a 0-100 score scale.
HumanEval (Show measurement, test conditions, and source)
- Source value
- 46.3
- Score
- 46.3
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 9
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 9 exact StarCoder2-15B row; pass@1 with greedy decoding.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
HumanEval+ (Show measurement, test conditions, and source)
- Source value
- 37.8
- Score
- 37.8
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 9
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 9 exact StarCoder2-15B row; pass@1 with greedy decoding.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MBPP (Show measurement, test conditions, and source)
- Source value
- 66.2
- Score
- 66.2
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 9
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 9 exact StarCoder2-15B row; pass@1 with greedy decoding.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MBPP+ (Show measurement, test conditions, and source)
- Source value
- 53.1
- Score
- 53.1
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 9
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 9 exact StarCoder2-15B row; pass@1 with greedy decoding.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MultiPL-E C++ pass@1 (Show measurement, test conditions, and source)
- Source value
- 41.4
- Score
- 41.4
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 10
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 10 exact StarCoder2-15B row, C++; temperature 0.2, top-p 0.95, 50 samples per problem.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MultiPL-E C# pass@1 (Show measurement, test conditions, and source)
- Source value
- 29.2
- Score
- 29.2
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 10
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 10 exact StarCoder2-15B row, C#; temperature 0.2, top-p 0.95, 50 samples per problem.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MultiPL-E D pass@1 (Show measurement, test conditions, and source)
- Source value
- 23.6
- Score
- 23.6
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 10
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 10 exact StarCoder2-15B row, D; temperature 0.2, top-p 0.95, 50 samples per problem.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
MultiPL-E Go pass@1 (Show measurement, test conditions, and source)
- Source value
- 26.2
- Score
- 26.2
- Metric
- pass@1 (%)
- Unit
- %
- Benchmark version
- StarCoder 2 technical report, Table 10
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- StarCoder 2 and The Stack v2: The Next Generation technical report
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Table 10 exact StarCoder2-15B row, Go; temperature 0.2, top-p 0.95, 50 samples per problem.
- Context
- Vendor-reported technical-report result. The named StarCoder2-15B row is not bound to an immutable evaluation checkpoint. No per-task sample count or confidence interval is added unless the cited table reports it.
Page 1 of 9
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 101 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 7, 2026
StarCoder2 15BHermes 4 405B 35Kimi K2 35Kimi K2 0711 35Amazon Nova Micro 35
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as StarCoder2 15B, chosen by model line, developer, size, price and release date.
- Phi-3 Medium (14B)Microsoft · similar size
- Phi-4Microsoft · similar size
- InternLM2-Chat-20BShanghai AI Lab · similar size
- SOLAR 10.7B Instruct v1.0Upstage · similar size
- SOLAR 10.7B v1.0Upstage · similar size
- Pixtral 12BMistral AI · similar size
- Qwen 3 14BAlibaba · similar size
- OLMo 2 13BAllen Institute · similar size
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
Frequently asked questions about StarCoder2 15B
How large is the context window of StarCoder2 15B?
16,384 tokens. That is how much text StarCoder2 15B reads at once, about 20 A4 pages.
Page count as a rule of thumb from Artificial Analysis
Can I run StarCoder2 15B locally?
Yes, the weights are open.
Stated by BigCode
- Open weights
- yes
- License
- BigCode Open RAIL-M v1
- Parameters
- 15B
- active per token
- n/a
Sources and data date
Show all 22 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.
Without vendor claims and historical or disputed results; those stay labelled in the single values.