Specialist modelOpen weights
AstaBrief-8B
Allen Institute for AI
- Released
- October 2, 2026
- Data date
- October 7, 2026
AstaBrief-8B turns a research question and already retrieved scientific excerpts into a report with citations. It handles the writing and attribution, not the preceding literature search.
Ai2 released the checkpoint for Asta’s fast report mode. It is based on Qwen3-8B, intended for research and education, and licensed under Apache-2.0.
Specifications and access
| Specification | Value and source |
|---|---|
| Model class | Specialist model for cited reports grounded in scientific literatureSource |
| Input modalities | Text: research question and already retrieved scientific literature excerptsSource |
| Output modalities | Text report with citationsSource |
| Base model | Qwen3-8BSource |
| License | Apache-2.0; Ai2 states research and educational useSource |
Page 1 of 2
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
Win rate vs Asta SQA, SQA-CS2 Dev (Show measurement, test conditions, and source)
- Source value
- 55
- Score
- 55
- Metric
- pairwise win rate
- Unit
- %
- Benchmark version
- SQA-CS2 Dev
- Category
- pairwise report comparison
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. Win-rate column in the AstaBrief-8B comparison table against Asta SQA.
- Context
- The model card does not publish the number of compared report pairs or tie handling for this table.
Win rate vs Asta SQA, SQA-CS2 Test (Show measurement, test conditions, and source)
- Source value
- 72
- Score
- 72
- Metric
- pairwise win rate
- Unit
- %
- Benchmark version
- SQA-CS2 Test
- Category
- pairwise report comparison
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. Win-rate column in the AstaBrief-8B comparison table against Asta SQA.
- Context
- The model card does not publish the number of compared report pairs or tie handling for this table.
ScholarQA-CS2 test set, Average (Show measurement, test conditions, and source)
- Source value
- 87
- Score
- 87
- Metric
- Average
- Unit
- No unit provided
- Benchmark version
- ScholarQA-CS2 test
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 100
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B DPO checkpoint; ScholarQA-CS2 test set of 100 user-written computer science research questions; questions are paired with already-retrieved literature excerpts, so this measures report generation and citation grounding, not retrieval.
- Context
- The model card prints scalar scores without a unit or explicit percent sign; values are preserved without conversion.
ScholarQA-CS2 test set, Ingredient Recall (Show measurement, test conditions, and source)
- Source value
- 90.2
- Score
- 90.2
- Metric
- Ingredient Recall
- Unit
- No unit provided
- Benchmark version
- ScholarQA-CS2 test
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 100
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B DPO checkpoint; ScholarQA-CS2 test set of 100 user-written computer science research questions; questions are paired with already-retrieved literature excerpts, so this measures report generation and citation grounding, not retrieval.
- Context
- The model card prints scalar scores without a unit or explicit percent sign; values are preserved without conversion.
ScholarQA-CS2 test set, Answer Precision (Show measurement, test conditions, and source)
- Source value
- 89
- Score
- 89
- Metric
- Answer Precision
- Unit
- No unit provided
- Benchmark version
- ScholarQA-CS2 test
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 100
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B DPO checkpoint; ScholarQA-CS2 test set of 100 user-written computer science research questions; questions are paired with already-retrieved literature excerpts, so this measures report generation and citation grounding, not retrieval.
- Context
- The model card prints scalar scores without a unit or explicit percent sign; values are preserved without conversion.
ScholarQA-CS2 test set, Citation Precision (Show measurement, test conditions, and source)
- Source value
- 90.5
- Score
- 90.5
- Metric
- Citation Precision
- Unit
- No unit provided
- Benchmark version
- ScholarQA-CS2 test
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 100
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B DPO checkpoint; ScholarQA-CS2 test set of 100 user-written computer science research questions; questions are paired with already-retrieved literature excerpts, so this measures report generation and citation grounding, not retrieval.
- Context
- The model card prints scalar scores without a unit or explicit percent sign; values are preserved without conversion.
ScholarQA-CS2 test set, Citation Recall (Show measurement, test conditions, and source)
- Source value
- 78.2
- Score
- 78.2
- Metric
- Citation Recall
- Unit
- No unit provided
- Benchmark version
- ScholarQA-CS2 test
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 100
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B DPO checkpoint; ScholarQA-CS2 test set of 100 user-written computer science research questions; questions are paired with already-retrieved literature excerpts, so this measures report generation and citation grounding, not retrieval.
- Context
- The model card prints scalar scores without a unit or explicit percent sign; values are preserved without conversion.
SQA-CS2 (Dev) (Show measurement, test conditions, and source)
- Source value
- 86.3
- Score
- 86.3
- Metric
- reported benchmark score
- Unit
- No unit provided
- Benchmark version
- SQA-CS2 (Dev)
- Category
- scientific report generation
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- disputed
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B row in the model-card comparison table; the card does not define the score scale or report a split-level sample count.
- Context
- The exact model card reports 86.3 for the dev aggregate but does not define its score scale or the dev-split sample count. The official Ai2 chart defines the four component metrics on a 0-100 scale and reports 86.5 as their mean. The discrepancy between the aggregate reports remains unresolved; this record retains the model-card value.
SQA-CS2 (Test) (Show measurement, test conditions, and source)
- Source value
- 87
- Score
- 87
- Metric
- reported benchmark score
- Unit
- No unit provided
- Benchmark version
- SQA-CS2 (Test)
- Category
- scientific report generation
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B row in the model-card comparison table; the card does not define the score scale or report a split-level sample count.
- Context
- The model-card table does not state a score unit, exact rubric, or split-level sample count.
DeepScholarBench (Show measurement, test conditions, and source)
- Source value
- 53.5
- Score
- 53.5
- Metric
- mean of Organization, Nugget coverage, Citation precision, and Claim coverage
- Unit
- points
- Benchmark version
- DeepScholarBench
- Category
- scientific report generation
- Direction
- Higher is better
- Sample
- 63
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief-8B model card, cross-checked with the official evaluation post
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief-8B row in the model-card table; official chart identifies this as the mean of four metrics on the held-out 63-query DeepScholarBench. Higher is better on the chart’s 0-100 scale.
- Context
- Vendor-reported. The source does not specify a per-query breakdown or the evaluator and judgment aggregation procedure for these component scores.
SQABench-CS2 dev, Ingredient Recall (Show measurement, test conditions, and source)
- Source value
- 89.3
- Score
- 89.3
- Metric
- Ingredient Recall
- Unit
- points
- Benchmark version
- SQABench-CS2 development split
- Category
- scientific report generation
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief official SQABench-CS2 evaluation chart
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief 8B report-generation result on the SQABench-CS2 development split. The chart defines the four measures and uses a 0-100 scale with higher scores better; Ai2 does not report the dev-split sample count.
- Context
- The model card reports a dev aggregate of 86.3, while the official chart reports 86.5 as the mean of these four dev metrics. The aggregate discrepancy is unresolved; these individual values are copied from the chart.
SQABench-CS2 dev, Answer Precision (Show measurement, test conditions, and source)
- Source value
- 89.2
- Score
- 89.2
- Metric
- Answer Precision
- Unit
- points
- Benchmark version
- SQABench-CS2 development split
- Category
- scientific report generation
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Ai2 AstaBrief official SQABench-CS2 evaluation chart
- Status
- active
- Retrieved at
- 2026-10-07
- Methodology
- Vendor-reported result. AstaBrief 8B report-generation result on the SQABench-CS2 development split. The chart defines the four measures and uses a 0-100 scale with higher scores better; Ai2 does not report the dev-split sample count.
- Context
- The model card reports a dev aggregate of 86.3, while the official chart reports 86.5 as the mean of these four dev metrics. The aggregate discrepancy is unresolved; these individual values are copied from the chart.
Page 1 of 2
Sources and data date
| Type | Evidence and data date |
|---|---|
| Research status | Research date October 7, 2026. 6 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication. |
| Unresolved | SQABench-CS2 dev split size and aggregate reconciliation; pairwise comparison counts Original wording Ai2 defines the four SQABench-CS2 metrics and says the main set contains 200 user-written computer-science questions, but it does not give the dev-split size. The exact model card reports 86.3 for the dev aggregate; the official chart reports 86.5 as the mean of its four dev values. Pairwise comparison sample counts and judge aggregation remain unspecified. The post does define DeepScholarBench as a 63-query held-out set, and its component metrics are now recorded. |
| Additional source | Ai2 AstaBrief announcement (retrieved October 7, 2026) · Ai2 AstaBrief announcement · Editorial description reviewed October 7, 2026 |
| Additional source | AstaBrief-8B model card (retrieved October 7, 2026) · AstaBrief-8B model card · Editorial description reviewed October 7, 2026 |
| Additional source | Ai2 AstaBrief official SQABench-CS2 evaluation chart (retrieved October 7, 2026) |
| Additional source | Ai2 AstaBrief official DeepScholarBench evaluation chart (retrieved October 7, 2026) |
| Additional source | Model metadata (retrieved October 7, 2026) |
| Additional source | Model metadata (retrieved October 7, 2026) |