BAAILanguage modelOpen weights
AREX-2
AREX-2 has no overall index value yet because there are too few independent benchmark results.
40out of 100new calculation
based on two data groups, mostly search, range 29 to 52
Summary
AREX-2 learns to improve a solution over multiple rounds. BAAI trains the 27B checkpoint to propose, measure, reflect, and revise. Verifiable feedback from machine-learning and programming tasks is intended to transfer this behavior to deep research.
The multimodal model is available as open weights under Apache 2.0. BAAI documents local deployment through Transformers.
On this page
All benchmark results
Every published result with its source, test setup and comparison models.
Show all 11 published benchmark results
Measurements without matching peer values
These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.
MLE-Lite (Show measurement, test conditions, and source)
- Source value
- 81.8
- Metric
- Any Medal rate: percentage of competitions earning a medal, averaged over three seeds
- Unit
- %
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated with skills in context under the OpenMLE MLE-bench Lite protocol, with a maximum 12-hour budget per task. Any Medal is the percentage of competitions in which the agent earns a medal, averaged over three seeds.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
Frontier-CS Agent Track / mean best-so-far / 1-hour budget (Show measurement, test conditions, and source)
- Source value
- 54.4
- Metric
- Mean best-so-far Frontier-CS score after one hour per task
- Unit
- %
- Benchmark version
- Frontier-CS Agent Track as reported in AREX-2 arXiv:2609.38288v1
- Category
- coding
- Direction
- Higher is better
- Sample
- 188
- Source type
- vendor-reported
- Evaluator
- AREX Team, Beijing Academy of Artificial Intelligence, AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks, arXiv:2609.38288v1
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Exact BAAI/AREX-2 agent, built on Qwen3.8-27B, in the authors' optimized harness. The study uses the 188 Frontier-CS Agent Track problems with a five-hour maximum budget per problem. In each round the agent writes or revises code, compiles it, runs provided sample cases plus its own randomized and edge-case tests, submits a solution, and uses evaluation feedback for the next round. Section 3.3.1 reports this one-hour cell; Figure 4 defines the curve as mean best-so-far Frontier-CS score across the 188 tasks. Figure 2 labels the same Frontier-CS score axis (%) and shows the matching five-hour score of 70.7%, providing the source's percent-scale anchor. Section 3.3.1 describes checkpoint gains as points.
- Context
- Section 3.3.1 describes changes between checkpoints as points, while Figure 2 explicitly labels Frontier-CS Score (%) and shows the same five-hour endpoint that Figure 4 reports. The absolute one-hour checkpoint is therefore stored as a percentage on that metric; no relative gain or percentage-point change is inferred. The paper does not separately define whether its points wording for changes means percentage points. No confidence interval is published. This checkpoint shares the same 188-task evaluation cohort and agent runs as the later checkpoints, so it is not an independent rerun.
Frontier-CS Agent Track / mean best-so-far / 2-hour budget (Show measurement, test conditions, and source)
- Source value
- 65.9
- Metric
- Mean best-so-far Frontier-CS score after two hours per task
- Unit
- %
- Benchmark version
- Frontier-CS Agent Track as reported in AREX-2 arXiv:2609.38288v1
- Category
- coding
- Direction
- Higher is better
- Sample
- 188
- Source type
- vendor-reported
- Evaluator
- AREX Team, Beijing Academy of Artificial Intelligence, AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks, arXiv:2609.38288v1
- Status
- active
- Retrieved at
- 2026-10-17
- Methodology
- Vendor-reported result. Exact BAAI/AREX-2 agent, built on Qwen3.8-27B, in the authors' optimized harness. The study uses the 188 Frontier-CS Agent Track problems with a five-hour maximum budget per problem. In each round the agent writes or revises code, compiles it, runs provided sample cases plus its own randomized and edge-case tests, submits a solution, and uses evaluation feedback for the next round. Section 3.3.1 reports this two-hour cell; Figure 4 defines the curve as mean best-so-far Frontier-CS score across the 188 tasks. Figure 2 labels the same Frontier-CS score axis (%) and shows the matching five-hour score of 70.7%, providing the source's percent-scale anchor. Section 3.3.1 describes checkpoint gains as points.
- Context
- Section 3.3.1 describes changes between checkpoints as points, while Figure 2 explicitly labels Frontier-CS Score (%) and shows the same five-hour endpoint that Figure 4 reports. The absolute two-hour checkpoint is therefore stored as a percentage on that metric; no relative gain or percentage-point change is inferred. The paper does not separately define whether its points wording for changes means percentage points. No confidence interval is published. This checkpoint shares the same 188-task evaluation cohort and agent runs as the one- and five-hour checkpoints, so it is not an independent rerun.
MLE-bench Lite (Show measurement, test conditions, and source)
- Source value
- 75.8
- Metric
- Any Medal, percentage of competitions with a medal averaged over three seeds
- Unit
- %
- Benchmark version
- arXiv:2609.38288v1, stage-wise ablation (Figure 6 and section 3.3.3), stage M3
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- AREX Team, Beijing Academy of Artificial Intelligence, AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks, Figure 6 and section 3.3.3
- Status
- active
- Retrieved at
- 2026-10-18
- Methodology
- Vendor-reported result. AREX-2 (27B), MLE-bench Lite with task-specific skills in context, OpenMLE protocol (maximum 12 hours per task), stage M3; M3 uses the same skills as M2 but the base round count used by M0/M1. The paper does not give the numeric round count or an immutable evaluated checkpoint or serving route.
- Context
- The paper does not state the number of competitions scored for the 75.8 cell or report a confidence interval. The three-seed average is not itself a confidence interval. The evaluated checkpoint or serving route is not pinned.
MLE-bench Lite (Show measurement, test conditions, and source)
- Source value
- 81.8
- Metric
- Any Medal rate; mean over three seeds
- Unit
- %
- Benchmark version
- AREX-2 stage M4
- Direction
- Higher is better
- Source type
- independent-evaluator
- Evaluator
- OpenMLE, as reported in AREX-2 (arXiv:2609.38288)
- Status
- active
- Retrieved at
- 2026-10-22
- Methodology
- Independent evaluation. AREX-2 (27B) on MLE-bench Lite with skills in context. Section 3.3.3 identifies the 81.8 cell as stage M4, using the same task-specific skills as M2 and more rounds, specifically the same round count as M2. Table 1 says OpenMLE reproduces and reports the MLE-Lite results and defines Any Medal as the mean over three seeds. The exact competition count, numeric round count, immutable checkpoint, and serving endpoint are not disclosed.
- Context
- The source assigns 81.8 to the M4 configuration with more rounds, distinct from the pinned 75.8 M3 baseline with the base round count. The paper gives the mean over three seeds, but neither the exact competition count nor a confidence interval. A three-seed mean is not a confidence interval. This is an agent-system evaluation with skills and iterative rounds, not an unassisted base-model score.
Frontier-CS (Show measurement, test conditions, and source)
- Source value
- 70.7
- Metric
- Mean best-so-far score on the Frontier-CS Agent Track
- Unit
- No unit provided
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Sample
- 188
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated on the 188-task Frontier-CS Agent Track with its optimized harness and a maximum five-hour execution budget per task. The paper reports the mean best-so-far score over the track. It does not specify the score unit or scale in the row.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
BrowseComp (Show measurement, test conditions, and source)
- Source value
- 84
- Metric
- Accuracy on BrowseComp
- Unit
- No unit provided
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated with the paper's non-coding deep-research protocol, with at most 300 inner turns and 1,500 overall turns. The score is reported as accuracy. The source does not give an item count for this model cell.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
HLE text-only (Show measurement, test conditions, and source)
- Source value
- 52.6
- Metric
- Accuracy on the text-only subset of Humanity's Last Exam
- Unit
- No unit provided
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated on the text-only HLE subset with the paper's non-coding deep-research protocol, capped at 300 inner turns and 1,500 overall turns. The score is reported as accuracy. The source does not give an item count for this model cell.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
GAIA (Show measurement, test conditions, and source)
- Source value
- 92.2
- Metric
- Accuracy on GAIA
- Unit
- No unit provided
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated with the paper's non-coding deep-research protocol, with at most 300 inner turns and 1,500 overall turns. The score is reported as accuracy. The source does not give an item count for this model cell.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
DeepSearchQA F1 (Show measurement, test conditions, and source)
- Source value
- 93.8
- Metric
- F1 on DeepSearchQA
- Unit
- No unit provided
- Benchmark version
- arXiv:2609.38288v1
- Category
- source-specific
- Direction
- Higher is better
- Source type
- vendor-reported
- Evaluator
- Technical report
- Status
- active
- Retrieved at
- 2026-10-04
- Methodology
- AREX-2 was evaluated with the paper's non-coding deep-research protocol, with at most 300 inner turns and 1,500 overall turns. The reported metric is F1. The source does not give an item count for this model cell.
- Context
- Source-specific observation; it is not a shared comparison cohort. The source table does not state a unit or scale for this numeric score.
BrowseComp (Show measurement, test conditions, and source)
- Source value
- 64.8
- Metric
- accuracy at 47 turns
- Unit
- No unit provided
- Direction
- No direction published
- Source type
- vendor-reported
- Evaluator
- AREX-2 paper (arXiv)
- Status
- active
- Retrieved at
- 2026-10-19
- Methodology
- Vendor-reported result. AREX-2 paper BrowseComp budget curve; score reported at 47 turns, with no correctness score fed back during a run. The paper does not publish a model-run N or confidence interval for this point.
- Context
- The pinned record already contains the 84.0 result at 143 turns. This candidate is only the separate 47-turn condition; the source excerpt does not specify a score denominator, unit, or confidence interval.
Popularity compared
How often the model is searched for, used and mentioned, compared with every model of its category.
Popularity index
The 100 most popular models of this category, 0 to 100, higher is more popular
20 of 101 models
As of November 2026 · Source: gradually.ai popularity index
Over time
Weekly values from October 18, 2026 to November 20, 2026
AREX-2GPT-5.3-Codex 41Mistral Small 3 40Magistral Small 40Mixtral 8x22B 40
Method changes in the timeline, a jump at these points is no market event:
- 1November 1, 2026: Value only with search or usage data, 30-day attention
- 2November 1, 2026: Method version 4 (common scale, shrinkage)
- 3November 9, 2026: New topics measurement V20 (daily quorum, short forms)
What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.
Similar models
More models of the same kind as AREX-2, chosen by model line, developer, size, price and release date.
All specifications
Architecture
Input and output
Watermarking and provenance
AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.
- AI watermarking
- Unknown
- no source
- Watermarking technique
- Not known
- no source
Licence and access
- Shutdown
- None announced
- no source
API
Frequently asked questions about AREX-2
How large is the context window of AREX-2?
262,144 tokens. Separate limits may apply to input and output.
Can I run AREX-2 locally?
Yes, the weights are open.
Stated by BAAI
- Open weights
- yes
- License
- Apache 2.0
- Parameters
- 27B
- active per token
- n/a
Sources and data date
Show all 11 sources
Benchmark sources at a glance
Which independent evaluators tested this model. This is coverage, not a quality ranking.
No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.
Without vendor claims and historical or disputed results; those stay labelled in the single values.