Skip to main content
Language modelOpen weights

K2 Horizon MoVA 36B-A4B

Institute of Foundation Models

Released
September 3, 2026
Data date
October 7, 2026
Input
Text
Output
Text

K2 Horizon MoVA 36B-A4B complements the dense 32B model with a sparse architecture. Mixture-of-Value Attention routes attention through experts, while roughly 4 billion of its 36 billion parameters are active per token.

Specifications and access

SpecificationValue and source
Parameters
36B total parametersSource
Active parameters
4B parameters per tokenSource
Context window
524,288 tokensSource
Input modalities
TextSource
Output modalities
TextSource
Documented training stages
Pretraining, four midtraining stages, and two SFT phases.Source
License
Apache 2.0Source