Skip to main content
Language model

Mixtral 8x7B

Mistral AI

Released
December 2023
Input
Text
Output
Text

What is distinctive about Mixtral 8x7B is its sparse mixture-of-experts computation. A router selects two of eight experts per token; Mistral gives 46.7 billion total parameters and about 12.9 billion active parameters with a 32,000-token context window. The release supports five languages, coding, and Apache 2.0.

The larger variant Mixtral 8x22B retains Mixtral 8x7B’s router-and-expert design.

Specifications and access

SpecificationValue and source
Weights
Open weightsSource
Access
Parameters
46.7 billionSource
Active parameters
12.9 billionSource
Context window
32,768 tokensSource
License
Apache 2.0Source
Inputs
TextSource
Outputs
TextSource