WizardLM-2 8x22B

WizardLM-2 8x22B is Microsoft’s large sparse Mixture-of-Experts instruction model. The checkpoint has 141 billion total parameters, with roughly 39 billion activated per token. Microsoft positions it as the largest WizardLM-2 variant for complex conversation, multilingual work, reasoning, and agentic tasks. Its context window is 65,536 tokens.

The model card documents serving through Transformers, vLLM, and SGLang. Its model list also includes quantized versions for llama.cpp, Ollama, and LM Studio. The Apache-2.0 license applies to the published checkpoint.