DBRX

DBRX's defining feature is its fine-grained MoE architecture with 132 billion total parameters and 36 billion active. It selects four of 16 experts per token and was trained on roughly 12 trillion tokens. The instruct checkpoint combines a 32K context with coding, RAG, and SQL use for controlled enterprise deployment.

Databricks released base and instruct weights and described API use for Databricks customers alongside private deployment. This description concerns the instruct checkpoint, not DBRX Base.