Skip to main content
Language model

Falcon 3 10B

TII

Released
December 2024

Falcon 3 10B is TII’s larger open model in the Falcon 3 cycle. TII describes it as the result of depth upscaling, where redundant layers from the Falcon 3 7B training line were duplicated and then continued with another 2 trillion high-quality tokens.

The base checkpoint has a 32,768-token context and uses the Falcon license. A separate instruction-tuned derivative is available for dialogue.

Specifications and access

SpecificationValue and source
Weights
Open weightsSource
Parameters
10 billionSource
Context window
32,768 tokensSource
License
Falcon LicenseSource
Inputs
TextSource
Outputs
TextSource