Skip to main content

Convai InnovationsDecision modelOpen source

Laya Multilingual

PopularityRank 159 of 876

46out of 100same as last month

Summary

Laya Multilingual shifts the family from English general use toward multilingual local readouts. The checkpoint has 322 million parameters and supports up to 8,192 tokens, while the default is max_len=1024. Like the other Laya models, it answers typed questions with Choice, Noul, and Score and returns structured results.

Set max_len=8192 to use the maximum context. The checkpoint scores lower than Laya on the reported English tasks. Laya Multilingual is therefore the family variant for mixed-language input, while Laya leads the English comparison.

GraduallyBench Decision

Automatically evaluated against predefined rules.

The pilot covers 60 test cases from 30 scenarios in German and English. Five equally weighted task families form the score from 0 to 100. Failed requests and invalid answers count as unsuccessful decisions. The 30 robustness variants and 40 development cases are reported separately.

The reference answers follow stored rules and calculations. The automatic reference check took place after the model runs. The task wording and translations have not been checked by human reviewers. Results apply to the exact tested model and route.

Protocol decision-choice-automated-v1. No verified provider invoice.

Laya Multilingual

Convai Innovations · laya-multilingual · local-cpu · Tested Nov 5, 2026

Required decisions across the test plan
36.7 % (22 / 60)
Valid test responses
60 / 60
Correct robustness variants
13 / 30
Correct development cases, outside the test score
16 / 40
Unnecessary escalations
2 / 52
Missed required escalations
7 / 8
Local CPU, median / p95
53 / 78 ms (n = 130)
API cost
$0.00

Local compute cost is not measured. CPU timing and hosted API timing describe different environments.

Apple M5 Max, 4 CPU-Threads

Tested checkpoint e4e9ddf21a7b

Results by task family
TaskCorrect / plannedTimeoutsTransport errorsInvalid answers
Triage2 / 12000
Routing6 / 12000
Selection with constraints3 / 12000
Numbers and tables4 / 12000
Sufficient evidence7 / 12000

All benchmark results

Every published result with its source, test setup and comparison models.

Show all 46 published benchmark results

Page 1 of 4

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

StreamDecisionBench (Show measurement, test conditions, and source)
Source value
0.21
Score
0.21
Metric
Mean log-AUC 0.5–8 s
Unit
%
Direction
No direction published
Sample
480
Source type
independent-evaluator
Evaluator
StreamDecisionBench GitHub README
Status
active
Retrieved at
2026-10-18
Methodology
Independent evaluation. Self-hosted open-weight setting, exact source row “Laya multilingual”. Three passes over all 480 states per pass (8 scenarios in 4 families), sampled at a 2 s step; the source reports means. The README says all 17 self-hosted settings use BF16 backbones, each on one RTX PRO 6000 (96 GB), with benchmark and model on the same GPU host. Same-host latency includes request processing, runtime queueing, and inference, and excludes download, initialization, and warmup. Serving stack, exact checkpoint revision, and other runtime settings are not stated.
Source StreamDecisionBench GitHub README
StreamDecisionBench (Show measurement, test conditions, and source)
Source value
0.21
Score
0.21
Metric
Mean untimed (%)
Unit
%
Direction
No direction published
Sample
480
Source type
independent-evaluator
Evaluator
StreamDecisionBench GitHub README
Status
active
Retrieved at
2026-10-18
Methodology
Independent evaluation. Self-hosted open-weight setting, exact source row “Laya multilingual”. Three passes over all 480 states per pass (8 scenarios in 4 families), sampled at a 2 s step; the source reports means. The README says all 17 self-hosted settings use BF16 backbones, each on one RTX PRO 6000 (96 GB), with benchmark and model on the same GPU host. Same-host latency includes request processing, runtime queueing, and inference, and excludes download, initialization, and warmup. Serving stack, exact checkpoint revision, and other runtime settings are not stated.
Source StreamDecisionBench GitHub README
Jev docs bilingual v2 Compact EN accuracy (Show measurement, test conditions, and source)
Source value
35.7
Score
35.7
Metric
Compact EN accuracy
Unit
%
Benchmark version
Jev docs bilingual expansion v2, Compact condition
Category
decision
Direction
Higher is better
Sample
84
Source type
community-reported
Evaluator
akitaonrails/jev-docs-benchmarks, bilingual v2 results
Status
active
Retrieved at
2026-10-23
Methodology
Community-reported result. Bilingual expansion v2, Compact condition; source reports 84 authored families with paired translations and three correlated conditions. The model row is `laya-multilingual`; target checkpoint revision is not stated.
Context
The source says 84 authored families; it does not separately report model-specific completion counts or checkpoint revision. Task cells have only 12 families and are exploratory.
Source akitaonrails/jev-docs-benchmarks, bilingual v2 results
Jev docs bilingual v2 Compact PT-BR accuracy (Show measurement, test conditions, and source)
Source value
38.1
Score
38.1
Metric
Compact PT-BR accuracy
Unit
%
Benchmark version
Jev docs bilingual expansion v2, Compact condition
Category
decision
Direction
Higher is better
Sample
84
Source type
community-reported
Evaluator
akitaonrails/jev-docs-benchmarks, bilingual v2 results
Status
active
Retrieved at
2026-10-23
Methodology
Community-reported result. Bilingual expansion v2, Compact condition; source reports 84 authored families with paired translations and three correlated conditions. The model row is `laya-multilingual`; target checkpoint revision is not stated.
Context
The source says 84 authored families; it does not separately report model-specific completion counts or checkpoint revision. Task cells have only 12 families and are exploratory.
Source akitaonrails/jev-docs-benchmarks, bilingual v2 results
typed-decisions (Show measurement, test conditions, and source)
Source value
0.342
Score
0.342
Metric
accuracy
Unit
ratio
Category
decision
Direction
Higher is better
Sample
2000
Source type
vendor-reported
Evaluator
ConvAI Innovations Laya model card
Status
active
Retrieved at
2026-10-15
Methodology
Vendor-reported result. Shared typed-decisions benchmark: 400 cases, 2,000 decisions, four workflows; exact checkpoint row.
Source ConvAI Innovations Laya model card
Typed-decisions soft accuracy (Show measurement, test conditions, and source)
Source value
0.326
Score
0.326
Metric
soft accuracy
Unit
ratio
Benchmark version
convaiinnovations/laya-typed-decisions README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0
Direction
Higher is better
Sample
2000
Source type
vendor-reported
Evaluator
ConvAI Innovations, Laya typed-decisions model card
Status
active
Retrieved at
2026-10-16
Methodology
Vendor-reported result. Official comparison table; 400-case, 2,000-decision test split; checkpoint named Laya multilingual; card states 1,200 training cases. Pinned README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0; evaluated weight hash is not disclosed.
Context
The table reports one shared typed-decision test; it is not evidence of multilingual quality or performance outside its stated test distribution.
Source ConvAI Innovations, Laya typed-decisions model card
Typed-decisions Brier score (Show measurement, test conditions, and source)
Source value
0.439
Score
0.439
Metric
Brier score
Unit
ratio
Benchmark version
convaiinnovations/laya-typed-decisions README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0
Direction
Lower is better
Sample
2000
Source type
vendor-reported
Evaluator
ConvAI Innovations, Laya typed-decisions model card
Status
active
Retrieved at
2026-10-16
Methodology
Vendor-reported result. The official comparison table's 400-case, 2,000-decision test split; checkpoint named Laya multilingual. The table says 1,200 training cases. Reviewed README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0; evaluated weight hash is not disclosed.
Context
The public card reports one shared English typed-decision test; it does not establish multilingual quality. No exact evaluated weight hash or uncertainty interval is published.
Source ConvAI Innovations, Laya typed-decisions model card
Typed-decisions expected calibration error (Show measurement, test conditions, and source)
Source value
0.285
Score
0.285
Metric
expected calibration error (ECE)
Unit
ratio
Benchmark version
convaiinnovations/laya-typed-decisions README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0
Direction
Lower is better
Sample
2000
Source type
vendor-reported
Evaluator
ConvAI Innovations, Laya typed-decisions model card
Status
active
Retrieved at
2026-10-16
Methodology
Vendor-reported result. The official comparison table's 400-case, 2,000-decision test split; checkpoint named Laya multilingual. The table says 1,200 training cases. Reviewed README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0; evaluated weight hash is not disclosed.
Context
The public card reports one shared English typed-decision test; it does not establish multilingual quality. No exact evaluated weight hash or uncertainty interval is published.
Source ConvAI Innovations, Laya typed-decisions model card
Typed-decisions score mean absolute error (Show measurement, test conditions, and source)
Source value
0.687
Score
0.687
Metric
score mean absolute error (MAE)
Unit
No unit provided
Benchmark version
convaiinnovations/laya-typed-decisions README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0
Direction
Lower is better
Source type
vendor-reported
Evaluator
ConvAI Innovations, Laya typed-decisions model card
Status
active
Retrieved at
2026-10-16
Methodology
Vendor-reported result. The official comparison table's test has 400 cases and 2,000 decisions overall; the score-MAE denominator is not stated separately. Checkpoint named Laya multilingual; the card says 1,200 training cases. Pinned README revision e929ae5cf69bc34259cd2f95c9e91145b818b1f0; evaluated weight hash is not disclosed.
Context
The card does not define the score target range/scale or a separate MAE denominator. Do not interpret the value as a normalized percentage or infer a sample count; no exact evaluated weight hash or uncertainty interval is published.
Source ConvAI Innovations, Laya typed-decisions model card
System One SST-2: accuracy (Show measurement, test conditions, and source)
Source value
0.8567
Score
0.8567
Metric
accuracy
Unit
ratio
Benchmark version
System One 2026-09-26 run; evaluator commit ed4544869a9480aa98db6572a21733803eadcbb8; Laya 0.3.20, release-default multilingual checkpoint; SST-2
Category
classification
Direction
Higher is better
Sample
300
Lower confidence bound
0.812
Upper confidence bound
0.892
Source type
official-leaderboard
Evaluator
System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8
Status
active
Retrieved at
2026-10-17
Methodology
Official benchmark leaderboard. System One shared Jev-style POST /v1/systemone typed-choice request; release defaults; no benchmark fine-tuning; one sequential request per example on Apple M5 Pro. Laya 0.3.20, release-default multilingual checkpoint. The evaluator does not report the loaded weight hash. Hugging Face revision e4e9ddf21a7b1903b7acffd8814ad4307bf63a67 was the latest public revision with lastModified no later than the 2026-09-26 run; this is a chronology-based candidate, not evaluator-confirmed loaded bytes. SST-2 test subset, N=300. Accuracy. Fixed sample seed 42 per the public report.
Context
One run on one Apple M5 Pro; no evaluator-confirmed loaded weight hash. The report says these datasets are not listed as training data for the named model, but non-mention does not prove no overlap. MASSIVE question/options remain English. All 104 System One cells in this bundle come from the same 2026-09-26 run. The benchmark-author report is hosted on Laya AI’s vendor site. The linked GitHub repository describes itself as independent and unaffiliated; neither that assertion nor the author/vendor relationship has been independently confirmed.
Source System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8
System One SST-2: expected calibration error (top answer; 10 equal-width bins) (Show measurement, test conditions, and source)
Source value
0.1009
Score
0.1009
Metric
expected calibration error (top answer; 10 equal-width bins)
Unit
ratio
Benchmark version
System One 2026-09-26 run; evaluator commit ed4544869a9480aa98db6572a21733803eadcbb8; Laya 0.3.20, release-default multilingual checkpoint; SST-2
Category
classification
Direction
Lower is better
Sample
300
Source type
official-leaderboard
Evaluator
System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8
Status
active
Retrieved at
2026-10-17
Methodology
Official benchmark leaderboard. System One shared Jev-style POST /v1/systemone typed-choice request; release defaults; no benchmark fine-tuning; one sequential request per example on Apple M5 Pro. Laya 0.3.20, release-default multilingual checkpoint. The evaluator does not report the loaded weight hash. Hugging Face revision e4e9ddf21a7b1903b7acffd8814ad4307bf63a67 was the latest public revision with lastModified no later than the 2026-09-26 run; this is a chronology-based candidate, not evaluator-confirmed loaded bytes. SST-2 test subset, N=300. Top-answer ECE with 10 equal-width bins. Fixed sample seed 42 per the public report.
Context
One run on one Apple M5 Pro; no evaluator-confirmed loaded weight hash. The report says these datasets are not listed as training data for the named model, but non-mention does not prove no overlap. MASSIVE question/options remain English. All 104 System One cells in this bundle come from the same 2026-09-26 run. The benchmark-author report is hosted on Laya AI’s vendor site. The linked GitHub repository describes itself as independent and unaffiliated; neither that assertion nor the author/vendor relationship has been independently confirmed.
Source System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8
System One TREC: accuracy (Show measurement, test conditions, and source)
Source value
0.86
Score
0.86
Metric
accuracy
Unit
ratio
Benchmark version
System One 2026-09-26 run; evaluator commit ed4544869a9480aa98db6572a21733803eadcbb8; Laya 0.3.20, release-default multilingual checkpoint; TREC
Category
classification
Direction
Higher is better
Sample
300
Lower confidence bound
0.816
Upper confidence bound
0.895
Source type
official-leaderboard
Evaluator
System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8
Status
active
Retrieved at
2026-10-17
Methodology
Official benchmark leaderboard. System One shared Jev-style POST /v1/systemone typed-choice request; release defaults; no benchmark fine-tuning; one sequential request per example on Apple M5 Pro. Laya 0.3.20, release-default multilingual checkpoint. The evaluator does not report the loaded weight hash. Hugging Face revision e4e9ddf21a7b1903b7acffd8814ad4307bf63a67 was the latest public revision with lastModified no later than the 2026-09-26 run; this is a chronology-based candidate, not evaluator-confirmed loaded bytes. TREC test subset, N=300. Accuracy. Fixed sample seed 42 per the public report.
Context
One run on one Apple M5 Pro; no evaluator-confirmed loaded weight hash. The report says these datasets are not listed as training data for the named model, but non-mention does not prove no overlap. MASSIVE question/options remain English. All 104 System One cells in this bundle come from the same 2026-09-26 run. The benchmark-author report is hosted on Laya AI’s vendor site. The linked GitHub repository describes itself as independent and unaffiliated; neither that assertion nor the author/vendor relationship has been independently confirmed.
Source System One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8

Page 1 of 4

Popularity compared

How often the model is searched for, used and mentioned, compared with every model of its category.

Popularity index

0 to 100, higher is more popular

20 of 37 models

AutoTrust JEV-27B-VL70
AutoTrust JEV-27B63
GLiNER2.5-Decide56
Decider 2B52
Clef52
Clef-Flash51
Laya Multilingual46
Kev 0.8B45
GLiNER2.5-multi-Decide45
Decider 4B43
Jev43
pplx-decider-v1.1-27b42
Intern-Decision-4B40
Laya Typed Decisions40
Kev 9B40
pplx-decider-v1-27b39
Decider 0.8B39
Julia 139
Intern-Decision-0.8B38
Intern-Decision-2B37

As of November 2026 · Source: gradually.ai popularity index

Over time

Weekly values from October 18, 2026 to November 6, 2026

Laya MultilingualKev 0.8B 45GLiNER2.5-multi-Decide 45Decider 4B 43Jev 43

Method changes in the timeline, a jump at these points is no market event:

  1. 1November 1, 2026: Value only with search or usage data, 30-day attention

What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.

Similar models

More models of the same kind as Laya Multilingual, chosen by model line, developer, size, price and release date.

All specifications

Architecture

Model class
Multilingual System One decision model
Source: Model class
Parameters
322 million
Source: Parameters

Input and output

Input
Multilingual text and typed questions
Source: Input
Output
Structured probabilities, no text generation
Source: Output
Context window
8,192 tokens (1,024 by default)
Source: Context window

Watermarking and provenance

AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.

AI watermarking
Unknown
no source
Watermarking technique
Not known
no source

Licence and access

Access
Local
Source: Access
License
Apache-2.0
Source: License

API

Model ID
convaiinnovations/laya/multilingual
Source: Model ID

Further details

Result
Choice, Noul and Score with probabilities
Source: Result
Scope
Multilingual Laya variant for local System One readouts.
Source: Scope
Notes
The documented maximum requires max_len=8192. This checkpoint is weaker in English than the English Laya checkpoint.
Source: Notes

Sources and data date

Show all 28 sources

Benchmark sources at a glance

Which independent evaluators tested this model. This is coverage, not a quality ranking.

No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.

Without vendor claims and historical or disputed results; those stay labelled in the single values.

TypeEvidence and data date
Research statusResearch date October 15, 2026. 22 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not published

non-english-heldout-benchmarks

Note

The model card's published typed-decision comparison is the same English test split and supplies no separate non-English evaluation.

Not published

evaluated-weight-artifact-hash

Note

The card names the evaluated checkpoint but does not bind the comparison to an immutable weight-file hash.

Not published

evaluator-loaded-weight-digest

Note

The report does not disclose loaded checkpoint hash or revision. The matching public Hugging Face revision by lastModified date is only a chronology candidate.

Not published

test-dataset-revision

Note

Fixed samples and sample counts are reported, but dataset revision hashes are not pinned in the captured protocol.

Not published

unreported-confidence-intervals

Note

No CI is published for ECE or MASSIVE per-language accuracy; none was calculated or inferred.

Not published

loaded-checkpoint-bytes

Note

The paper identifies the model revision; it does not publish a run artifact hash proving the evaluator loaded bytes from that revision.

Not found in the inspected sources

Exact model result in the public Decision Index 0.4.1 panel

Note

This index edition has no row that can be mapped to the profile repository/checkpoint exactly. This does not imply the model lacks results in other benchmarks.

Additional sourceLaya repository (retrieved September 24, 2026) · Laya repository · Editorial description reviewed October 2, 2026
Additional sourceLaya multilingual checkpoint (retrieved September 24, 2026; October 17, 2026) · Laya multilingual checkpoint · Editorial description reviewed October 2, 2026
Additional sourceConvAI Innovations Laya model card (retrieved October 15, 2026; October 19, 2026)
Additional sourceConvAI Innovations, Laya typed-decisions model card (retrieved October 16, 2026)
Additional sourceSystem One report, Laya AI-hosted result summary for laya-multilingual, pinned source commit ed4544869a9480aa98db6572a21733803eadcbb8 (retrieved October 17, 2026)
Additional sourceFadli et al., Does System One Transfer?, arXiv:2610.14112v1, Table 3 (retrieved October 17, 2026)
Additional sourceLaya GitHub issue #450, public result report by bhushankinge (retrieved October 17, 2026)
Additional sourceStreamDecisionBench GitHub README (retrieved October 18, 2026)
Additional sourceLaya Benchmarks, upstream measurement table (citing Laya BENCHMARKS.md) (retrieved October 19, 2026)
Additional sourceaashish254, independent multilingual rerun reported in Laya issue #208 (retrieved October 22, 2026)
Additional sourceLaya upstream BENCHMARKS.md (retrieved October 22, 2026)
Additional sourceakitaonrails/jev-docs-benchmarks, bilingual v2 results (retrieved October 23, 2026)
Additional sourceModel metadata (retrieved October 16, 2026)
Additional sourceModel metadata (retrieved October 17, 2026)123456789
Additional sourceOpenRouter: Popularity index (derived), CC BY 4.0
Additional sourceVercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional sourceOllama: cumulative downloads for the popularity index
Additional sourceReplicate: cumulative runs for the popularity index
Additional sourceGoogle Trending Now: limited feed selection for Germany and the US
Additional sourceCreative Commons Attribution 4.0