Gestalt LabsDecision modelOpen source

Jeff 1

PopularityRank 358 of 876

36out of 100-2 vs. last month

Summary

Jeff 1 is a LoRA adapter on a Qwen3-4B-Instruct-2507 base model that scores candidate labels through the language model’s token probabilities. The adapter requires separately loaded base weights. Jeff runs locally, accepts text or structured data, and returns labels and probabilities for Choice, Noul, and Score questions.

A Jev-compatible interface describes technical compatibility, not an affiliation with TypeSafe AI or identical results to Jev. Jeff is an independent project with an Apache-2.0 license for the adapter and source code. You supply the controlled label sets at call time.

All benchmark results

Every published result with its source, test setup and comparison models.

Show all 17 published benchmark results

Measurements without matching peer values

These measurements have no matching peer values under the same test conditions. Their original values and sources remain available here.

Jeff 1 paired fact-check audit, supported predictions on gold-refuted rows (Show measurement, test conditions, and source)

Source value: 12.3
Score: 12.3
Metric: rate of 'supported' predictions among gold-refuted rows
Unit: %
Category: unsupported-positive-rate
Direction: No direction published
Source type: vendor-reported
Evaluator: Gestalt-Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-20
Methodology: Vendor-reported result. Jeff 1 paired fact-check audit. Pinned model-card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a.
Context: The card states that the 9,730-row evaluation set informed development and is not a pristine holdout. No subgroup denominator is published in the opened card.

Source Gestalt-Lab Jeff 1 model card

Jeff 1 paired fact-check audit, supported predictions on gold-not-enough-info rows (Show measurement, test conditions, and source)

Source value: 24.4
Score: 24.4
Metric: rate of 'supported' predictions among gold-not_enough_info rows
Unit: %
Category: unsupported-positive-rate
Direction: No direction published
Source type: vendor-reported
Evaluator: Gestalt-Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-20
Methodology: Vendor-reported result. Jeff 1 paired fact-check audit. Pinned model-card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a.
Context: The card states that the 9,730-row evaluation set informed development and is not a pristine holdout. No subgroup denominator is published in the opened card.

Source Gestalt-Lab Jeff 1 model card

Paired fact-check accuracy (Show measurement, test conditions, and source)

Source value: 0.8183
Score: 0.8183
Metric: accuracy
Unit: ratio
Category: fact-checking
Direction: Higher is better
Sample: 9730
Source type: vendor-reported
Evaluator: Gestalt Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-15
Methodology: Vendor-reported result. Canonical paired audit on same 9,730 human-labeled VitaminC, FEVER, SciFact and Climate-FEVER rows.
Context: Evaluation split informed development and error analysis; it is not a pristine holdout.

Source Gestalt Lab Jeff 1 model card

Paired fact-check macro-F1 (Show measurement, test conditions, and source)

Source value: 0.7789
Score: 0.7789
Metric: macro-F1
Unit: ratio
Category: fact-checking
Direction: Higher is better
Sample: 9730
Source type: vendor-reported
Evaluator: Gestalt Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-15
Methodology: Vendor-reported result. Canonical paired audit on same 9,730 human-labeled VitaminC, FEVER, SciFact and Climate-FEVER rows.
Context: Evaluation split informed development and error analysis; it is not a pristine holdout.

Source Gestalt Lab Jeff 1 model card

Paired fact-check Brier score (Show measurement, test conditions, and source)

Source value: 0.2839
Score: 0.2839
Metric: Brier score
Unit: ratio
Benchmark version: Gestalt-Lab Jeff-1 card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a
Direction: Lower is better
Sample: 9730
Source type: vendor-reported
Evaluator: Gestalt-Lab, Jeff-1 model card
Status: active
Retrieved at: 2026-10-16
Methodology: Vendor-reported result. GestaltLabs/Jeff-1 adapter artifacts/jev_clf/lora_4b_multi, SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a; base Qwen3-4B-Instruct-2507; paired fact-check test.
Context: This is the released Jeff adapter result, not the base Qwen checkpoint's performance.

Source Gestalt-Lab, Jeff-1 model card

Paired fact-check expected calibration error (Show measurement, test conditions, and source)

Source value: 0.0807
Score: 0.0807
Metric: expected calibration error (ECE; max probability, 10 equal-width bins)
Unit: ratio
Benchmark version: Gestalt-Lab Jeff-1 card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a
Direction: Lower is better
Sample: 9730
Source type: vendor-reported
Evaluator: Gestalt-Lab, Jeff-1 model card
Status: active
Retrieved at: 2026-10-16
Methodology: Vendor-reported result. GestaltLabs/Jeff-1 adapter artifacts/jev_clf/lora_4b_multi, SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a; max-probability ECE with 10 equal-width bins.
Context: This is the released Jeff adapter result, not the base Qwen checkpoint's performance.

Source Gestalt-Lab, Jeff-1 model card

Paired fact-check sealed sanity split accuracy (Show measurement, test conditions, and source)

Source value: 0.794
Score: 0.794
Metric: accuracy
Unit: ratio
Benchmark version: Gestalt-Lab Jeff-1 card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a
Direction: Higher is better
Sample: 199
Source type: vendor-reported
Evaluator: Gestalt-Lab, Jeff-1 model card
Status: active
Retrieved at: 2026-10-16
Methodology: Vendor-reported result. GestaltLabs/Jeff-1 adapter artifacts/jev_clf/lora_4b_multi, SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a; separate sealed 199-row sanity split, not for leaderboard ranking.
Context: The card labels this small split sanity-only; do not treat it as a comparable leaderboard result.

Source Gestalt-Lab, Jeff-1 model card

Paired fact-check sealed sanity split expected calibration error (Show measurement, test conditions, and source)

Source value: 0.0634
Score: 0.0634
Metric: expected calibration error (ECE)
Unit: ratio
Benchmark version: Gestalt-Lab Jeff-1 card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a
Direction: Lower is better
Sample: 199
Source type: vendor-reported
Evaluator: Gestalt-Lab, Jeff-1 model card
Status: active
Retrieved at: 2026-10-16
Methodology: Vendor-reported result. GestaltLabs/Jeff-1 adapter artifacts/jev_clf/lora_4b_multi, SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a; separate sealed 199-row sanity split, not for leaderboard ranking.
Context: The card labels this small split sanity-only; do not treat it as a comparable leaderboard result.

Source Gestalt-Lab, Jeff-1 model card

Jeff 1 paired fact-check audit, not-enough-info recall overall (Show measurement, test conditions, and source)

Source value: 0.578
Score: 0.578
Metric: recall on gold not_enough_info rows
Unit: ratio
Category: not-enough-info-recall
Direction: Higher is better
Source type: vendor-reported
Evaluator: Gestalt-Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-20
Methodology: Vendor-reported result. Jeff 1 paired fact-check audit. Pinned model-card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a.
Context: The card states that the 9,730-row evaluation set informed development and is not a pristine holdout. The Jev comparison is not included as a Jeff 1 score.

Source Gestalt-Lab Jeff 1 model card

Jeff 1 paired fact-check audit, not-enough-info recall on single-passage rows (Show measurement, test conditions, and source)

Source value: 0.523
Score: 0.523
Metric: recall on gold not_enough_info rows with a single passage
Unit: ratio
Category: not-enough-info-recall-single-passage
Direction: Higher is better
Source type: vendor-reported
Evaluator: Gestalt-Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-20
Methodology: Vendor-reported result. Jeff 1 paired fact-check audit. Pinned model-card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a.
Context: The card states that the 9,730-row evaluation set informed development and is not a pristine holdout. The Jev comparison is not included as a Jeff 1 score.

Source Gestalt-Lab Jeff 1 model card

Jeff 1 paired fact-check audit, accuracy on Climate-FEVER multi-passage rows (Show measurement, test conditions, and source)

Source value: 0.669
Score: 0.669
Metric: accuracy on Climate-FEVER multi-passage rows
Unit: ratio
Category: climate-fever-multipassage-accuracy
Direction: Higher is better
Sample: 859
Source type: vendor-reported
Evaluator: Gestalt-Lab Jeff 1 model card
Status: active
Retrieved at: 2026-10-20
Methodology: Vendor-reported result. Jeff 1 paired fact-check audit. Pinned model-card revision 14ee67e2814a19ebd0a67f508ac1b0f424ba1ebc; released adapter SHA-256 13cc3805495f7e901ca3121c7a3647fc6abcfe1fdc098ddc9ab1acd74f436a6a.
Context: The card states that the 9,730-row evaluation set informed development and is not a pristine holdout. The Jev comparison is not included as a Jeff 1 score.

Source Gestalt-Lab Jeff 1 model card

System One Mosaic Benchmark Noul composite (Show measurement, test conditions, and source)

Source value: 38.37
Score: 38.37
Metric: baseline-adjusted score averaged with equal benchmark weights on a 0-100 scale; Noul uses adjusted balanced accuracy
Unit: points
Benchmark version: S1MB result commit 29b48e35759743baf63715f62d7ee3f49ba15235; dataset revision b8dc678ecefb471c6d37311d8c93f65249622842; evaluator 0.1.0 revision 02bdeeb4536835e40012a6985df9c8f05045dd5f
Category: noul-composite
Direction: Higher is better
Source type: official-leaderboard
Evaluator: System One Mosaic Benchmark (S1MB) result submission
Status: active
Retrieved at: 2026-10-20
Methodology: Official benchmark leaderboard. Jeff-1 submission; model ID GestaltLabs/Jeff-1 at revision 6071dda782e1e45873f9f1ac001d7d7d47582ed3, base Qwen/Qwen3-4B-Instruct-2507 at revision cdbee75f17c01a7cc42f958dc650907174af0554. The composite averages 59 Noul benchmarks; the general Noul subset covers 2 benchmarks and is separately reported. One fixed submission/run: jeff-full-20260930-01; the six columns are not six independent runs. The source reports uncommitted evaluator/runtime changes.
Context: These are columns from one fixed submission/run, not independent runs. General-subset values are already included in their full category scores and do not establish unseen-task generalization. The source reports uncommitted evaluator/runtime changes.

Source System One Mosaic Benchmark (S1MB) result submission

Page 1 of 2

Popularity compared

How often the model is searched for, used and mentioned, compared with every model of its category.

Popularity index

0 to 100, higher is more popular

20 of 37 models

AutoTrust JEV-27B-VL70

Clef52

GLiNER2.5-multi-Decide45

Decider 4B43

Jev43

pplx-decider-v1.1-27b42

Intern-Decision-4B40

Laya Typed Decisions40

Kev 9B40

pplx-decider-v1-27b39

Decider 0.8B39

Julia 139

Intern-Decision-0.8B38

Jeff 136

GLiNER2.5-multi-Decide

Decider 4B

Jev

pplx-decider-v1.1-27b

Jeff 1

As of November 2026 · Source: gradually.ai popularity index

Over time

Weekly values from October 18, 2026 to November 6, 2026

Jeff 1Intern-Decision-2B 37CLM-v0.1-8B 37Decider 2B Vision 36GLiNER2.5-Decide-1B 34

Method changes in the timeline, a jump at these points is no market event:

1November 1, 2026: Value only with search or usage data, 30-day attention

What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.

Similar models

More models of the same kind as Jeff 1, chosen by model line, developer, size, price and release date.

All specifications

Architecture

Model class: Typed-decision LoRA adapter; Source: Model class
Parameters: 4 billion base model; Source: Parameters
Base model: Qwen/Qwen3-4B-Instruct-2507; Source: Base model

Input and output

Input: Text or structured data and typed questions; Source: Input
Output: Structured probabilities, no text generation; Source: Output

Watermarking and provenance

AI watermarking refers to invisible signals embedded in the output. Visible marks and provenance metadata are listed separately. Unknown means there is insufficient evidence for that particular field.

AI watermarking: Unknown; no source
Watermarking technique: Not known; no source

Licence and access

Access: Local; Source: Access
License: Apache-2.0 for the adapter and source code; Source: License

API

Model ID: GestaltLabs/Jeff-1; Source: Model ID

Further details

Result: Choice, Noul and Score with labels and probabilities; Source: Result
Scope: LoRA adapter on Qwen3-4B-Instruct-2507 for local inference.; Source: Scope
Notes: The adapter does not include the base-model weights and is not affiliated with TypeSafe AI.; Source: Notes
Adapter: LoRA, rank 16, alpha 32, dropout 0.05; Source: Adapter

Sources and data date

Show all 11 sources

Benchmark sources at a glance

Which independent evaluators tested this model. This is coverage, not a quality ranking.

No active result from the reviewed external evaluators can currently be attributed to this model. Other source evidence remains available below.

Without vendor claims and historical or disputed results; those stay labelled in the single values.

Type	Evidence and data date
Research status	Research date October 15, 2026. 4 source URLs checked. This documents the inspected sources, not an exhaustive inventory of every publication.
Not found in the inspected sources	Exact model result in the public Decision Index 0.4.1 panel Note This index edition has no row that can be mapped to the profile repository/checkpoint exactly. This does not imply the model lacks results in other benchmarks.
Additional source	Jeff repository (retrieved September 24, 2026) · Jeff repository · Editorial description reviewed October 4, 2026
Additional source	Jeff 1 adapter (retrieved September 24, 2026) · Jeff 1 adapter · Editorial description reviewed October 4, 2026
Additional source	Gestalt Lab Jeff 1 model card (retrieved October 15, 2026)
Additional source	Gestalt-Lab, Jeff-1 model card (retrieved October 16, 2026; October 20, 2026)
Additional source	System One Mosaic Benchmark (S1MB) result submission (retrieved October 20, 2026)
Additional source	OpenRouter: Popularity index (derived), CC BY 4.0
Additional source	Vercel AI Gateway: Popularity index (derived), CC BY 4.0
Additional source	Ollama: cumulative downloads for the popularity index
Additional source	Replicate: cumulative runs for the popularity index
Additional source	Google Trending Now: limited feed selection for Germany and the US
Additional source	Creative Commons Attribution 4.0