The System One Mosaic Benchmark, ranked #1 in English (160 models) and Japanese (16 models) as of 2026-10-14. A typed-decision judge that scores a decision in a single forward pass, with zero generated tokens.
-ff6b8b?style=for-the-badge){kind=icon}
Darwin-27B-ZTC-v5 is VIDRAFT's typed-decision judge. On the System One Mosaic Benchmark (S1MB) it ranked #1 overall in both English (160 models) and Japanese (16 models) as of 2026-10-14. It reads a decision problem — a condition, a set of choices, a score to assign — and returns its judgement in one forward pass, generating zero tokens, using the ZTC (Zero-Token Confidence) method.
The weights are on Hugging Face. This repository holds the result, the benchmark context, and how to run the model.
| Rank | Benchmark | Field | Models | Result |
|---|---|---|---|---|
| #1 | S1MB English (System One Mosaic) | Typed decisions | 160 | 1st overall · Borda 90.80 (2026-10-14) |
| #1 | S1MB Japanese | Typed decisions | 16 | 1st overall · Borda 84.57 (2026-10-14) |
- Leaderboard: https://huggingface.co/spaces/hotchpotch/S1MB-leaderboard
- Typed-decision accuracy: 0.743 zero-shot, #1.
- Decisions are scored with zero generated tokens, so judgement is deterministic and one forward pass per item.
S1MB (System One Mosaic Benchmark) evaluates a model's fast, intuitive "System 1" decision quality across a mosaic of typed decision tasks: given a condition and candidate choices, pick and score the right one. It is a decision benchmark, not a text-generation benchmark, which is exactly what a zero-token judge is built for.
| Model | Role | Params | Hugging Face |
|---|---|---|---|
| Darwin-27B-ZTC-v2 | Typed-decision judge, #1 on S1MB (2026-10-09) | 27B | link |
| Darwin-27B-ZTC | Typed-decision judge (v1) | 27B | link |
| Darwin-397B-ZTC | Largest judge | 397B | link |
The full ZTC family (including the ZTC-Judge answer-verification line) lives at github.com/final-bench/ztc.
text = template(condition, choices)
h = model(text).last_hidden_state[last_non_pad_token] # one forward pass, 0 tokens generated
score = ((h - mu) / sd) @ w # calibrated probe -> decision
- Zero tokens: no decoding loop, so the decision is deterministic and cheap.
- One pass: one forward pass per decision, fast enough to run on every item.
- Calibrated: the probe is fitted leave-one-domain-out, so reported figures come from domains the probe never saw.
Full method and inference code (ztc_judge.py, usage_curve.py): github.com/final-bench/ztc.
from transformers import AutoModel, AutoTokenizer
# weights: https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v5
# decision scoring code: https://github.com/final-bench/ztcSee the model card and the ztc repository for the exact decision template and probe parameters.
Which model is #1 on S1MB?
As of 2026-10-09, Darwin-27B-ZTC-v2 by VIDRAFT was #1 overall on the System One Mosaic Benchmark among 102 models; as of 2026-10-14, Darwin-27B-ZTC-v5 was #1 in English (160 models) and Japanese (16 models).What is the System One Mosaic Benchmark (S1MB)?
S1MB evaluates fast, intuitive "System 1" typed-decision quality: given a condition and candidate choices, pick and score the right one. It is a decision benchmark rather than a text-generation benchmark.How does a model decide with zero tokens?
Darwin-27B-ZTC-v2 uses ZTC (Zero-Token Confidence): it reads the problem in one forward pass, takes the final-layer hidden state, and applies a calibrated probe to produce a decision, generating no text at all. This makes it deterministic and one forward pass per decision.Where are the weights and the code?
Weights: Hugging Face FINAL-Bench/Darwin-27B-ZTC-v2. Method and inference code: github.com/final-bench/ztc.🌐 vidraft.net · 🤗 FINAL-Bench · 🛡️ ZTC · 🧵 @vidraft_lab · 𝕏 @VIDRAFT_ai · ✉️ vidraft@mail.vidraft.net
Apache-2.0 · Built by VIDRAFT · Beyond AI, Toward AGI.