ZTC Gate Tetris
๐งฑ
38
One gate call: 2.1 s vs 0.0615 s. Watch it play out.
ginigen-ai PRO
ginigen-ai
AI & ML interests
None yet
Recent Activity
liked a Space about 14 hours ago
FINAL-Bench/Tetris-JEV-LAYA-ZTC reacted to SeaWolf-AI's post with ๐ about 14 hours ago
The cost of a judging gate is usually quoted as a number. This puts it on a Tetris board.
Three boards get the same piece order, and on every move the same proposal and the same noise โ a paired comparison. The gate decides one thing: keep this move, or draw again. Each board gets the same 60 seconds of gate time.
The text-writing gates get through 15โ22 moves. The generation-free gate gets through 40โ50. The boards that stop simply run out of clock.
It does not win on accuracy: on the same 2,018-question LODO set, JEV scores AUC 0.7350 against ZTC-Judge-27B's 0.7289. The separation is elsewhere. Clock โ 2.1 s vs 0.0615 s per call, and on a 200-candidate agent screen one judging call measured 3.206 s generative vs 0.033 s readout, same server. Calibration โ a gate is a threshold, and at ECE 0.4985 (vs ZTC 0.0245) a threshold stops carrying information. Mechanism โ a text judge can name option 42 when there is no option 42; a scoring readout cannot. Not a lower error rate. No path.
The curve in the ZTC panel is real online fitting, scored prequentially โ predict first, learn after โ with base weights untouched. Not recursive self-improvement.
Limits, also stated on the page: Laya's AUC and latency are not our measurements and are set equal to JEV's, so calibration is the only measured axis it differs on. The page is a simulation driven by measured constants.
KO / EN / ZH.
https://huggingface.co/spaces/FINAL-Bench/Tetris-JEV-LAYA-ZTC
https://huggingface.co/FINAL-Bench/ZTC-Judge-27B upvoted an article 3 days ago
Your model already knows it's wrong. Asking costs 0.06 seconds and zero tokens.Organizations
liked a
Space about 14 hours ago
reacted to SeaWolf-AI's post with ๐ about 14 hours ago
Post
1497
The cost of a judging gate is usually quoted as a number. This puts it on a Tetris board.
Three boards get the same piece order, and on every move the same proposal and the same noise โ a paired comparison. The gate decides one thing: keep this move, or draw again. Each board gets the same 60 seconds of gate time.
The text-writing gates get through 15โ22 moves. The generation-free gate gets through 40โ50. The boards that stop simply run out of clock.
It does not win on accuracy: on the same 2,018-question LODO set, JEV scores AUC 0.7350 against ZTC-Judge-27B's 0.7289. The separation is elsewhere. Clock โ 2.1 s vs 0.0615 s per call, and on a 200-candidate agent screen one judging call measured 3.206 s generative vs 0.033 s readout, same server. Calibration โ a gate is a threshold, and at ECE 0.4985 (vs ZTC 0.0245) a threshold stops carrying information. Mechanism โ a text judge can name option 42 when there is no option 42; a scoring readout cannot. Not a lower error rate. No path.
The curve in the ZTC panel is real online fitting, scored prequentially โ predict first, learn after โ with base weights untouched. Not recursive self-improvement.
Limits, also stated on the page: Laya's AUC and latency are not our measurements and are set equal to JEV's, so calibration is the only measured axis it differs on. The page is a simulation driven by measured constants.
KO / EN / ZH.
FINAL-Bench/Tetris-JEV-LAYA-ZTC
FINAL-Bench/ZTC-Judge-27B
Three boards get the same piece order, and on every move the same proposal and the same noise โ a paired comparison. The gate decides one thing: keep this move, or draw again. Each board gets the same 60 seconds of gate time.
The text-writing gates get through 15โ22 moves. The generation-free gate gets through 40โ50. The boards that stop simply run out of clock.
It does not win on accuracy: on the same 2,018-question LODO set, JEV scores AUC 0.7350 against ZTC-Judge-27B's 0.7289. The separation is elsewhere. Clock โ 2.1 s vs 0.0615 s per call, and on a 200-candidate agent screen one judging call measured 3.206 s generative vs 0.033 s readout, same server. Calibration โ a gate is a threshold, and at ECE 0.4985 (vs ZTC 0.0245) a threshold stops carrying information. Mechanism โ a text judge can name option 42 when there is no option 42; a scoring readout cannot. Not a lower error rate. No path.
The curve in the ZTC panel is real online fitting, scored prequentially โ predict first, learn after โ with base weights untouched. Not recursive self-improvement.
Limits, also stated on the page: Laya's AUC and latency are not our measurements and are set equal to JEV's, so calibration is the only measured axis it differs on. The page is a simulation driven by measured constants.
KO / EN / ZH.
FINAL-Bench/Tetris-JEV-LAYA-ZTC
FINAL-Bench/ZTC-Judge-27B
- 1 reply
upvoted an article 3 days ago
Article
Your model already knows it's wrong. Asking costs 0.06 seconds and zero tokens.
FINAL-Bench
โข โข 12 liked 2
models 3 days ago
reacted to SeaWolf-AI's post with ๐ฅ 3 days ago
Post
3132
Ask a language model how confident it is and you get an AUC of 0.5000. Exactly a coin flip. We measured it across 2,018 items.
FINAL-Bench/gate-tetris
https://huggingface.co/blog/FINAL-Bench/ztc
Collection: https://huggingface.co/collections/FINAL-Bench/ztc-models-jev-ecosystems
Zero-Token Confidence (ZTC) reads it. One forward pass over the model's hidden state returns a calibrated probability that the answer is correct. Zero generated tokens.
It sits at the top of the shared board. Same 2,018 items, same harness for every entry: ZTC on Darwin-397B 0.7394, JEV 0.7335, ZTC-Judge-27B 0.7255, a surface baseline that reads only answer length and formatting 0.7036, Lynx 8B 0.5157, the model's own self-reported confidence 0.5000, HHEM 0.4852. First and third place both emit nothing at all.
The number worth staring at is 0.7036. That is a baseline reading no content whatsoever, just how long the answer is and how it is formatted. Any verifier scoring below it is not reading content either.
On speed, one gate call costs 0.0615 seconds, measured on four B200s across 2,000 items. Generating a single candidate answer takes 1.631 seconds, so the gate is 26 times cheaper than the work it guards. A verifier that generates competes with your agent for the same budget. A verifier that only reads can be attached to every action instead of a sampled few.
We built it so you can watch it decide. Three lanes receive the same stream of proposed actions and the same time budget. One has no gate and must execute everything. One uses a text-reading verifier. One uses ZTC. Right action plus one, wrong action minus one, hold zero. Over 400 matches: no gate minus 3.9, text verifier plus 13.0, ZTC plus 29.1, with ZTC taking 98 percent of matches. Gating lifts executed accuracy from 49 percent to 65 percent.
FINAL-Bench/gate-tetris
https://huggingface.co/blog/FINAL-Bench/ztc
Collection: https://huggingface.co/collections/FINAL-Bench/ztc-models-jev-ecosystems
Zero-Token Confidence (ZTC) reads it. One forward pass over the model's hidden state returns a calibrated probability that the answer is correct. Zero generated tokens.
It sits at the top of the shared board. Same 2,018 items, same harness for every entry: ZTC on Darwin-397B 0.7394, JEV 0.7335, ZTC-Judge-27B 0.7255, a surface baseline that reads only answer length and formatting 0.7036, Lynx 8B 0.5157, the model's own self-reported confidence 0.5000, HHEM 0.4852. First and third place both emit nothing at all.
The number worth staring at is 0.7036. That is a baseline reading no content whatsoever, just how long the answer is and how it is formatted. Any verifier scoring below it is not reading content either.
On speed, one gate call costs 0.0615 seconds, measured on four B200s across 2,000 items. Generating a single candidate answer takes 1.631 seconds, so the gate is 26 times cheaper than the work it guards. A verifier that generates competes with your agent for the same budget. A verifier that only reads can be attached to every action instead of a sampled few.
We built it so you can watch it decide. Three lanes receive the same stream of proposed actions and the same time budget. One has no gate and must execute everything. One uses a text-reading verifier. One uses ZTC. Right action plus one, wrong action minus one, hold zero. Over 400 matches: no gate minus 3.9, text verifier plus 13.0, ZTC plus 29.1, with ZTC taking 98 percent of matches. Gating lifts executed accuracy from 49 percent to 65 percent.
liked a
Space 3 days ago
liked a
model 3 days ago
upvoted a collection 3 days ago
upvoted an article 3 days ago
Article
Inside the JEV Ecosystem: 13 Answer Verifiers on One Test Set
mayafree
โข โข 12 liked 2
Spaces 3 days ago
reacted to mayafree's post with ๐ค๐๐๐ฅ 3 days ago
Post
3443
JEV Ecosystems โ every answer-verification vendor publishes a benchmark, and every one of them wins it. So we ran 13 of them on one test set: 2,018 items, identical labels, same grading code.
๐ฏ Leaderboard
mayafree/typed-decision-leaderboard
๐ Full write-up (method, mechanism, limits)
https://huggingface.co/blog/mayafree/jve-ecosystems
๐งช Try it โ ZTC, JEV and Laya on the same input, side by side
mayafree/verifier-playground
Three results
1๏ธโฃ Only three systems clear 0.70 โ ZTC (397B) 0.7364 ยท JEV 0.7350 ยท ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2๏ธโฃ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3๏ธโฃ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 โ a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline:
ZTC +1.34 pp ยท JEV โ0.07 pp ยท random โ0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it โ paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space.
๐ฏ Leaderboard
mayafree/typed-decision-leaderboard
๐ Full write-up (method, mechanism, limits)
https://huggingface.co/blog/mayafree/jve-ecosystems
๐งช Try it โ ZTC, JEV and Laya on the same input, side by side
mayafree/verifier-playground
Three results
1๏ธโฃ Only three systems clear 0.70 โ ZTC (397B) 0.7364 ยท JEV 0.7350 ยท ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2๏ธโฃ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3๏ธโฃ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 โ a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline:
ZTC +1.34 pp ยท JEV โ0.07 pp ยท random โ0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it โ paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space.
upvoted a paper 4 days ago
reacted to their post with ๐ค๐โค๏ธ 4 days ago
Post
4463
A local edge VLM you can run on a phone โ with a calibration readout attached.
ginigen-ai/Edge-4B-TELL
Image in, answer out, nothing leaving the device. Google's Gemma 4 E4B QAT checkpoint carried unmodified, with the vision and audio projector, plus one thing that is ours: GINIGEN TELL, a 10 KB readout that estimates whether the answer it just gave is likely to be wrong.
On a Galaxy S25: zero network calls, 3.6 GB resident, a 12.6 MB inference binary.
Calibration matters more here than on a server: nothing downstream catches a bad answer. No retrieval, no second opinion, no reviewer. The model is alone with the user.
And its own confidence is unusable. Prompted for it, this checkpoint averages 0.863 over 665 Korean disaster-procedure questions โ ranking answers by it gives AUROC 0.441, below a coin flip. It sounds more certain when it is wrong.
TELL reads the last-layer hidden state instead of asking. Same questions, 0.759. Surface cues (length, formatting) already reach 0.736, so the readout clears that baseline by +0.023 ยฑ 0.009 (2.6ฯ). We publish the baseline because without it, "the hidden state carries the signal" is unfalsifiable.
Same job as JEV: a confidence number you can act on instead of the model's own. Different structure, and on a device that splits three ways.
No second model โ JEV is a separate judge reading the answer as text; we fill that slot with a 10 KB vector.
Zero generated tokens โ a judge writes its verdict, TELL re-reads a finished computation (3.8 s on an S25).
No network โ a verdict fetched over an API stops when the signal does.
The trade is real: a readout is fitted per checkpoint, so on a server the judge wins. On a phone there is no second model to run.
TELL never says what the right answer is. It says whether the answer wobbled, and a low score falls back to source text bundled with the app. Shipping today in HeliGO, an offline disaster-response app.
ginigen-ai/Edge-4B-TELL
Image in, answer out, nothing leaving the device. Google's Gemma 4 E4B QAT checkpoint carried unmodified, with the vision and audio projector, plus one thing that is ours: GINIGEN TELL, a 10 KB readout that estimates whether the answer it just gave is likely to be wrong.
On a Galaxy S25: zero network calls, 3.6 GB resident, a 12.6 MB inference binary.
Calibration matters more here than on a server: nothing downstream catches a bad answer. No retrieval, no second opinion, no reviewer. The model is alone with the user.
And its own confidence is unusable. Prompted for it, this checkpoint averages 0.863 over 665 Korean disaster-procedure questions โ ranking answers by it gives AUROC 0.441, below a coin flip. It sounds more certain when it is wrong.
TELL reads the last-layer hidden state instead of asking. Same questions, 0.759. Surface cues (length, formatting) already reach 0.736, so the readout clears that baseline by +0.023 ยฑ 0.009 (2.6ฯ). We publish the baseline because without it, "the hidden state carries the signal" is unfalsifiable.
Same job as JEV: a confidence number you can act on instead of the model's own. Different structure, and on a device that splits three ways.
No second model โ JEV is a separate judge reading the answer as text; we fill that slot with a 10 KB vector.
Zero generated tokens โ a judge writes its verdict, TELL re-reads a finished computation (3.8 s on an S25).
No network โ a verdict fetched over an API stops when the signal does.
The trade is real: a readout is fitted per checkpoint, so on a server the judge wins. On a phone there is no second model to run.
TELL never says what the right answer is. It says whether the answer wobbled, and a low score falls back to source text bundled with the app. Shipping today in HeliGO, an offline disaster-response app.