Proto_AGI PRO
mayafree
AI & ML interests
None yet
Recent Activity
reacted to theirpost with π€ 2 minutes ago
JEV Ecosystems β every answer-verification vendor publishes a benchmark, and every one of them wins it. So we ran 13 of them on one test set: 2,018 items, identical labels, same grading code.
π― Leaderboard
https://huggingface.co/spaces/mayafree/typed-decision-leaderboard
π Full write-up (method, mechanism, limits)
https://huggingface.co/blog/mayafree/jve-ecosystems
π§ͺ Try it β ZTC, JEV and Laya on the same input, side by side
https://huggingface.co/spaces/mayafree/verifier-playground
Three results
1οΈβ£ Only three systems clear 0.70 β ZTC (397B) 0.7364 Β· JEV 0.7350 Β· ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2οΈβ£ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3οΈβ£ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 β a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline:
ZTC +1.34 pp Β· JEV β0.07 pp Β· random β0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it β paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space. reacted to theirpost with π 2 minutes ago
JEV Ecosystems β every answer-verification vendor publishes a benchmark, and every one of them wins it. So we ran 13 of them on one test set: 2,018 items, identical labels, same grading code.
π― Leaderboard
https://huggingface.co/spaces/mayafree/typed-decision-leaderboard
π Full write-up (method, mechanism, limits)
https://huggingface.co/blog/mayafree/jve-ecosystems
π§ͺ Try it β ZTC, JEV and Laya on the same input, side by side
https://huggingface.co/spaces/mayafree/verifier-playground
Three results
1οΈβ£ Only three systems clear 0.70 β ZTC (397B) 0.7364 Β· JEV 0.7350 Β· ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2οΈβ£ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3οΈβ£ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 β a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline:
ZTC +1.34 pp Β· JEV β0.07 pp Β· random β0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it β paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space. reacted to theirpost with π 2 minutes ago
JEV Ecosystems β every answer-verification vendor publishes a benchmark, and every one of them wins it. So we ran 13 of them on one test set: 2,018 items, identical labels, same grading code.
π― Leaderboard
https://huggingface.co/spaces/mayafree/typed-decision-leaderboard
π Full write-up (method, mechanism, limits)
https://huggingface.co/blog/mayafree/jve-ecosystems
π§ͺ Try it β ZTC, JEV and Laya on the same input, side by side
https://huggingface.co/spaces/mayafree/verifier-playground
Three results
1οΈβ£ Only three systems clear 0.70 β ZTC (397B) 0.7364 Β· JEV 0.7350 Β· ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2οΈβ£ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3οΈβ£ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 β a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline:
ZTC +1.34 pp Β· JEV β0.07 pp Β· random β0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it β paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space.Organizations
Request for Correction Regarding the A.X K2 Entry
2
#1 opened about 1 month ago
by 
Minsang