Instructions to use PugarHuda/takar-outcome-evaluator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use PugarHuda/takar-outcome-evaluator with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Takar Outcome Evaluator
A reward-model adapter for the Telegraph network's EVENT_OUTCOME_RESOLUTION
family. It is a rank-8 LoRA plus a one-logit reward head on the shared
Foundation Model modernbert-base-lora16-v1 (ModernBERT-base), packaged in
Telegraph's lora-v1 safetensors delta format.
It scores how well a miner's answer resolves the event that was asked. It ranks last the answers that have nothing to do with the question, which the network's default judging did not catch.
| Package | EVENT_OUTCOME_RESOLUTION.safetensors, 6,776,916 bytes |
| keccak256 (this file, v2) | 0xfeac3f314909f8a881f67226552445156d347df6682f4f3743eec60e50eedd3e |
| Network status | rejected (registration 156, v2.1 = same weights, re-hashed). See below |
| Class | hybrid (the answer is the miner's whole response body) |
| Input | "### Task:\n" + question + "\n\n### Answer:\n" + answer, node tokenizer, β€512 tokens |
| Output | one reward logit; higher is better |
Network result: rejected
On 20 Nov 2026 the node's remote check sent 15 public pairs to our executor. The adapter put 7 in the right order (46.7%, floor 70%), so it was rejected. The 99.8% below was measured on pairs we generated; it did not transfer to the network's material. We did not capture or train on the node's pairs. Treat this adapter as a study of five failure modes, not as a working evaluator.
What it catches
Real answers from the Telegraph question feed, scored on the network's own WASM engine against an on-topic answer to the same question:
| Question | Off-topic answer a ranked miner returned | Its score | On-topic |
|---|---|---|---|
| Will Heidelberg Engineering ANTERION study complete? | Bundesliga goal list | β3.73 | β2.45 |
| Will Phrack 74 be released by 2027? | NFL Polymarket listing | β6.48 | β2.18 |
| Will Phrack 74 be released by July 2027? | F1 Bahrain race data | β9.46 | β0.93 |
Training
The pairs are built from real Manifold markets (3,697 resolved, 1,865 open) and 4,000 real off-topic payloads from the network feed. There are five kinds:
- topic: an answer to a different event
- consistent: an outcome that contradicts its own evidence
- premature: a confident answer to an open market
- settled: dodging a closed market
- junk: a misrouted payload
Plain YES-versus-NO flips are left out, because a 150M encoder cannot know who won.
| Pairs (train / val / test) | 4,454 / 542 / 537, split by market |
| Dataset gate (Telegraph kit) | pass. Bag-of-words probe 62.2%, length shortcut 50.2% |
| Test pairwise accuracy | 99.81% (length baseline 50.8%) |
| Real engine, 40 held-out pairs | 40 / 40 |
The test pairs come from the same generator as the training pairs, so 99.8% measures those five failure modes. It does not predict the network's hidden pairs.
Use
Scoring needs Telegraph's foundation and engine, because validators compare scores bit for bit:
git clone https://github.com/telegraphprotocol/telegraph-wasm-baseline
# foundation_lora_fp32.onnx + foundation_engine.wasm from
# github.com/telegraphprotocol/telegraph-foundation/releases/tag/modernbert-base-lora16-v1
go run ./cmd/adapterexec --package EVENT_OUTCOME_RESOLUTION.safetensors
Source, data, training run and every check: https://github.com/PugarHuda/takar/tree/main/evaluator
- Downloads last month
- -
Model tree for PugarHuda/takar-outcome-evaluator
Base model
answerdotai/ModernBERT-base