Instructions to use Heliusnetwork/zeus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Heliusnetwork/zeus with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2") model = PeftModel.from_pretrained(base_model, "Heliusnetwork/zeus") - Notebooks
- Google Colab
- Kaggle
Zeus v22: open claim verification
Zeus reads a claim and a few passages and says whether the passages support it, contradict it or do not cover it. It quotes the exact words that carry the evidence. It is the checker behind Olo, which tests every sentence of an answer against sources from the open Olo Commons. It is built in the open: the data pipeline, training code and every measured score are in Hashim57/olo-commons under zeus/.
Measured scores
Measured on 2026-11-16 for this exact version by zeus/eval.py. Nothing here is estimated. The held-out sets come first because the frozen first split has been at or near 100 percent for weeks and no longer shows progress.
| Set | Label accuracy |
|---|---|
| VitaminC dev (300 human-written claims, held out) | 73.3% |
| Commons eval v3 (frozen, hard patterns, n=200) | 92.0% |
| Commons eval v2 (frozen, harder split, n=200) | 100.0% |
| Commons eval (frozen first split, n=200) | 99.5% |
Evidence hit on the frozen split (quoted span found in the right passage): 99.5%. JSON valid: 100.0%.
Training
- Base model:
mistralai/Mistral-7B-Instruct-v0.2(Apache 2.0). - Method: LoRA rank 8 on q_proj, v_proj, learning rate 0.0002, batch 16, 1034 optimiser steps in 60.0 minutes on one A100-40GB.
- Training examples: 36,255.
- Trained by the public GitHub workflow
zeus-train.ymlon a cloud GPU; the full log of every version iszeus/STATUS.md.
Data and licences
Three parts, described in zeus/DATA_CARD.md: examples made by rule from Olo Commons atoms (each atom keeps its own source licence), hand-written gold examples checked against the atoms they quote, and human-written claim and evidence pairs from VitaminC (Schuster, Fisch and Barzilay, NAACL 2021, CC BY-SA 3.0). Text from Wikipedia and VitaminC is CC BY-SA: attribute Wikipedia and the VitaminC authors. The adapter is released under Apache 2.0, like its base model.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "Heliusnetwork/zeus")
tok = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
Send one user turn with the system prompt from zeus/cloud/recipe.py followed by:
Claim: <claim>
Passages:
[p1] (<source>) <text>
[p2] (<source>) <text>
The answer is one JSON object: {"label": "supported" | "contradicted" | "not_covered", "evidence": [{"passage": "<id>", "quote": "<exact words>"}], "rationale": "<one sentence>"}.
Limits
Most training contradictions are made by rule (one changed relation, year, number or name), so subtle contradictions are harder for it than clear ones; the weakest pattern in each version is reported in the weekly learning report. It checks claims against the passages it is given and knows nothing beyond them. It is not a substitute for reading the source: every Olo answer carries its passages so a person can.
Card generated by zeus/cloud/publish_hf.py on 2026-11-17 from zeus/out/zeus-v22/.
- Downloads last month
- 24
Model tree for Heliusnetwork/zeus
Base model
mistralai/Mistral-7B-Instruct-v0.2