GyrTechnology/saif-0.8b-lightning

Saif 0.8B Lightning (v0.3)

A small, fast decision model. Send it a situation and a set of typed questions in one request; it returns a probability for every answer. It does no text generation and no chat: it only makes decisions.

Saif 0.8B Lightning speaks the System One / Jev typed-decision format, so an existing Jev integration can switch to it by changing the endpoint and model name.

• Decision Index 0.3, public suite: 47.75 (raw 60.73), all 140,178 requests answered, none unsupported. v0.1 scored 42.79.
• Fast: 18 ms median, 54 ms mean, 30 ms p80 per request, one request at a time.
• Small: 0.8B parameters, about 1.5 GB in bf16. Runs on any recent GPU, and on CPU.
• Decision-only by design: no text-generation head is shipped.

Earlier versions stay available: v0.1 is tagged in this repository.

What it does

Every request has a state (any JSON: a message, a tool catalogue, a document, a conversation) and a set of questions in three types:

Type	Asks	Returns
noul	a yes/no question	the probability of yes
choice	pick one option from a set you define	the chosen option and a probability for every option
score	rate on an ordered scale	the expected level and a probability for every level

Typical uses: routing and intent, tool selection, guardrails, classification and triage, claim checking and ranking.

Results

Decision Index 0.3, public suite (37 benchmarks), run with the official Decision Index kit on one RTX 5070 Ti, one pass per request. Scores are chance-corrected skill (0 is random guessing, 100 is perfect).

Area	v0.3
Overall index	47.75
Tools & Automation	68.1
Retrieval & Classification	64.1
Arts & Human Taste	47.1
Language Understanding	43.9
Knowledge & Reasoning	24.8

Strongest benchmarks: BFCL 90.3, CLINC150+OOS 90.2, When2Call MCQ 85.6, PhishNChips phishing decisions 85.4, BANKING77 83.9, FinEntity 78.7, POP909-CL 75.6, BPoMP 72.8, Home appliance simulator 69.3, HellaSwag 67.5.

Weakest, as you would expect at this size: HLE, CRUXEval, ACOS, ChessBench, GPQA Diamond, iSarcasmEval.

Quick start

import sys
sys.path.insert(0, '/path/to/saif-0.8b-lightning')
from decision_index_engine import SaifEngine   # needs the decision-index kit on PYTHONPATH

engine = SaifEngine(model='/path/to/saif-0.8b-lightning', device='cuda')   # or device='cpu'
result, _ = engine('The parcel arrived broken and I want my money back today.', {
    'sentiment': {'type': 'choice', 'criteria': {'positive': 'happy', 'negative': 'unhappy'}},
    'refund': {'type': 'noul', 'instructions': 'Does the customer ask for a refund?'},
    'urgency': {'type': 'score', 'criteria': ['not urgent', 'somewhat urgent', 'very urgent']},
})
print(result['answers'])

To reproduce the public run:

git clone https://huggingface.co/GyrTechnology/saif-0.8b-lightning
PYTHONPATH=saif-0.8b-lightning:<decision-index kit> python -m decision_index pipeline --edition 0.3 \
  --suite-dir <kit>/suite-0.3 --engine decision_index_engine:SaifEngine \
  --model saif-0.8b-lightning --out saif-0.8b-lightning-run

Requirements: Python 3.12, torch 2.8, transformers with Qwen3.5 support, safetensors, accelerate. No PEFT, Unsloth or base-model download is needed.

How it works

The Qwen3.5-0.8B decoder reads the state once and every question in its own segment that attends to the shared state (bidirectional, single pass). Each option is scored from a small set of label rows, so all options of all questions come out of one forward pass. Inputs are never truncated.

Training

Full fine-tuning of Qwen/Qwen3.5-0.8B (revision 2fc06364) on typed decisions drawn only from public training splits (tool calling and tool retrieval, intent, NLI, claim verification, retrieval relevance, reading comprehension, sentiment, stance and sarcasm, phishing, causal reasoning, music and text style, chess) plus programmatically generated decisions. Constant learning rate in rounds with a fixed per-benchmark coverage mix, followed by a 250-step cosine cooldown on the same effective mix; output temperatures are fitted on a held-out split of the training data. No Decision Index item is used for training or selection.

License

The weights are released under CC-BY-NC-4.0 (non-commercial), because some training sources are licensed for non-commercial or research use only (for example ANLI, RACE, VAST, NLI4CT, ACOS and MS MARCO). Base model: Qwen/Qwen3.5-0.8B, Apache-2.0 (LICENSE-QWEN).