Julia 1.5
From context to decisions. Fast, small, and now far more reliable.
Julia 1.5 is a decision model. Give it a state (JSON or free text), a typed question, and 2โ20 candidate options; in one forward pass it returns a calibrated probability for every option. No text generation, no reasoning loop, no prompt engineering. Just a decision, in milliseconds, on a CPU or a GPU.
144M parameters ยท 1,024-token context ยท English, Portuguese and 1,800+ languages in the vocabulary ยท Julia 1 API
What's new
This release keeps the size, speed and interface of Julia 1, and fixes the problems teams hit in production:
| Known issue in Julia 1 / early 1.5 | Julia 1.5 now |
|---|---|
| Ignored written business rules and guessed from surface patterns | Follows written rules 87% of the time (Julia 1: 30%) |
| Read "is it NOT eligible?" as "is it eligible?" | Handles negation: 94% (Julia 1: 28%) |
| Answer changed when the same options were listed in another order | Order-stable on 87% of questions (Julia 1: 59%), and fully stable with views in the runtime |
| Could be pulled toward an unrelated option slipped into the list | Ignores irrelevant options 99% of the time (Julia 1: 70%) |
| Strong on yes/no tasks, weak on everything else | Better on every SLCMB question family: choice, yes/no and score, in English and Portuguese |
| Weak outside its training mix | 73% on public tests in 5 languages (Julia 1: 67%) |
On SLCMB, the Supersonic Labs decision benchmark (920 cases ร 100 option orders, 16 areas), Julia 1.5 scores
46.2, versus 33.3 for Julia 1 at the same size (46.9 with views=3 in the runtime).
SLCMB by track
| Track | Julia 1 | Julia 1.5 |
|---|---|---|
| en ยท core choice | 25.2 | 38.8 |
| en ยท core yes/no | 41.3 | 45.0 |
| en ยท core score | 35.9 | 36.7 |
| pt ยท core choice | 20.5 | 42.8 |
| pt ยท core yes/no | 36.6 | 40.8 |
| pt ยท core score | 29.7 | 40.8 |
| AG News | 77.4 | 85.5 |
| Banking77 | 26.4 | 45.5 |
| BRIGHTER (en) | 38.8 | 58.3 |
| BRIGHTER (pt) | 41.6 | 56.3 |
| NASA RSS | 56.9 | 71.8 |
| Agรชncia Brasil RSS | 58.9 | 78.8 |
| SpamAssassin | 45.8 | 82.6 |
| Typed decisions | 34.1 | 39.9 |
| Emotion | 91.6 | 66.0 |
| ARC | 19.3 | 24.8 |
| MMLU-Pro | 10.8 | 9.2 |
| ENEM 2024 | 12.2 | 7.6 |
Julia 1 still leads on Emotion; exam-style knowledge (MMLU-Pro, ENEM) is out of scope at this size.
How it works
- One sequence. The question and every option go up front, each option behind a
<mask>marker; the state fills the rest of the 1,024 tokens. - Encoder. A 22-layer mmBERT-small encoder reads everything once. Most layers look at ยฑ64 neighbouring tokens; every third layer sees the whole sequence, so each option sees the question, the other options and the state.
- Decision head. A question-type embedding (choice / score / noul) and two attention layers compare the options; a
small scorer turns each
<mask>into one number. - Answer. Softmax over the options gives the probabilities and a typed answer: winning key, expected score level, or P(true).
Start here
Python 3.10+, PyTorch (CPU or CUDA). No transformers needed.
pip install huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('SupersonicLabs/Julia-1.5', local_dir='Julia-1.5')"
pip install -e ./Julia-1.5/runtime
from julia15 import Runtime
rt = Runtime('Julia-1.5') # load once, keep it alive (CUDA if available, else CPU)
result = rt.predict_questions(
state={'player_hp': 2, 'boss_hp': 50, 'potions': 1},
questions={
'move': {
'type': 'choice',
'instructions': 'What is the best next move?',
'criteria': {'attack': 'Attack the boss', 'heal': 'Drink the potion', 'flee': 'Run away'},
},
'danger': {'type': 'noul', 'instructions': 'Is the player about to die?'},
'risk': {'type': 'score', 'instructions': 'Risk level', 'criteria': ['low', 'medium', 'high']},
},
views=3, # optional: order-robust choice answers
)
print(result['answers']['move']['choice'], result['answers']['move']['probabilities'])
rt.decide('Card charged twice this month', 'Which team?', {'billing': 'Billing', 'tech': 'Tech support'})
Question types
| Type | criteria |
Answer |
|---|---|---|
choice |
map of 2โ20 IDs โ descriptions | choice: winning ID |
score |
ordered list of 2โ20 rubric levels | score: expected zero-based level, level: most likely |
noul |
optional {"false": ..., "true": ...} |
noul: probability of true |
probabilities are full softmax values keyed by your IDs. Answers carry truncated: true when the state did not fit.
HTTP server
python -m julia15 serve --port 8080 --views 2
curl -s localhost:8080/v1/predict -d '{
"state": "Customer: I was charged twice for my order #8812.",
"questions": {"refund": {"type": "noul", "instructions": "Should we refund the duplicate charge?"}}}'
| Method | Path | |
|---|---|---|
| POST | /v1/predict |
{state, questions, views?} โ {model, answers} |
| POST | /v1/predict_batch |
many {state, questions} in one batched call |
| POST | /v1/logits |
raw option scores for {state, question, type, options} rows |
| GET | /v1/info ยท /health ยท /metrics |
config, checkpoint fingerprint, Prometheus counters |
Same request and answer format as the Julia 1 runtime. Concurrent requests are batched together automatically.
Runtime details: runtime/README.md.
Speed
| Hardware | Throughput | One request |
|---|---|---|
| NVIDIA L40S, fp16 encoder | ~3,700 rows/s | ~15 ms |
| NVIDIA L40S, fp32 | ~1,470 rows/s | ~15 ms |
| 8-vCPU cloud VM, fp32 | ~5 rows/s | ~250 ms (325 tokens) |
Repeated requests hit an in-memory cache (< 1 ms). On GPU the decision head stays fp32, so half precision changes fewer than 0.4% of answers.
Files
| Path | Contents |
|---|---|
model.safetensors |
weights (FP32, 577 MB) |
julia_config.json, encoder/, tokenizer/ |
architecture and tokenizer, read by the runtime |
runtime/ |
julia15 runtime: package, tests, tools/bench.py |
metrics/, provenance.json |
evaluation results and checkpoint provenance |
config.json |
model manifest (architecture, sizes, checkpoint hash; also used by the Hub to count downloads) |
The model artifacts are licensed under Apache 2.0. The training pipeline is not included.
Limits of the architecture
Julia 1.5 is a small encoder that compares options. That is what makes it fast, and it also sets its limits:
- It chooses, it does not compute. Long arithmetic, multi-step math and close numeric comparisons ("is 8,143 more than 8,134?") are unreliable; compute those upstream and pass the result in the state.
- It does not know what you did not tell it. World knowledge at exam level (MMLU-Pro, ENEM) is weak at 144M parameters; put the needed facts in the state.
- 1,024 tokens per question. Longer states are truncated (flagged with
truncated: true);--strictrejects them. - 2โ20 options, each up to 48 tokens; question plus options up to 512 tokens.
- Choice and yes/no views can disagree: asking "which option?" and asking "is option X right?" for each option do not always agree. Use the question type that matches your decision.
- Score levels: answers on long rubrics are less consistent than choices; prefer a short rubric.
- Calibration is good on average but not perfect on data unlike its training mix; evaluate on your own questions before relying on probabilities as thresholds.
- Not a Transformers
AutoModel: use the runtime inruntime/.
Supersonic Labs
- Downloads last month
- 41
Model tree for SupersonicLabs/Julia-1.5
Base model
SupersonicLabs/Julia-1Collection including SupersonicLabs/Julia-1.5
Evaluation results
- SLCMB score on SLCMB 3.0 (920 cases x 100 option orders)self-reported46.200
- Accuracy on Emotion, AG News, Banking77, Enron spam, MASSIVE pt/es/de, tweets ptself-reported0.730


