Supersonic Labs logo

Julia 1.5

From context to decisions. Fast, small, and now far more reliable.

Julia 1.5 โ€” monochrome geometric artwork

Julia 1.5 is a decision model. Give it a state (JSON or free text), a typed question, and 2โ€“20 candidate options; in one forward pass it returns a calibrated probability for every option. No text generation, no reasoning loop, no prompt engineering. Just a decision, in milliseconds, on a CPU or a GPU.

144M parameters ยท 1,024-token context ยท English, Portuguese and 1,800+ languages in the vocabulary ยท Julia 1 API

Julia 1.5 vs Julia 1

What's new

This release keeps the size, speed and interface of Julia 1, and fixes the problems teams hit in production:

Known issue in Julia 1 / early 1.5 Julia 1.5 now
Ignored written business rules and guessed from surface patterns Follows written rules 87% of the time (Julia 1: 30%)
Read "is it NOT eligible?" as "is it eligible?" Handles negation: 94% (Julia 1: 28%)
Answer changed when the same options were listed in another order Order-stable on 87% of questions (Julia 1: 59%), and fully stable with views in the runtime
Could be pulled toward an unrelated option slipped into the list Ignores irrelevant options 99% of the time (Julia 1: 70%)
Strong on yes/no tasks, weak on everything else Better on every SLCMB question family: choice, yes/no and score, in English and Portuguese
Weak outside its training mix 73% on public tests in 5 languages (Julia 1: 67%)

On SLCMB, the Supersonic Labs decision benchmark (920 cases ร— 100 option orders, 16 areas), Julia 1.5 scores 46.2, versus 33.3 for Julia 1 at the same size (46.9 with views=3 in the runtime).

SLCMB by track
Track Julia 1 Julia 1.5
en ยท core choice 25.2 38.8
en ยท core yes/no 41.3 45.0
en ยท core score 35.9 36.7
pt ยท core choice 20.5 42.8
pt ยท core yes/no 36.6 40.8
pt ยท core score 29.7 40.8
AG News 77.4 85.5
Banking77 26.4 45.5
BRIGHTER (en) 38.8 58.3
BRIGHTER (pt) 41.6 56.3
NASA RSS 56.9 71.8
Agรชncia Brasil RSS 58.9 78.8
SpamAssassin 45.8 82.6
Typed decisions 34.1 39.9
Emotion 91.6 66.0
ARC 19.3 24.8
MMLU-Pro 10.8 9.2
ENEM 2024 12.2 7.6

Julia 1 still leads on Emotion; exam-style knowledge (MMLU-Pro, ENEM) is out of scope at this size.

How it works

How Julia 1.5 decides

  1. One sequence. The question and every option go up front, each option behind a <mask> marker; the state fills the rest of the 1,024 tokens.
  2. Encoder. A 22-layer mmBERT-small encoder reads everything once. Most layers look at ยฑ64 neighbouring tokens; every third layer sees the whole sequence, so each option sees the question, the other options and the state.
  3. Decision head. A question-type embedding (choice / score / noul) and two attention layers compare the options; a small scorer turns each <mask> into one number.
  4. Answer. Softmax over the options gives the probabilities and a typed answer: winning key, expected score level, or P(true).

Start here

Python 3.10+, PyTorch (CPU or CUDA). No transformers needed.

pip install huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('SupersonicLabs/Julia-1.5', local_dir='Julia-1.5')"
pip install -e ./Julia-1.5/runtime
from julia15 import Runtime

rt = Runtime('Julia-1.5')            # load once, keep it alive (CUDA if available, else CPU)

result = rt.predict_questions(
    state={'player_hp': 2, 'boss_hp': 50, 'potions': 1},
    questions={
        'move': {
            'type': 'choice',
            'instructions': 'What is the best next move?',
            'criteria': {'attack': 'Attack the boss', 'heal': 'Drink the potion', 'flee': 'Run away'},
        },
        'danger': {'type': 'noul', 'instructions': 'Is the player about to die?'},
        'risk': {'type': 'score', 'instructions': 'Risk level', 'criteria': ['low', 'medium', 'high']},
    },
    views=3,                          # optional: order-robust choice answers
)
print(result['answers']['move']['choice'], result['answers']['move']['probabilities'])

rt.decide('Card charged twice this month', 'Which team?', {'billing': 'Billing', 'tech': 'Tech support'})

Question types

Type criteria Answer
choice map of 2โ€“20 IDs โ†’ descriptions choice: winning ID
score ordered list of 2โ€“20 rubric levels score: expected zero-based level, level: most likely
noul optional {"false": ..., "true": ...} noul: probability of true

probabilities are full softmax values keyed by your IDs. Answers carry truncated: true when the state did not fit.

HTTP server

python -m julia15 serve --port 8080 --views 2
curl -s localhost:8080/v1/predict -d '{
  "state": "Customer: I was charged twice for my order #8812.",
  "questions": {"refund": {"type": "noul", "instructions": "Should we refund the duplicate charge?"}}}'
Method Path
POST /v1/predict {state, questions, views?} โ†’ {model, answers}
POST /v1/predict_batch many {state, questions} in one batched call
POST /v1/logits raw option scores for {state, question, type, options} rows
GET /v1/info ยท /health ยท /metrics config, checkpoint fingerprint, Prometheus counters

Same request and answer format as the Julia 1 runtime. Concurrent requests are batched together automatically. Runtime details: runtime/README.md.

Speed

Hardware Throughput One request
NVIDIA L40S, fp16 encoder ~3,700 rows/s ~15 ms
NVIDIA L40S, fp32 ~1,470 rows/s ~15 ms
8-vCPU cloud VM, fp32 ~5 rows/s ~250 ms (325 tokens)

Repeated requests hit an in-memory cache (< 1 ms). On GPU the decision head stays fp32, so half precision changes fewer than 0.4% of answers.

Files

Path Contents
model.safetensors weights (FP32, 577 MB)
julia_config.json, encoder/, tokenizer/ architecture and tokenizer, read by the runtime
runtime/ julia15 runtime: package, tests, tools/bench.py
metrics/, provenance.json evaluation results and checkpoint provenance
config.json model manifest (architecture, sizes, checkpoint hash; also used by the Hub to count downloads)

The model artifacts are licensed under Apache 2.0. The training pipeline is not included.

Limits of the architecture

Julia 1.5 is a small encoder that compares options. That is what makes it fast, and it also sets its limits:

  • It chooses, it does not compute. Long arithmetic, multi-step math and close numeric comparisons ("is 8,143 more than 8,134?") are unreliable; compute those upstream and pass the result in the state.
  • It does not know what you did not tell it. World knowledge at exam level (MMLU-Pro, ENEM) is weak at 144M parameters; put the needed facts in the state.
  • 1,024 tokens per question. Longer states are truncated (flagged with truncated: true); --strict rejects them.
  • 2โ€“20 options, each up to 48 tokens; question plus options up to 512 tokens.
  • Choice and yes/no views can disagree: asking "which option?" and asking "is option X right?" for each option do not always agree. Use the question type that matches your decision.
  • Score levels: answers on long rubrics are less consistent than choices; prefer a short rubric.
  • Calibration is good on average but not perfect on data unlike its training mix; evaluate on your own questions before relying on probabilities as thresholds.
  • Not a Transformers AutoModel: use the runtime in runtime/.

Supersonic Labs

Downloads last month
41
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for SupersonicLabs/Julia-1.5

Finetuned
(10)
this model

Collection including SupersonicLabs/Julia-1.5

Evaluation results

  • SLCMB score on SLCMB 3.0 (920 cases x 100 option orders)
    self-reported
    46.200
  • Accuracy on Emotion, AG News, Banking77, Enron spam, MASSIVE pt/es/de, tweets pt
    self-reported
    0.730