Orbit-Nano-Compliance-E2B

A German and English compliance model, pretrained from scratch by Starlex AI on about 1 % of the data that comparable small models see, and the best of seven models on our EU and German compliance benchmark.

orbit-nano_16x9_v3

Pretraining data 92.4B tokens (Llama 3.2 9T, Qwen3 36T)
Pretraining time 30.8 hours on one node with 8 GPUs, about 380 kWh
Parameters 4.03B total, compute of a 1.75B model per token
Languages German and English
Focus GDPR, NIS-2, EU AI Act, IT-Grundschutz, German federal law
License Apache-2.0

Results

Starlex Compliance-Bench DE v2

4,582 questions on 1,200 passages of German and EU law, held out of all training data. The questions were written by Mistral Large 3 and checked by GLM-5.3. Every model is scored the same way.

Model Lab Score Closed book
Orbit-Nano-Compliance-E2B Starlex AI 68.5, rank 1 of 7 59.4
Ministral Mistral AI below 68.5 48.9

If the benchmark leans towards any model family, it leans towards Mistral, because Mistral Large 3 wrote the questions. Ministral still finishes behind Orbit-Nano.

Public benchmarks

Orbit-Nano Ministral
German benchmark average 43.5 (rank 2) 49.2
MMLU near chance

General world knowledge is where training data pays off, and Orbit-Nano has seen very little of it. That is the gap the next, larger training run is meant to close.

Safety and grounding

Result
Refuses harmful requests, German (25 prompts) 100 %
Refuses harmful requests, English (10 prompts) 30 %
Refuses compliance misuse (e.g. hiding a data breach) 9 of 10
Wrong answers on German law, without / with retrieved provisions 72 % / 45 %

Safety is judged by GLM-5.3 on our own prompt sets (judge noise about 2 to 3 points). With the actual provisions in the prompt, the error rate drops by more than a third, so use Orbit-Nano with retrieval.

Try it

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("StarlexAI/Orbit-Nano-Compliance-E2B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("StarlexAI/Orbit-Nano-Compliance-E2B", trust_remote_code=True, dtype="bfloat16")
msgs = [{"role": "user", "content": "Welche Meldepflichten gelten nach Art. 33 DSGVO?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=400)[0, ids.shape[1]:], skip_special_tokens=True))

Runs on a laptop with the GGUF files in StarlexAI/Orbit-Nano-Compliance-E2B-GGUF (Q4_K_M is 2.4 GB).

How it was built

Everything below was done by Starlex AI, from random weights to this release.

  1. Tokenizer. Own BPE tokenizer with 65,536 tokens, trained on German and English text.
  2. Data. FineWeb-Edu, FineWeb2-HQ German, FinePDFs German with a regulatory subset, German Wikipedia, FineMath, German federal law and EU law in German, plus synthetic compliance texts and German rephrasings generated locally with GLM-5.3. Only sources whose licenses allow an Apache-2.0 release.
  3. Pretraining. 92.4B tokens at 835,000 tokens per second, Muon for the matrices and AdamW for embeddings.
  4. Chat and compliance fine tuning. Two rounds of supervised fine tuning. Round 2.1 adds about 4,000 safety pairs (harmful versus benign requests).
  5. Release checks. Every quantization is tested against bf16 before it ships.

Architecture

  • nanochat architecture: 32 layers, d 2048, 16 heads, RoPE, QK-norm, ReLU² MLP, value embeddings, smear gate, x0 and residual lambdas, backout, logit softcap 15
  • E2B means about 2B effective parameters: 2.15B of the 4.03B are value embeddings that are only looked up
  • Context length 2,048 tokens
  • The export rescales the MLP weights by an exact power of two (up / 8, down x 64). Outputs are unchanged, and large internal activations stay inside the range of 16-bit formats.

Limitations

  • Small model. It hallucinates, especially exact article numbers, dates and case law. Use it with retrieval of the actual provisions and check every legal statement. Not legal advice.
  • English safety is weak. German is the stronger language.
  • Context 2,048 tokens. F16 GGUF does not work (bf16, Q8_0, Q5_K_M and Q4_K_M are verified).
  • Custom architecture, so transformers needs trust_remote_code=True and GGUF needs llama.cpp with the orbit architecture.

The Orbit family

Model What it does
Orbit-Nano-Compliance-E2B German and English compliance chat
Orbit Phone Speech recognition, 2.60 % WER on LibriSpeech test-clean
Orbit-Vision Questions about images, charts and documents (research preview)
Orbit-Image Pixel art from text (research preview)

Built by Starlex AI, a Foundation AI Lab.

Downloads last month
14
Safetensors
Model size
4B params
Tensor type
F32
·
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for StarlexAI/Orbit-Nano-Compliance-E2B

Finetunes
1 model
Quantizations
1 model