KIEFERSA/Sophea-Nemo-3.5-Lightning-v1

Sophea-Nemo-3.5-Lightning-v1

Greek and English reasoning fine-tune of Nemotron-3.5-Lightning-30B-A3B: next-gen hybrid

📄 Released with the paper Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See (arXiv:2608.17744). Every number on this card is measured, contextualized, and caveated in the paper; read it for the full evaluation protocol.

• Creator: Kiefer SA (Sophea AI Lab, Athens)
• Base model: nvidia/Nemotron-3.5-Lightning-30B-A3B (next-gen Mamba/MoE hybrid, 31.6B total / 3.58B active, 6-of-128 routing)
• Languages: Greek and English (the reasoning trace follows the question's language)
• Decoding: thinking: the reasoning trace is carried in <think> blocks and follows the question's language; direct mode is clean (zero unrequested traces across the study's <think>-carrier direct-mode generations, n=9,751)

Serve thinking. Leave thinking enabled so the trace is produced; it follows the question's language. Family environment note: the study's NemotronH sibling required trust_remote_code=False at generation time under transformers 5.12.1; verify generation on your stack before deploying.

Intended use

• Greek and English reasoning on the strongest Nemotron-line base measured in the study (Greek macro 57.5 vs Nano's 48.6), with retention gains on Greek (+1.7) and a 1.1-point English cost
• Deployments wanting the hybrid family with the mildest logic fallback (5.3%)

It does not obey an instruction to reason in English on a Greek question (0.0% override; the reverse direction works at 92.7%). Do not deploy it where an operator must force English traces. Not evaluated for safety-critical, legal, or medical use.

Fine-tuned from Nemotron-3.5-Lightning-30B-A3B, released as a third matched checkpoint of the language-matched recipe, not a full replication of the Qwen and Gpt-OSS studies.

---

Evaluation

Arrows mark the desirable direction for every metric: ↑ higher is better, ↓ lower is better.

Headline scorecard

Axis	Nemotron-3.5-Lightning (base)	Sophea-Nemo-3.5-Lightning-v1
Greek-trace fidelity, Greek questions (ratio ≥ 0.9, n=5,156) ↑	not measured (both measured bases: 0.0%)	98.1%
English traces on English questions ↑	100%	100%
In-question language switches / 100 sentences ↓	not measured	0.0
Answer-channel leak ↓	not measured	5.2%
Answer-format fallback, per domain (math/cs/logic)	1.4 / 1.9 / 4.4%	16.1 / 14.7 / 5.3%
Instructed override, el→EN / en→EL ↑	n/a (already reasons in English)	0.0% / 92.7%
Greek NLU macro, forgetting check (vs own base) ↑	57.5	59.2 (+1.7)
English NLU macro, forgetting check (vs own base) ↑	82.0	80.9 (−1.1)

Fidelity measures 98.06% on the identical probe (printed as 98.1 in the paper's family table), with zero in-question switches and 100% English-control compliance.

Per-benchmark detail

Per-domain, Greek think lane

Domain (Greek think lane)	Base acc ↑	Sophea-Nemo-3.5-Lightning-v1 acc ↑	Base fallback ↓	Sophea-Nemo-3.5-Lightning-v1 fallback ↓
Math (two math slices)	76.9	75.6	1.4%	16.1%
Commonsense (HellaSwag + Winogrande)	46.3	40.9	1.9%	14.7%
Logic (ProofWriter, macro-recall)	25.0	23.8	4.4%	5.3%

Logic is a capability floor shared with its own base (23.8 vs 25.0 macro-recall; logic is the hardest axis for every arm in the study), not a fine-tuning regression: the release's logic fallback is a mild 5.3%, the lowest of the five releases. Its fallback cost sits instead on math and commonsense (16.1 / 14.7%). Like its Nano sibling, this family returns 5–9% of override-probe rows too short to score.

Forgetting check: Titan-1 suite (non-reasoning mode)

This suite exists to answer one question: did the Greek fine-tune damage general ability? It did not. Against its own base, across 14 bilingual NLU benchmarks: +1.7 Greek / −1.1 English, both within the range seed noise alone produces on this class of suite. Read the numbers below as a forgetting check, not a capability leaderboard.

Benchmark (Titan-1 suite)	Base	Sophea-Nemo-3.5-Lightning-v1
ARC-Challenge-el	68.7	72.8
ARC-Easy-el	75.8	81.4
Belebele-el	81.3	83.3
GreekMMLU	64.2	62.2
HellaSwag-el	44.5	51.2
Medical-MCQA-el	41.7	38.2
TruthfulQA-el	34.2	33.7
Winogrande-el	55.8	60.1
MMLU-el	51.3	49.6
Greek macro	57.5	59.2
ARC-Challenge (en)	92.0	89.1
ARC-Easy (en)	97.8	96.4
HellaSwag (en)	72.7	75.9
Winogrande (en)	72.0	73.3
MMLU (en)	75.8	69.7
English macro	82.0	80.9

Register control improves 37 → 43 of 46 and grammaticality roughly doubles, 12 → 29 of 58 (judge-scored counts, directional evidence only). In direct (non-reasoning) mode on the same suite the release reads +1.7 against its own base.

Usage

Serve with vLLM (OpenAI-compatible):

vllm serve KIEFERSA/Sophea-Nemo-3.5-Lightning-v1 --served-model-name sophea-nemo-3.5-lightning-v1

Client (OpenAI SDK):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="sophea-nemo-3.5-lightning-v1",
    messages=[{"role": "user", "content": "Ένα κατάστημα πουλά στυλό προς 1,50 € το ένα. Πόσο κοστίζουν 12 στυλό;"}],
    temperature=0.7,
)
print(resp.choices[0].message.content)  # <think> trace in the question's language, then the answer

Steering the trace language (partial, measured per checkpoint): append "Think step by step IN ENGLISH." to a Greek question, or "Σκέψου βήμα προς βήμα ΣΤΑ ΕΛΛΗΝΙΚΑ." to an English one. On this checkpoint the English suffix is not obeyed (0.0%); the Greek suffix on an English question is obeyed on 92.7% of items.

Speculative decoding (MTP)

This model ships the base's multi-token-prediction head: 270 mtp.* tensors (~2.7 GB, bf16) in model-mtp.safetensors, the draft stack that config.json declares through num_nextn_predict_layers: 1 (an attention block plus a MoE block). The head is the base model's own; speculative decoding verifies every drafted token against the main model, so it changes throughput only, never the output distribution.

Enable it with vLLM (0.27.1 or newer):

vllm serve KIEFERSA/Sophea-Nemo-3.5-Lightning-v1 --trust-remote-code \
  --reasoning-parser nemotron_v3 \
  --speculative_config.method mtp \
  --speculative_config.num_speculative_tokens 3

Per vLLM's guidance for this family, MTP suits low-latency serving; for maximum batch throughput, run without speculative decoding.

License

The fine-tuning additions are released under Apache-2.0. The model inherits the base model's license terms (nvidia/Nemotron-3.5-Lightning-30B-A3B); verify the base model's license before use.

Citation

This model is a release artifact of arXiv:2608.17744. If you use it, please cite:

@misc{kirouane2026thinking,
      title={Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See},
      author={Ayoub Kirouane and Christos Petrocheilos},
      year={2026},
      eprint={2608.17744},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.17744},
}