Surogate-3.5-4B

Surogate is a Romanian-first bilingual (RO/EN) assistant built by Invergent (Bucharest, Romania). It is a dual-mode reasoner: with thinking enabled it reasons step-by-step in the language of the prompt (Romanian prompts → Romanian chain-of-thought, including math); with thinking disabled it answers directly. Trained with the open-source Surogate training framework.

Highlights

The cleanest Romanian at its size: invented word-forms at 0.99 per 1k words (~4× cleaner than comparable general-purpose models), English leakage 0.70 per 1k, missing diacritics just 0.23 per 1k — measured on a 13,963-prompt greedy benchmark (dexonline-authoritative scorer over a 1.52M-form lexicon).
Dual-mode reasoning: 100% prompt-language reasoning (RO and EN probes), 0% <think> leakage in no-think mode (1,392 deterministic probes).

Benchmarks

Benchmark	Surogate-3.5-4B
Romanian text quality
RO invented word-forms / 1k	0.99
RO English-leak / 1k	0.70
RO missing diacritics / 1k	0.23
Knowledge & STEM
MMLU-Pro	76.7
MMLU-Redux	86.1
GPQA Diamond	73.9
SuperGPQA	51.3
Instruction following
RO IFEval (prompt / instruction strict)	0.254 / 0.516
EN IFEval (prompt strict)	0.679
IFEval	74.1
IFBench	48.8
MultiChallenge	40.4
Math & reasoning
RO GSM8K (strict)	0.712
EN GSM8K (strict)	0.880
HMMT Feb 25 / Nov 25	73.3 / 76.0
PolyMATH	50.6
Coding & agents
LiveCodeBench v6	54.1
BFCL-V4	48.8
TAU2-Bench	77.5
Long context
LongBench v2	48.5
Multilingual & translation
Translation EN→RO / RO→EN (chrF2)	53.2 / 62.6
WMT24++	64.6
MMMLU	73.8
MMLU-ProX	69.4
INCLUDE	68.9
Global PIQA	76.5

Usage

vLLM

vllm serve surogate/Surogate-3.5-4B --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser hermes

The repository ships a generation_config.json (temperature 0.6, top_p 0.95, top_k 20) that vLLM applies automatically — these are the recommended sampling settings. For orthography-critical use, greedy decoding (temperature=0) gives the best results.

Thinking control (chat template)

tok.apply_chat_template(messages, add_generation_prompt=True,
                        enable_thinking=True)   # step-by-step reasoning in the prompt's language
                        # enable_thinking=False -> direct answer

Notes

Surogate is optimized Romanian-first; English remains fully supported for bilingual workflows.
As with any model of this size, verify critical outputs.