Euclid-1.0

Bidirectional SAS ↔ R ↔ Python program translation. A 31B hybrid Mamba-2 / MoE / attention model, LoRA-tuned and merged to 16-bit weights.

Model Summary

Euclid-1.0 translates complete statistical and data-processing programs across all six directed pairs over {SAS, Python, R}. The training objective is behavioural equivalence: given identical inputs, the translated program must compute identical values and bind them to identically-named results.

Developer	ProCogia
Method	LoRA SFT (r=64), response-only supervision, adapter merged to 16-bit
Parameters	31.58B total / 45.4M trained (0.14%)
Layers	52 — 23 Mamba-2 mixer, 23 MoE, 6 attention (indices 5, 12, 19, 26, 33, 42)
hidden_size	2688
Attention	32 Q heads / 2 KV heads (GQA), head_dim 128
Mamba-2	64 heads × 64 head_dim, n_groups 8, ssm_state_size 128, conv_kernel 4
MoE	128 routed experts, top-6, plus 1 shared expert; relu2 ungated
Vocab	131,072, untied embeddings
Context	262,144 architectural; trained at 8,192
Precision	BF16, ~60 GiB across 14 shards
License	See LICENSE in this repository

Reasoning is disabled by design. Every training row was rendered with <think></think> already closed, so labels begin exactly where generation begins under enable_thinking=False. Serving with enable_thinking=True under the training system prompt produces the trained reflex — an immediate </think> — or degenerate filler, which under greedy decoding can loop until the context window fills. This is the specification working as written. Reasoning-before-translating is a data change, not a serving flag.

Intended Use

• Single-turn translation of complete programs across the six directed pairs over {SAS, Python, R}.
• Deterministic serving under the verbatim inference contract below: greedy decoding, enable_thinking=False, exact system prompt and user format.
• Output contract: the translated program only, no prose, no markdown fences, result names preserved exactly.

How to Use

Loader class

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "ProCogia/Euclid-1.0",
    torch_dtype=torch.bfloat16,
    device_map={"": 0},
)
tokenizer = AutoTokenizer.from_pretrained("ProCogia/Euclid-1.0")

mamba_ssm and causal_conv1d must be installed. Without them the 23 Mamba layers fall back to torch_forward, which materializes a single 32 GiB contraction tensor at 8,192 tokens and OOMs regardless of batch size. No prebuilt wheels exist for this stack; the kernels compile from source in ~25 minutes and should be cached.

The saved tokenizer carries a pad_token substitution (<SPECIAL_999>). The original pad_token collided with EOS, which trains padded positions as real EOS.

vLLM

vllm serve ProCogia/Euclid-1.0 \
    --max-model-len 8192 \
    --mamba-ssm-cache-dtype float16

An R→SAS probe through the vLLM API returned byte-identical output to the same probe through transformers, confirming merge fidelity and deterministic greedy decoding across serving stacks.

Context can be raised well beyond the trained 8,192 at negligible memory cost — only 6 of 52 layers are attention, so KV cache is ~6 KB/token and Mamba state is length-independent. Quality beyond 8,192 total tokens is outside the trained regime and unvalidated.

Note that prompt_tokens + max_tokens ≤ max_model_len is enforced per request before generation; max_tokens is a hard reservation, not a preference.

Inference contract

Non-negotiable — the model is conditioned on exactly this:

Setting	Value
System prompt	system_prompt.txt, verbatim (1,394 chars)
User message	Translate the following {SAS|Python|R} program to {SAS|Python|R}.\n\n{program}
Chat template	enable_thinking=False
Decoding	greedy (temperature=0)
Stop tokens	model EOS and <|im_end|> (id 11)
max_tokens	≥ 6000 for full programs, or omit to auto-size
Output	bare program — no fences, no prose

Prompt format

The model was trained against a single system prompt across every row, and a drift guard asserted the notebook prompt matched the dataset prompt at eval time. Deviating from its contract is off-distribution. The verbatim prompt ships as system_prompt.txt; the block below reproduces its contract:

You are a code translation engine for statistical and data-processing programs written in SAS, Python, and R.

You are given one complete program in a source language and produce the equivalent program in the target language. Behavioural equivalence is the only criterion: given the same inputs, your program must compute the same values and place them in results carrying the same names.

- Preserve the source program's structure, step order, and intent. Carry its comments across as comments in the target language, and add a brief comment where the target expresses a source construct non-obviously.
- Name every result exactly as the source names it, so results can be compared name for name.
- SAS semantics decide the answer even when SAS is not the target language. A SAS date is whole days since 1960-01-01 and a datetime is seconds since 1960-01-01, both stored as plain numbers. A SAS FORMAT changes only how a value is displayed, never what is stored. Missing (.) sorts below every number, so `x < 5` is true when x is missing. Character values are blank-padded to a declared length and compare ignoring trailing blanks.
- A step that only prints or plots produces no data and has no translation outside SAS; leave it out rather than inventing an equivalent.
- Respond with the translated program and nothing else: no prose, no explanation, no markdown code fences, no placeholders.

The user turn is a single line followed by the bare source program:

Translate the following {SOURCE} program to {TARGET}.

{program}

The source program is not fenced. The model emits a bare program, likewise unfenced.

Training Data

Proprietary. Not released. Composition is disclosed below for reproducibility of method, not of data.

Training rows	12,893 after filtering (from 12,964)
In-training eval subset	120 (stratified, 20 per direction)
Held-out eval set	544
Format	JSONL, single-turn system + user + assistant
Length (tokens, rendered template)	p50 1,369 / p95 4,754 / p99 7,142 / max 43,654

Training Procedure

Adaptable surface

93% of this model was frozen by necessity rather than by choice, and the reasons are architectural.

Routed experts are unreachable. They account for 29.375B parameters (93.0%) and are stored as fused 3-D nn.Parameter tensors. Attaching LoRA to them routes every expert forward through PEFT's ParamWrapper.get_delta_weight, which materializes the full delta per fused tensor — 128 × 2688 × 1856 in fp32, roughly 2.3 GiB per MoE layer per forward, across 23 layers. This OOMs regardless of sequence length or batch size. An exclude_modules regex cannot prevent it, because the fused experts are parameters rather than modules and there is nothing to exclude; dotted target names are what keep suffix-matching confined to the shared expert's nn.Linear.

Quantization is unavailable. bitsandbytes quantizes nn.Linear only, so load_in_4bit=True against fused expert parameters either errors or silently leaves 93% of the model in bf16 while appearing to be QLoRA. This sets a hard 80 GB floor with no memory escape hatch.

Only in_proj is adaptable on the Mamba side. PEFT's _check_lora_target_modules_mamba raises on out_proj and conv1d as a guard against selective-scan gradient instability. The cost is negligible — about 4M parameters at r=64.

Packing is forbidden. It is incompatible with response-only masking, and more fundamentally this is a hybrid Mamba model: the SSM recurrence carries state along the sequence, so packed documents leak into each other. Attention can be masked; a recurrence cannot.

Capacity was therefore redirected to the dense pathway.

LoRA configuration

Parameter	Value
Rank r	64
alpha	128
Dropout	0
Bias	none
Target modules	q_proj, k_proj, v_proj, o_proj, in_proj, shared_experts.up_proj, shared_experts.down_proj
Scope	Attention (6 layers), Mamba input projection (23 layers), shared expert (23 MoE layers)
Routed experts, router .gate, MTP head	Never targeted; enforced by three runtime assertions
Trainable	45.4M (0.14%)

The shared expert is the highest-value target available: it is dense and runs on every token of all 23 MoE layers, unlike routed experts which each see roughly 6 of every 128 tokens. r = 64 compensates for the small module count.

Optimization

Parameter	Value
Learning rate	2e-4, cosine
Warmup	48 steps (3%)
Optimizer	adamw_8bit, wd 0.01
Gradient clipping	0.5
Micro-batch × accumulation	1 × 16 (effective batch 16)
Sequence length	8,192, packing off
Loss	Response-only
Precision	BF16, sdpa attention, unsloth gradient checkpointing
Epochs	2, eval and save every 100 steps
Seed	3407
Total steps	1,611

Response-only supervision masks the system prompt, instruction, and source program to -100; measured supervised fraction is mean 0.344, min 0.083. The response marker includes <think></think>, so training labels begin exactly where generation begins at serve time. Masking at the shorter assistant\n marker would train the model to emit a second <think></think> that the inference prompt already supplies. Assertions guard both failure directions — all-masked labels produce a permanent loss of 0.0, and unmasked labels train on prompts.

Gradient norms stayed well under the 0.5 clip (0.20–0.30 at full learning rate) and the clip monitor never fired, retroactively validating the 2e-4 rate.

Infrastructure

GPU	1× H100 80GB SXM5, single GPU, no sharding
Framework	unsloth 2026.8.18, torch 2.11.0+cu128, transformers 5.5.4, trl 1.10.0
Kernels	mamba_ssm 2.3.2.post1, causal_conv1d 1.6.2.post1, compiled from source
Loaded footprint	58.8 GiB bf16
Peak VRAM	65.1 GiB steady across the full run, no leak
Throughput	~10.4 s/step average
Wall clock	~10 h

The torch +cu128 build is load-bearing: the container toolkit is CUDA 12.8, and torch refuses to compile extensions across a CUDA major-version gap, which makes the Mamba kernels unbuildable on the +cu130 default.

Evaluation

Pending. The planned protocol:

Element	Specification
Set	544 examples, repo-disjoint, per-direction
Method	Execute source and translation on identical inputs; diff results name-for-name
Metric	pass@1, pooled and per-direction
Prompt	Training system prompt, greedy decoding, enable_thinking=False

Any comparison against an untuned baseline must run each model under its own contract. Measuring this model with reasoning forced on and temperature=0.6 measures nothing about training quality.

Spot-checks surfaced two error classes that token-level loss cannot see: invented I/O (a proc means data=sales source with an unbound dataset reference produced a Python translation that added pd.read_csv("sales.csv"), dropped the n statistic, and added a print), and statistical-option semantics (an R aggregate(...) → SAS translation added NOMISS to PROC MEANS, which affects class-variable missing handling rather than the statistic).

Hardware Requirements

Weights on disk	~60 GiB (BF16, 14 shards)
Loaded footprint	58.8 GiB
KV cache	~6 KB/token (only 6 of 52 layers are attention)
Mamba state	Length-independent
Practical single-GPU floor	80 GB (H100 / H200 / A100 80GB)
Quantization	Unavailable — see below

4-bit and 8-bit loading do not work on this architecture. bitsandbytes quantizes nn.Linear only, while the routed experts holding 93% of the parameters are fused 3-D nn.Parameter tensors. The 80 GB floor is hard.

For reference, training peaked at 65.1 GiB on a single H100 80GB at batch size 1, sequence length 8,192, with gradient checkpointing on and packing off.

Citation

@misc{euclid_1_0,
  title  = {Euclid-1.0: Bidirectional SAS/R/Python Program Translation},
  author = {ProCogia},
  year   = {2026},
  url    = {https://huggingface.co/ProCogia/Euclid-1.0}
}