rpn-gpt-lightning

NVIDIA Nemotron 3.5 Lightning (30B, 3B active) fine-tuned to use an exact Forth machine when a problem needs exact computation, and to answer directly when it does not. This repository is the round-7 LoRA adapter (s2a_r7) merged into the BF16 base: a plain checkpoint that loads wherever Lightning loads. Code and the full story: github.com/markab21/rpn-gpt.

A real turn of the round-7 model (the adapter on the NVFP4 base) with the kernel in the weights (full example). The problem: "Calculate the remainder when 779951090558722902 is divided by 112974328."

After a short thinking part, the model writes:

779951090558722902 112974328 mod .

89002478

The remainder is 89002478.

\boxed{89002478}

The kernel inside the model wrote the output block. Before it, the kernel ran 348 hidden tokens. A hidden token is a reserved special token that decodes to an empty string. All 361 kernel-owned positions (348 hidden, 13 in the output block) were equal to the spec, kernel/text_vm.py. The answer is correct.

The model only writes the Forth. The machine that runs it is rpn_chat.py, a small client loop that works with any server, or the kernel in the weights (rpn-gpt-lightning-Forth). In a plain chat, the model stops after its program and nothing runs it.

Formats

Repository	Format	For	Accuracy (frozen test x1, external machine)
rpn-gpt-lightning (this one)	BF16 safetensors, 63 GB	vLLM, transformers	92.1% (x2, from the adapter)
rpn-gpt-lightning-NVFP4	NVFP4 + BF16, 22 GB	vLLM on Blackwell	90.5%
rpn-gpt-lightning-GGUF	GGUF Q4_K_M / Q5_K_M / Q8_0	llama.cpp, Ollama, LM Studio	Q4_K_M 91.2%, Q8_0 93.5%
rpn-gpt-lightning-MLX-4bit, -MLX-8bit	MLX	Apple silicon (mlx-lm)	3 of 3 hard demo problems (4-bit, M4 Max)
rpn-gpt-lightning-LoRA	PEFT LoRA, 23 MB	vLLM --enable-lora on the BF16 or NVFP4 base	92.1% (x2)
rpn-gpt-lightning-Forth	this model + the Forth kernel in the weights	transformers / vLLM, custom code

Merged and adapter are the same model: on 8 prompts (2025 answer tokens, vLLM 0.29), the merged model's top-1 token matched the base + adapter's at 99.5% of positions (stock Lightning: 89.8%). On 100 dev problems x2: merged 93.0%, base + adapter 89.5%, +3.5 [-1.5, +8.5] (noise).

How to use

The model needs its system prompt, the card reference_v4.txt, which describes the machine. Use the model card's sampling: temperature 1.0, top-p 0.95. Agents: follow AGENT.md, which has every tested command and the expected numbers.

With the machine (any OpenAI-compatible server)

vllm serve markab/rpn-gpt-lightning --served-model-name rpn --max-model-len 65536

git clone https://github.com/markab21/rpn-gpt && cd rpn-gpt && uv sync
uv run python rpn_chat.py "Calculate the remainder when 82225043186179281 is divided by 6469193." \
  --base-url http://localhost:8000 --model rpn

rpn_chat.py builds the prompt with the card, sends it to /v1/completions, runs each ```forth block in the spec (kernel/text_vm.py), appends the ```output block, and lets the model go on. It closes the thinking after 12000 tokens (--think-budget), as in our evaluations. --server llamacpp | ollama | mlx selects the other servers. If the server has an API key, set VLLM_API_KEY. For whole evaluation sets, use training/loop_sim.py.

Plain chat, no machine

from openai import OpenAI
card = open("reference_v4.txt").read()
r = OpenAI(base_url="http://localhost:8000/v1", api_key="none").chat.completions.create(
    model="rpn", messages=[{"role": "system", "content": card}, {"role": "user", "content": "What is 2^40?"}],
    temperature=1.0, top_p=0.95, max_tokens=16000, stop=["```output"])
print(r.choices[0].message.content)

The stop ends the reply after a program instead of letting the model guess its output. Set max_tokens: with no thinking budget, a hard problem can make the model compute by hand for a very long time. For real use, run the machine with rpn_chat.py.

transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("markab/rpn-gpt-lightning")
model = AutoModelForCausalLM.from_pretrained("markab/rpn-gpt-lightning", dtype=torch.bfloat16, device_map="cuda")
msgs = [{"role": "system", "content": open("reference_v4.txt").read()}, {"role": "user", "content": "What is 2^40?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda")
out = model.generate(**ids, max_new_tokens=4000, do_sample=True, temperature=1.0, top_p=0.95,
                     stop_strings=["```output"], tokenizer=tok)
print(tok.decode(out[0, ids["input_ids"].shape[1]:]))

The merge: W += (alpha / r) * B @ A in fp32, rounded once to BF16, for the 93 adapter targets (training/merge_lora.py). Every other tensor and file is NVIDIA's, unchanged.

Results

Set	Problems x samples	Stock Lightning	Round 7 (this model)	Delta [95% CI]
Frozen test	600 x2	79.1%	92.1%	+13.0 [+10.2, +15.8]
Frozen test, hard problems	194 x2	69.1%	89.7%	+20.6
Dev set	407 x4		93.1%	+1.2 [-0.1, +2.6] vs round 5
Code-heavy dev set	200 x2		83.0%	+5.0 [-0.2, +10.2] vs round 5
MATH-500, no machine	500 x2	97.1%	97.8%	+0.7 [+0.0, +1.4]

Settings: external machine, card v4, think budget 12000 tokens, turn cap 24000 tokens, temperature 1.0, top-p 0.95, grader of 2026-11-06. Both arms ran on the same vLLM --enable-lora engines, with stock Lightning as a zero adapter. A delta is in points, paired by problem, with a 95% bootstrap interval. MATH-500 ran with no machine (card nomachine.txt) and was graded with math_verify. The median answer on the frozen test fell from 3494 to 775 tokens.

We never trained on the evaluation sets. They are in markab/rpn-gpt-data.

Training

LoRA SFT with NeMo AutoModel on BF16 Lightning: rank 16, alpha 32, learning rate 1e-4, 16 rows a step, 503 steps. The rows are 9269 correct, clean turns: the model's own rollouts from earlier rounds plus 1177 program demonstrations. Round 7 added 1820 rows from the round-5 adapter's own correct, short, machine-backed answers. The rounds, and what did not work, are in training.

Limitations

• The machine has hard limits: integers only, 20 digits a number, at most 10 digits for the top operand of * and mod, 196608 kernel steps a block, and 192 cells of code memory a turn. The Forth page lists all of them.
• Some families are still weak on the frozen test (x2): polynomial roots 37.5% (stock 12.5%), big polynomial roots 50.0% (stock 56.3%), recurrences and LCG traces 81.0%. Polynomial roots have no compact exact program for this machine.
• L6.count_digitsum is worse than stock: 78.6% against 92.9%, -14.3 [-28.6, -2.4].
• We did not measure round 7 at scale with the kernel in the weights. The in-weights result (89.9% vs 89.3% external, 407 dev problems) is for the round-3 adapter.
• The kernel is slow: one forward pass for each kernel step. A / on 6-digit numbers takes about 115 hidden tokens.

License

The weights are NVIDIA Nemotron 3.5 Lightning with our fine-tune merged in. NVIDIA's model is under the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1); its full text and NVIDIA's copyright notice are in LICENSE, and NVIDIA's notices (bias.md, explainability.md, privacy.md, safety.md) are kept where the base has them. Our parts (the fine-tune, the card, the code) are MIT, Copyright (c) 2026 Mark Berry and rpn-gpt contributors (LICENSE-MIT).

Credit and citation

The MIT license requires that you keep the copyright and permission notice in copies. In addition, we ask you, politely, to credit rpn-gpt with a link to github.com/markab21/rpn-gpt when you use or change this adapter, the kernel or the data.

@misc{berry2026rpngpt,
  author       = {Mark Berry},
  title        = {rpn-gpt: An exact Forth machine compiled into the weights of a language model},
  year         = {2026},
  howpublished = {\url{https://github.com/markab21/rpn-gpt}}
}