rpn-gpt-lightning-LoRA

A LoRA adapter for NVIDIA Nemotron 3.5 Lightning. It teaches the model to use an exact Forth machine when a problem needs exact computation, and to answer directly when it does not. The machine can be an external Python loop or a hand-built kernel inside the model's own weights. Code and the full story: github.com/markab21/rpn-gpt.

A real turn of the round-7 model (the adapter on the NVFP4 base) with the kernel in the weights (full example). The problem: "Calculate the remainder when 779951090558722902 is divided by 112974328."

After a short thinking part, the model writes:

779951090558722902 112974328 mod .

89002478

The remainder is 89002478.

\boxed{89002478}

The kernel inside the model wrote the output block. Before it, the kernel ran 348 hidden tokens. A hidden token is a reserved special token that decodes to an empty string. All 361 kernel-owned positions (348 hidden, 13 in the output block) were equal to the spec, kernel/text_vm.py. The answer is correct.

This is the round-7 adapter (s2a_r7): LoRA rank 16, alpha 32, about 23 MB. The same model merged into a plain checkpoint is rpn-gpt-lightning, with GGUF (Ollama, llama.cpp), NVFP4 and MLX versions listed there. Use this adapter to hot-swap it on a vLLM server that already holds the BF16 base. On NVIDIA's NVFP4 base, the adapter at runtime scored 87.5% on the frozen test x1 against 91.7% on BF16 (-4.2 [-7.0, -1.3]); rpn-gpt-lightning-NVFP4 loses less (90.5%, -1.2 [-3.7, +1.3]).

The model only writes the Forth. The machine that runs it is rpn_chat.py (any server), training/loop_sim.py (evaluation sets), or the kernel in the weights (rpn-gpt-lightning-Forth).

How to use

The model needs its system prompt, the card reference_v4.txt. Use the model card's sampling: temperature 1.0, top-p 0.95. Agents: follow AGENT.md, which has every tested command and the expected numbers.

hf download markab/rpn-gpt-lightning-LoRA --local-dir rpn-gpt-lightning-LoRA
vllm serve nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 --served-model-name lightning --max-model-len 65536 \
  --enable-lora --max-lora-rank 16 --lora-modules rpn=$PWD/rpn-gpt-lightning-LoRA

git clone https://github.com/markab21/rpn-gpt && cd rpn-gpt && uv sync
uv run python rpn_chat.py "Calculate the remainder when 82225043186179281 is divided by 6469193." \
  --base-url http://localhost:8000 --model rpn

rpn_chat.py runs each ```forth block in the spec and appends the ```output block; --model rpn selects the adapter (--model lightning is stock Lightning on the same server). For whole sets, training/loop_sim.py. For the kernel in the weights with this adapter, kernel/composite/serve_forth.sh (AGENT.md), or the ready-made rpn-gpt-lightning-Forth.

Results

Set	Problems x samples	Stock Lightning	This adapter	Delta [95% CI]
Frozen test	600 x2	79.1%	92.1%	+13.0 [+10.2, +15.8]
Frozen test, hard problems	194 x2	69.1%	89.7%	+20.6
Dev set	407 x4		93.1%	+1.2 [-0.1, +2.6] vs round 5
Code-heavy dev set	200 x2		83.0%	+5.0 [-0.2, +10.2] vs round 5
MATH-500, no machine	500 x2	97.1%	97.8%	+0.7 [+0.0, +1.4]

Settings: external machine, card v4, think budget 12000 tokens, turn cap 24000 tokens, temperature 1.0, top-p 0.95, grader of 2026-11-06. Both arms ran on the same vLLM --enable-lora engines, with stock Lightning as a zero adapter. A delta is in points, paired by problem, with a 95% bootstrap interval. MATH-500 ran with no machine (card nomachine.txt) and was graded with math_verify. The median answer on the frozen test fell from 3494 to 775 tokens.

We never trained on the evaluation sets. They are in markab/rpn-gpt-data.

Training

LoRA SFT with NeMo AutoModel on BF16 Lightning: rank 16, alpha 32, learning rate 1e-4, 16 rows a step, 503 steps. The rows are 9269 correct, clean turns: the model's own rollouts from earlier rounds plus 1177 program demonstrations. Round 7 added 1820 rows from the round-5 adapter's own correct, short, machine-backed answers. The rounds, and what did not work, are in training.

Limitations

• The machine has hard limits: integers only, 20 digits a number, at most 10 digits for the top operand of * and mod, 196608 kernel steps a block, and 192 cells of code memory a turn. The Forth page lists all of them.
• Some families are still weak on the frozen test (x2): polynomial roots 37.5% (stock 12.5%), big polynomial roots 50.0% (stock 56.3%), recurrences and LCG traces 81.0%. Polynomial roots have no compact exact program for this machine.
• L6.count_digitsum is worse than stock: 78.6% against 92.9%, -14.3 [-28.6, -2.4].
• We did not measure this adapter with the kernel in the weights. The in-weights result (89.9% vs 89.3% external, 407 dev problems) is for the round-3 adapter.
• The kernel is slow: one forward pass for each kernel step. A / on 6-digit numbers takes about 115 hidden tokens.

License

The adapter, the card and the code are MIT, Copyright (c) 2026 Mark Berry and rpn-gpt contributors (LICENSE). MIT covers only our parts.

The base model, NVIDIA Nemotron 3.5 Lightning, is NVIDIA's model under the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1). To use this adapter, you need the base model, so its license applies to that use. This repository holds no NVIDIA weights.

Credit and citation

The MIT license requires that you keep the copyright and permission notice in copies. In addition, we ask you, politely, to credit rpn-gpt with a link to github.com/markab21/rpn-gpt when you use or change this adapter, the kernel or the data.

@misc{berry2026rpngpt,
  author       = {Mark Berry},
  title        = {rpn-gpt: An exact Forth machine compiled into the weights of a language model},
  year         = {2026},
  howpublished = {\url{https://github.com/markab21/rpn-gpt}}
}