Instructions to use markab/rpn-gpt-lightning-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use markab/rpn-gpt-lightning-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="markab/rpn-gpt-lightning-NVFP4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("markab/rpn-gpt-lightning-NVFP4")
model = AutoModelForCausalLM.from_pretrained("markab/rpn-gpt-lightning-NVFP4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use markab/rpn-gpt-lightning-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "markab/rpn-gpt-lightning-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/markab/rpn-gpt-lightning-NVFP4

SGLang

How to use markab/rpn-gpt-lightning-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "markab/rpn-gpt-lightning-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "markab/rpn-gpt-lightning-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "markab/rpn-gpt-lightning-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use markab/rpn-gpt-lightning-NVFP4 with Docker Model Runner:
```
docker model run hf.co/markab/rpn-gpt-lightning-NVFP4
```

rpn-gpt-lightning-NVFP4

rpn-gpt-lightning for vLLM on Blackwell GPUs: NVIDIA Nemotron 3.5 Lightning (30B, 3B active) fine-tuned to drive an exact Forth machine. The main model card has the results and the full story; code: github.com/markab21/rpn-gpt.

The model only writes the Forth. The machine that runs it is rpn_chat.py, a small client loop.

How it is made: NVIDIA's own NVFP4 checkpoint (ModelOpt mixed precision), unchanged, except the 93 linears our adapter changes (Mamba in_proj, attention q/k/v/o, shared experts): they hold the merged BF16 weights and are marked unquantized. NVFP4 rounding error on those weights is about 9%, larger than the fine-tune's change, so quantizing them again would erase most of it. Size: 22 GB (NVIDIA's NVFP4: 21 GB). Script: training/nvfp4_merge.py.

	Frozen test x1 (external machine, card v4)
This checkpoint	90.5% (BF16 + adapter: 91.7%)
NVIDIA NVFP4 + the LoRA adapter at runtime	~88%
BF16 + the LoRA adapter	91.7%

On 8 prompts (2025 tokens), its top-1 token matched the BF16 merged model's at 98.2% of positions (stock Lightning: 89.8%).

Use

vllm serve markab/rpn-gpt-lightning-NVFP4 --served-model-name rpn --max-model-len 65536 --kv-cache-dtype bfloat16
git clone https://github.com/markab21/rpn-gpt && cd rpn-gpt && uv sync
uv run python rpn_chat.py "Calculate the remainder when 82225043186179281 is divided by 6469193." \
  --base-url http://localhost:8000 --model rpn

Tested with vLLM 0.29.0 on an RTX PRO 6000 Blackwell (weight-only FP4 through the Marlin kernel). Keep the KV cache in BF16 (--kv-cache-dtype bfloat16). For a plain chat without the machine, see the main card (card as system prompt, stop: ["```output"], a max_tokens limit).

License

NVIDIA Nemotron 3.5 Lightning is under the OpenMDW License Agreement, version 1.1: full text and NVIDIA's notice in LICENSE. Our parts (the fine-tune, the card, the code) are MIT, Copyright (c) 2026 Mark Berry and rpn-gpt contributors (LICENSE-MIT). Please credit rpn-gpt with a link to github.com/markab21/rpn-gpt.