Instructions to use bytesbrains/naderu-loom-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bytesbrains/naderu-loom-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="bytesbrains/naderu-loom-7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bytesbrains/naderu-loom-7b")
model = AutoModelForCausalLM.from_pretrained("bytesbrains/naderu-loom-7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bytesbrains/naderu-loom-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bytesbrains/naderu-loom-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bytesbrains/naderu-loom-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bytesbrains/naderu-loom-7b

SGLang

How to use bytesbrains/naderu-loom-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bytesbrains/naderu-loom-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bytesbrains/naderu-loom-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bytesbrains/naderu-loom-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bytesbrains/naderu-loom-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use bytesbrains/naderu-loom-7b with Docker Model Runner:
```
docker model run hf.co/bytesbrains/naderu-loom-7b
```

naderu-loom-7b / README.md

nandalxo

naderu-loom-7b v0.1.0 — QLoRA(Mistral-7B-v0.3), eval PASS

dd0605e verified about 19 hours ago

preview code

Raw

History Blame Contribute Delete

5.65 kB

metadata

license: apache-2.0
base_model: mistralai/Mistral-7B-Instruct-v0.3
library_name: transformers
pipeline_tag: text-generation
language:
  - en
tags:
  - interactive-fiction
  - game-master
  - json
  - structured-output
  - lora
  - qlora
  - mistral
  - naderu

naderu-loom-7b

An offline interactive-fiction narrator by Naderu — a BytesBrains Pte. Ltd. venture.

naderu-loom is a compact, specialised game-master model. Given a game state and the player's action, it narrates the next scene and returns a single JSON turn an app can render and apply — built to run offline on-device. It is an honest portfolio/demonstration piece: a small creative model with a quantitative evaluation gate, not a reasoning engine.

Provenance

Fine-tuned from: mistralai/Mistral-7B-Instruct-v0.3 (Apache-2.0).
Method: LoRA (r=16, α=32) trained as QLoRA on the 4-bit MLX quant of the base (mlx-community/Mistral-7B-Instruct-v0.3-4bit), LoRA on 16 layers, lr 3e-5, 1000 iters, assistant-tokens-only (--mask-prompt), seq 1024. Trained with MLX on an Apple M4 Mac Mini (24 GB, no GPU); ~6.5 GB peak, final val loss 0.23. These weights are the adapter fused and de-quantized to bf16 for portability.
Training data: ~320 Naderu-authored (state, action) → JSON turn examples across five genres (fantasy, mystery, sci-fi, horror, fairytale). License-clean and reproducible.
License: Apache-2.0 (inherits the base model's terms).

The turn contract

Mistral-7B-v0.3 has no system role, so the contract is folded into the user message. Each turn the model receives STATE (genre, tone, hp, inventory, flags) + an ACTION, and replies with exactly one JSON object:

{
  "scene": "2–4 sentences of narration.",
  "choices": ["2 to 4 short action strings"],
  "state_delta": {"inventory_add": ["rusty key"], "inventory_remove": [], "flags_set": {"door_unlocked": true}, "hp": 0}
}

Rules: JSON only; 2–4 choices; never remove/use an item the player does not hold; flags stay consistent with the story; honor genre and tone. The app applies state_delta and renders choices as buttons.

How to use

import json
from transformers import AutoModelForCausalLM, AutoTokenizer

SYSTEM = (
    "You are naderu-loom, an offline interactive-fiction narrator (a game master) by "
    "Naderu (naderu.com). Each turn you receive the game STATE and the player's ACTION, "
    "and you reply with exactly one JSON object and nothing else, matching this schema: "
    '{"scene": <2-4 sentence narration>, "choices": [<2 to 4 short action strings>], '
    '"state_delta": {"inventory_add": [..], "inventory_remove": [..], '
    '"flags_set": {..}, "hp": <integer change, 0 if none>}}. '
    "Rules: reply with JSON only; never remove or use an item the player does not have; "
    "keep flags consistent with the story so far; honor the genre and tone; keep choices "
    "between 2 and 4."
)

mid = "bytesbrains/naderu-loom-7b"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid)

state = {"genre": "fantasy", "tone": "grim", "hp": 10, "inventory": [], "flags": {}}
user = f"{SYSTEM}\n\nSTATE: {json.dumps(state)}\nACTION: __start__"   # no system role — fold it in
enc = tok.apply_chat_template([{"role": "user", "content": user}],
                              add_generation_prompt=True, return_tensors="pt")
out = model.generate(enc, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0, enc.shape[1]:], skip_special_tokens=True))

Tip: clamp the returned choices list to ≤ 4 as a thin output-validation layer (see the eval note).

Evaluation

Suite: eval/suites/naderu-loom/run_eval.py (v1, greedy) · Run: 2026-07-15 (34 turns / 9 scripted playthroughs, 5 genres) · Result: PASS

Metric	Gate	Result
valid_json_rate	≥ 0.98	1.000 ✅
schema_rate	≥ 0.95	0.971 ✅ (33/34)
state_violations	0	0 ✅
constraint_rate	≥ 0.98	1.000 ✅

Honest note: 1 of 34 turns emitted 5 choices (over the 2–4 bound), reflected in schema_rate (0.971), which still clears its 0.95 bar. Clamp choices to ≤ 4 in the app. World-state tracking holds — the model correctly refuses to use an item it doesn't hold (0 state violations). Narrative quality is coherent and on-tone across genres but is reported, not gated (subjective).

Limitations & risks

7B → limited long-horizon reasoning; the app is the source of truth for state (the model proposes state_delta, the app applies and validates it).
Fiction may be clichéd, repetitive, or tonally off; English-first.
Trained via 4-bit QLoRA, so it carries the base's 4-bit quantization characteristics.
Model-generated fiction — add content controls for a general audience.

About Naderu

Naderu is an AI-models company (a BytesBrains Pte. Ltd. venture). We train foundation models into specialised ones, release them with model cards, provenance, and clear licensing, and serve the engineering around them. Every capability claim is backed by a real evaluation — never vibes. Recipe, dataset, and eval gate are open in the Naderu repo.

🌐 naderu.com · 🧱 Base Mistral-7B-Instruct-v0.3 (Apache-2.0) · 🏷️ v0.1.0 (2026-07-15)