Jotschi/nano-astronaut-d12

Kleiner Astronaut — d12 (SFT)

A small German language model that writes children's astronaut stories on request and answers a question about the story it just told.

• Architecture: NanoChat (NanoChatForCausalLM), 12 layers, hidden size 768, 6 heads, 2048 context
• Parameters: 97.5M (84.9M transformer + 12.6M embedding/unembedding)
• Vocabulary: 8,192, a custom BPE trained on this corpus (4.58 chars/token — better compression on German than GPT-2's 2.67 or GPT-4's 3.48 on the same text)
• Precision: bfloat16
• License: Apache-2.0
• Code: https://github.com/Jotschi/nanochat/tree/kleiner-astronaut
• Dataset: https://huggingface.co/datasets/Jotschi/kleiner-astronaut

Running it with vLLM

docker run --rm -it --gpus '"device=0"' --ipc=host -p 8000:8000 \
  -v /path/to/d12hf-sft:/model \
  vllm/vllm-openai:latest \
  --model /model \
  --served-model-name nano-astronaut-d12 \
  --model-impl transformers \
  --enforce-eager \
  --max-model-len 2048 \
  --chat-template-content-format string

Two flags are not optional:

• --model-impl transformers — there is no native vLLM kernel for NanoChat; it runs through transformers' implementation (PR #41634).
• --chat-template-content-format string — vLLM's OpenAI server otherwise rewrites message content into a list of parts, while the chat template indexes content as a plain string. Without it every request fails with 400 User messages must contain string content.

Then:

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' -d '{
    "model": "nano-astronaut-d12",
    "messages": [{"role": "user", "content": "Schreib ein Abenteuer von Mira im Raumschiff"}],
    "max_tokens": 300, "temperature": 0.6
  }' | jq -r '.choices[0].message.content'

Or with transformers directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./d12hf-sft", dtype="bfloat16", device_map="cuda")
tok = AutoTokenizer.from_pretrained("./d12hf-sft")

messages = [{"role": "user", "content": "Schreib ein Abenteuer von Mira im Raumschiff"}]
# return_dict=True: on transformers 5.x apply_chat_template returns a BatchEncoding,
# not a bare tensor, so generate() needs **inputs rather than a positional tensor.
inputs = tok.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to("cuda")
out = model.generate(**inputs, max_new_tokens=300, temperature=0.6, do_sample=True)
print(tok.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Example traces

Real output from this checkpoint, temperature 0.6. The intended shape is a story request followed by a question about that story.

A good one:

USER      Schreib ein Abenteuer von Mira und Aris im Raumschiff Sternchen
ASSISTANT In einem funkelnden Sternenmeer, wo die Sterne wie Diamanten glitzern,
          lebte ein kleiner Astronaut namens Mira. Eines Tages, als Mira durch
          das All flog, sah sie plötzlich ein leuchtendes Licht, das wie ein
          funkelnder Stern am Himmel tanzte. Es war ein Komet, der wie ein
          glitzernder, funkelnder Schweif durch das All schoss. Mira wusste, dass
          sie Abenteuer erleben wollte. Sie sprang in ihre Rakete und flog dem
          Kometen hinterher.

USER      Wer fliegt mit dem Raumschiff?
ASSISTANT Mira fliegt mit ihrem Raumschiff, um den Kometen zu verfolgen.

A typical failure, shown because it is representative, not rare — note the name looping and the collapsed answer:

USER      Erzähle mir eine Geschichte über den Roboter Zibu auf dem Mond
ASSISTANT In einem fernen Sternensystem ... Dort traf er auf den Roboter Zibu,
          der Roboter war ein kleiner Roboter, der wie ein kleiner Roboter
          aussah. Zibu war neugierig und spielte mit Zibu und Zibu. ...

USER      Wo spielt die Geschichte?
ASSISTANT Zibu Zibu war der beste Freund von Zibu.

Limitations

This is a 97.5M-parameter model trained for about 50 GPU-minutes on a single consumer card. Treat it as a demonstration, not a product.

• Requested keywords appear about a third of the time. Measured on held-out conversations: 32.99%. Only 58% of training examples had both requested keywords verifiably present in their reference story.
• Stories end abruptly and repeat. The instruction-tuning stories were clamped to 300 characters, so the model never saw many long, well-resolved endings.
• Question answering is weak (39.17% on held-out data). Only 14,289 of the 133,204 training stories carry a question/answer pair; the rest can teach request → story only.
• German only. It knows nothing outside children's space adventures, has no factual knowledge, and should not be relied on for anything.

Training

Corpus	133,204 synthetic German stories, 128.3M chars ≈ 28.0M tokens
Pretraining	600 steps, 131,072-token batches, final val bpb 1.3209
SFT	1 epoch over 24,407 conversations, final val bpb 1.2135
Hardware	1× RTX 4090 / RTX 3060

The corpus is an expanded regeneration produced by the pipeline in the repo, descended from the published Jotschi/kleiner-astronaut dataset (27,908 stories) — same generator lineage, roughly 4.8× the stories. Conversations pair a request with a story, and where available a follow-up question with its answer.

The training horizon was chosen by measurement rather than by a scaling rule: validation bpb bottoms out at roughly 550 steps on this corpus, and the default data:parameter ratio would have implied about 36 epochs of it.

This checkpoint is trained with --hf-compatible, which omits the components nanochat added after the transformers port (value embeddings, the smear gate, the per-layer residual lambdas) so the weights map 1:1 onto NanoChatForCausalLM. On a corpus this small that is not a downgrade — it reached a lower validation bpb than the full architecture (1.3209 vs 1.3300), which had begun to overfit.