Safetensors
GGUF
English
Japanese
pretrained
base-model
from-scratch
tessera
AIIT-Threshold commited on
Commit
b5c0da0
·
verified ·
1 Parent(s): c6c85b2

Document GGUF files: base-model-only, needs patched llama.cpp, raw-completion usage

Browse files
Files changed (1) hide show
  1. USAGE.md +23 -0
USAGE.md CHANGED
@@ -54,3 +54,26 @@ The **base** is a base model: fluent, drifts, loops — post-train it.
54
  The **adapters** are small hand-made SFT stages over ~1,031 human-written examples;
55
  see the model card for their personalities. This is a 1B: charming, honest about
56
  what it doesn't know, and not a reasoning engine. 12×7 may not be 84. That's the deal.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  The **adapters** are small hand-made SFT stages over ~1,031 human-written examples;
55
  see the model card for their personalities. This is a 1B: charming, honest about
56
  what it doesn't know, and not a reasoning engine. 12×7 may not be 84. That's the deal.
57
+
58
+ ## GGUF (llama.cpp)
59
+
60
+ Two GGUF conversions of the **base** pretrain checkpoint (not the `lora.py` chat adapters — those aren't merged in) are provided under `gguf/`:
61
+
62
+ | File | Size | Use |
63
+ |---|---|---|
64
+ | `gguf/tessera-1b-Q6_K.gguf` | ~883 MB | recommended — Q6_K quant, ~6.6 bits/weight |
65
+ | `gguf/tessera-1b-f16.gguf` | ~2.2 GB | full-precision source, requantize to other sizes from this |
66
+
67
+ ### Important: requires a patched llama.cpp build
68
+
69
+ ProtoGPT uses **RMSNorm** (see `model.py`), but llama.cpp's stock `gpt2` architecture — the correct match for everything else here (learned absolute position embeddings, GELU MLP, fused-QKV full attention, no biases) — hard-codes classic mean-centered LayerNorm. **A GGUF built for this model will not load correctly on unpatched/upstream llama.cpp, Ollama, LM Studio, or any other tool bundling stock llama.cpp** — it'll run without erroring but compute the wrong norm.
70
+
71
+ `tessera1b-rmsnorm-gpt2-arch.patch` is included in this repo: a 6-line diff against `llama.cpp/src/models/gpt2.cpp` that swaps the 3 `build_norm(..., LLM_NORM, ...)` call sites to `LLM_NORM_RMS` and makes the (unused, always-zero) norm bias tensors optional. Apply it to a fresh llama.cpp checkout, rebuild, then use that binary for both `llama-quantize` and inference.
72
+
73
+ ### Usage — raw completion, not chat
74
+
75
+ This is a base model. Use `llama-completion` (plain next-token continuation), not `llama-cli`'s interactive/conversation mode — that auto-applies a chat template (ChatML by default) this model was never trained on and produces garbage as a result.
76
+
77
+ ```
78
+ ./llama-completion -m tessera-1b-Q6_K.gguf -p "Your prompt here" -n 100 --temp 0.3
79
+ ```