Instructions to use AIIT-Threshold/Tessera-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AIIT-Threshold/Tessera-1B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AIIT-Threshold/Tessera-1B:Q6_K # Run inference directly in the terminal: llama cli -hf AIIT-Threshold/Tessera-1B:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AIIT-Threshold/Tessera-1B:Q6_K # Run inference directly in the terminal: llama cli -hf AIIT-Threshold/Tessera-1B:Q6_K
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf AIIT-Threshold/Tessera-1B:Q6_K # Run inference directly in the terminal: ./llama-cli -hf AIIT-Threshold/Tessera-1B:Q6_K
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf AIIT-Threshold/Tessera-1B:Q6_K # Run inference directly in the terminal: ./build/bin/llama-cli -hf AIIT-Threshold/Tessera-1B:Q6_K
Use Docker
docker model run hf.co/AIIT-Threshold/Tessera-1B:Q6_K
- LM Studio
- Jan
- Ollama
How to use AIIT-Threshold/Tessera-1B with Ollama:
ollama run hf.co/AIIT-Threshold/Tessera-1B:Q6_K
- Unsloth Studio
How to use AIIT-Threshold/Tessera-1B with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AIIT-Threshold/Tessera-1B to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for AIIT-Threshold/Tessera-1B to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for AIIT-Threshold/Tessera-1B to start chatting
- Docker Model Runner
How to use AIIT-Threshold/Tessera-1B with Docker Model Runner:
docker model run hf.co/AIIT-Threshold/Tessera-1B:Q6_K
- Lemonade
How to use AIIT-Threshold/Tessera-1B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AIIT-Threshold/Tessera-1B:Q6_K
Run and chat with the model
lemonade run user.Tessera-1B-Q6_K
List all available models
lemonade list
- Atomic Chat
Document GGUF files: base-model-only, needs patched llama.cpp, raw-completion usage
Browse files
USAGE.md
CHANGED
|
@@ -54,3 +54,26 @@ The **base** is a base model: fluent, drifts, loops — post-train it.
|
|
| 54 |
The **adapters** are small hand-made SFT stages over ~1,031 human-written examples;
|
| 55 |
see the model card for their personalities. This is a 1B: charming, honest about
|
| 56 |
what it doesn't know, and not a reasoning engine. 12×7 may not be 84. That's the deal.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
The **adapters** are small hand-made SFT stages over ~1,031 human-written examples;
|
| 55 |
see the model card for their personalities. This is a 1B: charming, honest about
|
| 56 |
what it doesn't know, and not a reasoning engine. 12×7 may not be 84. That's the deal.
|
| 57 |
+
|
| 58 |
+
## GGUF (llama.cpp)
|
| 59 |
+
|
| 60 |
+
Two GGUF conversions of the **base** pretrain checkpoint (not the `lora.py` chat adapters — those aren't merged in) are provided under `gguf/`:
|
| 61 |
+
|
| 62 |
+
| File | Size | Use |
|
| 63 |
+
|---|---|---|
|
| 64 |
+
| `gguf/tessera-1b-Q6_K.gguf` | ~883 MB | recommended — Q6_K quant, ~6.6 bits/weight |
|
| 65 |
+
| `gguf/tessera-1b-f16.gguf` | ~2.2 GB | full-precision source, requantize to other sizes from this |
|
| 66 |
+
|
| 67 |
+
### Important: requires a patched llama.cpp build
|
| 68 |
+
|
| 69 |
+
ProtoGPT uses **RMSNorm** (see `model.py`), but llama.cpp's stock `gpt2` architecture — the correct match for everything else here (learned absolute position embeddings, GELU MLP, fused-QKV full attention, no biases) — hard-codes classic mean-centered LayerNorm. **A GGUF built for this model will not load correctly on unpatched/upstream llama.cpp, Ollama, LM Studio, or any other tool bundling stock llama.cpp** — it'll run without erroring but compute the wrong norm.
|
| 70 |
+
|
| 71 |
+
`tessera1b-rmsnorm-gpt2-arch.patch` is included in this repo: a 6-line diff against `llama.cpp/src/models/gpt2.cpp` that swaps the 3 `build_norm(..., LLM_NORM, ...)` call sites to `LLM_NORM_RMS` and makes the (unused, always-zero) norm bias tensors optional. Apply it to a fresh llama.cpp checkout, rebuild, then use that binary for both `llama-quantize` and inference.
|
| 72 |
+
|
| 73 |
+
### Usage — raw completion, not chat
|
| 74 |
+
|
| 75 |
+
This is a base model. Use `llama-completion` (plain next-token continuation), not `llama-cli`'s interactive/conversation mode — that auto-applies a chat template (ChatML by default) this model was never trained on and produces garbage as a result.
|
| 76 |
+
|
| 77 |
+
```
|
| 78 |
+
./llama-completion -m tessera-1b-Q6_K.gguf -p "Your prompt here" -n 100 --temp 0.3
|
| 79 |
+
```
|