---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
pretty_name: LissaNano-Instruct
tags:
- text-generation
- llama
- my-little-pony
- mlp
- fanfiction
- small-language-model
- from-scratch
- lissa
- roleplay
- rp
- character
- edge-deployment
model_card_authors:
- CastIronMind
---
# LissaNano-Instruct
## 1. Introduction
**The smallest My Little Pony language model we could find: 32 million parameters and 64 MB of weights.** This is the supervised checkpoint, tuned for character-driven conversation.
- **Trained from scratch.** No distilled weights and no borrowed backbone: a Llama architecture with a 16,384-token vocabulary, pretrained on a fandom corpus and then supervised on dialogue.
- **Built for character work.** The supervised stage uses roleplay and persona data, so the model spends its capacity on voice, character and scene.
- **Dialogue format is required.** The model expects the exact `` / `` / `` layout described in section 6.
- **Trained by one person on one consumer GPU.** No cluster, no budget, no institutional support.
## 2. Model Overview
| field | value |
|---|---|
| Type | Causal Language Model |
| Training stage | Pretraining from scratch, then supervised fine-tuning |
| Architecture | Llama |
| Parameters | 31,990,272 (32M) |
| Layers | 8 |
| Hidden size | 512 |
| Attention heads | 16 (2 key/value heads) |
| Head dimension | 32 |
| Intermediate size | 1,536 |
| Context length | 1,024 tokens |
| Vocabulary | 16,384 (byte-level BPE) |
| Tied embeddings | yes |
| Precision | bfloat16 |
| License | Apache 2.0 |
## 3. The Lissa family
| model | parameters | weights | best for |
|---|---:|---:|---|
| [LissaNano-Base](https://huggingface.co/CastIronMind/LissaNano-Base) | 32M | 64 MB | continued pretraining, research |
| [LissaNano-Instruct](https://huggingface.co/CastIronMind/LissaNano-Instruct) | 32M | 64 MB | phones, single-board computers, embedded use |
| [LissaBrat-Base](https://huggingface.co/CastIronMind/LissaBrat-Base) | 103M | 206 MB | continued pretraining, research |
| [LissaBrat-Instruct](https://huggingface.co/CastIronMind/LissaBrat-Instruct) | 103M | 206 MB | laptops, longer and more coherent text |
| [Lissa 1.5B Instruct](https://huggingface.co/CastIronMind/Lissa-1.5B-Instruct) | 1.45B | 2.9 GB | a full roleplay partner |
| [Lissa 1.5B Base](https://huggingface.co/CastIronMind/Lissa-1.5B-Base) | 1.45B | 2.9 GB | pretraining, research |
**GGUF quantizations:** [LissaNano-Instruct-GGUF](https://huggingface.co/CastIronMind/LissaNano-Instruct-GGUF) — 15 files from Q8_0 down to IQ2, each with measured perplexity against this bf16 checkpoint.
**Demo:** [Lissa Playground](https://huggingface.co/spaces/CastIronMind/Lissa-Playground) — compare quantizations side by side in the browser, no server.
## 4. Requirements
```
pip install torch transformers
```
No custom kernels and no exotic dependencies: this is a plain Llama checkpoint and loads with any recent `transformers`.
## 5. Quick start
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "CastIronMind/LissaNano-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16)
model.eval()
SYS, USR, AST, EOS = 2, 3, 4, 1
system = "You are Twilight Sparkle, a studious purple unicorn who loves books."
user = "Twilight, I found a strange book in the library."
ids = [SYS] + tok.encode(system, add_special_tokens=False) \
+ [USR] + tok.encode(user, add_special_tokens=False) + [AST]
x = torch.tensor([ids], dtype=torch.long)
out = model.generate(x, max_new_tokens=120, do_sample=True,
temperature=0.8, top_p=0.9, repetition_penalty=1.1,
pad_token_id=0, eos_token_id=EOS)
print(tok.decode(out[0][len(ids):], skip_special_tokens=True))
```
## 6. Prompt format
The model was supervised on exactly this layout:
```
{persona}{message}{reply}
```
Tags are concatenated without spaces. A non-empty system message is required: every supervised example carried one.
**Special tokens.** `0 `, `1 `, `2 `, `3 `, `4 `.
**Sampling.** Temperature 0.8, top_p 0.9, repetition penalty 1.1.
## 7. Data
**Pretraining.** Fandom prose and forums from the My Little Pony community, about 1 billion unique tokens, two epochs.
**Supervised fine-tuning.** Roleplay and persona dialogues with the `` / `` / `` layout, two epochs.
## 8. Evaluation
This checkpoint is tuned for dialogue, so held-out perplexity is reported on the base model card, where it is measured on the same frozen set for every model in the family: see [LissaNano-Base](https://huggingface.co/CastIronMind/LissaNano-Base).
What the supervised stage targets: character voice, in-persona dialogue, and stable formatting across turns.
## 9. Hardware and cost
| item | value |
|---|---|
| Training hardware | one consumer GPU |
| Authors | one person |
| Weights on disk | 64 MB (bfloat16) |
| License | Apache 2.0 |
## 10. Intended use
- roleplay bots and character agents in chat applications
- embedded and offline deployments: games, visual novels, toys
- experiments with character consistency in small models
- further fine-tuning on a specific character; the model retrains in minutes
## 11. License
Apache License 2.0. Use it, fine-tune it, quantize it, embed it, and publish your own versions.
- ПАЦАНЫ, это малышок, она генерирует текст, и это иногда даже похоже на смысл!
