---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- tinybrainbot
- llama
- small-language-model
- gguf
base_model: nkthebass/tinybrainbot-303mV2-base
---

# TinyBrainBot 303M V2 — Instruct

A **~303M parameter** instruction-tuned language model, trained from scratch on a single 2×GPU workstation. This is the chat/instruct variant; the pretrained foundation is [nkthebass/tinybrainbot-303mV2-base](https://huggingface.co/nkthebass/tinybrainbot-303mV2-base).

It's a genuinely small model — think **GPT-2-small class** — built as a from-scratch LLM project. It knows a fair amount of factual trivia, holds a short chat, greets, gives simple advice, and does **basic add/subtract arithmetic with shown work**. It is *not* a general assistant and will confidently hallucinate; see Limitations.

## Architecture

Llama-family (RoPE, RMSNorm, SwiGLU, GQA), tied embeddings.

| | |
|---|---|
| Parameters | ~303M |
| Hidden size | 1024 |
| Layers | 24 |
| Attention heads | 16 (4 KV heads, GQA) |
| Head dim | 64 |
| FFN size | 2816 (SwiGLU) |
| Vocab | 32,000 (SentencePiece BPE) |
| Context length | 1024 |
| RoPE theta | 10000 |

## Chat format

Single-token role markers, EOS = `<|end|>`:

```
<|user|> {message} <|end|> <|assistant|>
```

The chat template is embedded in `tokenizer_config.json` (and in the GGUFs), so `apply_chat_template` / `llama-server --jinja` handle it for you.

## Usage

### transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
msgs = [{"role": "user", "content": "What is the capital of France?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))   # -> Paris.
```
**Tip:** use **greedy** (`do_sample=False`) for factual/arithmetic queries — at this size, sampling wanders. It is also **very** sensitive to typos (a misspelled "captial" derails it).

### llama.cpp / Jan / LM Studio
GGUF files (`F16`, `Q8_0`) are included in this repo. In llama.cpp: `llama-server -m tinybrainbot-303mV2-instruct-Q8_0.gguf --jinja`.

## Benchmarks

Standard log-likelihood multiple-choice (lm-eval style, n=200, seed 42). Headline = acc_norm (HellaSwag/ARC/OpenBookQA), acc (WinoGrande/MMLU):

| Benchmark | This model | GPT-2-124M | Pythia-410M | random |
|---|---:|---:|---:|---:|
| ARC-Easy | **46.0** | 44 | 52 | 25 |
| ARC-Challenge | **27.0** | 22 | 24 | 25 |
| OpenBookQA | **29.0** | 29 | 30 | 25 |
| HellaSwag | 26.0 | 31 | 34 | 25 |
| WinoGrande | 47.0 | 52 | 53 | 50 |
| MMLU | 21.0 | 26 | 25 | 25 |

Real signal is on the QA benches (beats GPT-2-124M on ARC-Easy and beats both GPT-2-124M and Pythia-410M on ARC-Challenge). HellaSwag / MMLU / WinoGrande sit at the random floor — the size ceiling of a 303M.

**Arithmetic:** trained with a verified "show-your-work" math set, so it does **addition and subtraction correctly with column steps** (e.g. `462 + 23` → shows the ones/tens/hundreds and answers 485). **Multiplication and division are still wrong** — it attempts the scratchpad but the digits are off.

## Training

Pretrained (see the base model) then SFT'd. SFT mix: `smoltalk`, a synthetic instruction set, multi-step reasoning word problems, verified arithmetic-with-work, multi-turn dialogues, and greetings. Precision fp16, WSD schedule, DDP on 2× Tesla P100.

## Limitations

- **It hallucinates.** It's 303M — it does not reliably know facts beyond common trivia, and states wrong answers with full confidence.
- **Mul/div arithmetic is broken** (add/sub is fine).
- **English only**, 1024-token context, **greedy-decoding recommended**, typo-fragile.
- No safety tuning / RLHF. Do not deploy in anything user-facing without your own guardrails.

## License

Apache-2.0. Free to use and build on — attribution appreciated. Trained on public/open datasets (FineWeb-Edu, Wikipedia, TinyStories, OpenWebText2, plus synthetic distillation data); please respect the licenses of those upstream sources.