Instructions to use nkthebass/tinybrainbot-303mV2-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nkthebass/tinybrainbot-303mV2-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nkthebass/tinybrainbot-303mV2-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use nkthebass/tinybrainbot-303mV2-instruct with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-303mV2-instruct:F16
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-303mV2-instruct:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nkthebass/tinybrainbot-303mV2-instruct:F16
# Run inference directly in the terminal:
llama cli -hf nkthebass/tinybrainbot-303mV2-instruct:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nkthebass/tinybrainbot-303mV2-instruct:F16
# Run inference directly in the terminal:
./llama-cli -hf nkthebass/tinybrainbot-303mV2-instruct:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nkthebass/tinybrainbot-303mV2-instruct:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nkthebass/tinybrainbot-303mV2-instruct:F16

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-303mV2-instruct:F16

LM Studio
Jan

vLLM

How to use nkthebass/tinybrainbot-303mV2-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nkthebass/tinybrainbot-303mV2-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-303mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nkthebass/tinybrainbot-303mV2-instruct:F16

SGLang

How to use nkthebass/tinybrainbot-303mV2-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nkthebass/tinybrainbot-303mV2-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-303mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nkthebass/tinybrainbot-303mV2-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nkthebass/tinybrainbot-303mV2-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use nkthebass/tinybrainbot-303mV2-instruct with Ollama:
```
ollama run hf.co/nkthebass/tinybrainbot-303mV2-instruct:F16
```

Unsloth Studio

How to use nkthebass/tinybrainbot-303mV2-instruct with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nkthebass/tinybrainbot-303mV2-instruct to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nkthebass/tinybrainbot-303mV2-instruct to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nkthebass/tinybrainbot-303mV2-instruct to start chatting

Docker Model Runner
How to use nkthebass/tinybrainbot-303mV2-instruct with Docker Model Runner:
```
docker model run hf.co/nkthebass/tinybrainbot-303mV2-instruct:F16
```

Lemonade

How to use nkthebass/tinybrainbot-303mV2-instruct with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nkthebass/tinybrainbot-303mV2-instruct:F16

Run and chat with the model

lemonade run user.tinybrainbot-303mV2-instruct-F16

List all available models

lemonade list

Atomic Chat

tinybrainbot-303mV2-instruct / README.md

nkthebass

TinyBrainBot 303M V2 (base+instruct, safetensors + GGUF)

7d16ca4 verified 10 days ago

preview code

Raw

History Blame Contribute Delete

4.27 kB

	---
	license: apache-2.0
	language:
	- en
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- tinybrainbot
	- llama
	- small-language-model
	- gguf
	base_model: nkthebass/tinybrainbot-303mV2-base
	---

	# TinyBrainBot 303M V2 — Instruct

	A ~303M parameter instruction-tuned language model, trained from scratch on a single 2×GPU workstation. This is the chat/instruct variant; the pretrained foundation is [nkthebass/tinybrainbot-303mV2-base](https://huggingface.co/nkthebass/tinybrainbot-303mV2-base).

	It's a genuinely small model — think GPT-2-small class — built as a from-scratch LLM project. It knows a fair amount of factual trivia, holds a short chat, greets, gives simple advice, and does basic add/subtract arithmetic with shown work. It is not a general assistant and will confidently hallucinate; see Limitations.

	## Architecture

	Llama-family (RoPE, RMSNorm, SwiGLU, GQA), tied embeddings.

	\| \| \|
	\|---\|---\|
	\| Parameters \| ~303M \|
	\| Hidden size \| 1024 \|
	\| Layers \| 24 \|
	\| Attention heads \| 16 (4 KV heads, GQA) \|
	\| Head dim \| 64 \|
	\| FFN size \| 2816 (SwiGLU) \|
	\| Vocab \| 32,000 (SentencePiece BPE) \|
	\| Context length \| 1024 \|
	\| RoPE theta \| 10000 \|

	## Chat format

	Single-token role markers, EOS = `<\|end\|>`:

	```
	<\|user\|> {message} <\|end\|> <\|assistant\|>
	```

	The chat template is embedded in `tokenizer_config.json` (and in the GGUFs), so `apply_chat_template` / `llama-server --jinja` handle it for you.

	## Usage

	### transformers
	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer
	tok = AutoTokenizer.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
	model = AutoModelForCausalLM.from_pretrained("nkthebass/tinybrainbot-303mV2-instruct")
	msgs = [{"role": "user", "content": "What is the capital of France?"}]
	ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
	out = model.generate(ids, max_new_tokens=64, do_sample=False)
	print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)) # -> Paris.
	```
	Tip: use greedy (`do_sample=False`) for factual/arithmetic queries — at this size, sampling wanders. It is also very sensitive to typos (a misspelled "captial" derails it).

	### llama.cpp / Jan / LM Studio
	GGUF files (`F16`, `Q8_0`) are included in this repo. In llama.cpp: `llama-server -m tinybrainbot-303mV2-instruct-Q8_0.gguf --jinja`.

	## Benchmarks

	Standard log-likelihood multiple-choice (lm-eval style, n=200, seed 42). Headline = acc_norm (HellaSwag/ARC/OpenBookQA), acc (WinoGrande/MMLU):

	\| Benchmark \| This model \| GPT-2-124M \| Pythia-410M \| random \|
	\|---\|---:\|---:\|---:\|---:\|
	\| ARC-Easy \| 46.0 \| 44 \| 52 \| 25 \|
	\| ARC-Challenge \| 27.0 \| 22 \| 24 \| 25 \|
	\| OpenBookQA \| 29.0 \| 29 \| 30 \| 25 \|
	\| HellaSwag \| 26.0 \| 31 \| 34 \| 25 \|
	\| WinoGrande \| 47.0 \| 52 \| 53 \| 50 \|
	\| MMLU \| 21.0 \| 26 \| 25 \| 25 \|

	Real signal is on the QA benches (beats GPT-2-124M on ARC-Easy and beats both GPT-2-124M and Pythia-410M on ARC-Challenge). HellaSwag / MMLU / WinoGrande sit at the random floor — the size ceiling of a 303M.

	Arithmetic: trained with a verified "show-your-work" math set, so it does addition and subtraction correctly with column steps (e.g. `462 + 23` → shows the ones/tens/hundreds and answers 485). Multiplication and division are still wrong — it attempts the scratchpad but the digits are off.

	## Training

	Pretrained (see the base model) then SFT'd. SFT mix: `smoltalk`, a synthetic instruction set, multi-step reasoning word problems, verified arithmetic-with-work, multi-turn dialogues, and greetings. Precision fp16, WSD schedule, DDP on 2× Tesla P100.

	## Limitations

	- It hallucinates. It's 303M — it does not reliably know facts beyond common trivia, and states wrong answers with full confidence.
	- Mul/div arithmetic is broken (add/sub is fine).
	- English only, 1024-token context, greedy-decoding recommended, typo-fragile.
	- No safety tuning / RLHF. Do not deploy in anything user-facing without your own guardrails.

	## License

	Apache-2.0. Free to use and build on — attribution appreciated. Trained on public/open datasets (FineWeb-Edu, Wikipedia, TinyStories, OpenWebText2, plus synthetic distillation data); please respect the licenses of those upstream sources.