---
license: mit
tags:
  - llama
  - text-generation
  - from-scratch
  - hobby-project
language:
  - en
---

# Microllm

-- Use the non GGUF version for now. --

A small (768-dim, 22-layer, ~50260 vocab) decoder-only
transformer, pretrained from scratch on streaming FineWeb-Edu and instruction
fine-tuned on Dolly-15k + No Robots. This is an independent hobbyist project,
not affiliated with any AI lab - trained end-to-end on a single rented GPU.

Architecturally this is a standard Llama-style model (RMSNorm, RoPE, SwiGLU,
tied embeddings), so it loads directly with `transformers`:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("MLVXN/microllm")
model = AutoModelForCausalLM.from_pretrained("MLVXN/microllm")

prompt = "<|user|>What's your name?<|assistant|>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8, top_k=50)
print(tok.decode(out[0]))
```

## Chat format
This model was fine-tuned on a simple turn structure, not a full chat
template - wrap each user message like this:

```
<|user|>{message}<|assistant|>
```

Generation should stop at `<|end|>` (id 50259).

## Checkpoint info
- Fine-tuning phase reached: `no_robots`
- Fine-tuning global step: `4845`
- seq_len: 1024

## Known limitations
This is a ~150M-parameter model trained on a modest compute budget. Expect
coherent grammar and conversational fluency, but unreliable facts and no real
multi-step reasoning - that's the honest ceiling for this size/budget, not a
bug. It reliably knows its own identity (name/creator) because that was
explicitly trained in, separately from general knowledge quality.

## Running in LM Studio / llama.cpp / Ollama
If a `.gguf` file is included in this repo, download it directly in LM Studio
via its Hugging Face search, or point llama.cpp / Ollama at the file. If no
`.gguf` is present, convert it yourself:

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/microllm --outfile microllm.gguf --outtype f16
# optional: quantize for a smaller file
./llama-quantize microllm.gguf microllm.Q4_K_M.gguf Q4_K_M
```