File size: 2,250 Bytes
1893351 64e5b04 1893351 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
license: mit
tags:
- llama
- text-generation
- from-scratch
- hobby-project
language:
- en
---
# Microllm
-- Use the non GGUF version for now. --
A small (768-dim, 22-layer, ~50260 vocab) decoder-only
transformer, pretrained from scratch on streaming FineWeb-Edu and instruction
fine-tuned on Dolly-15k + No Robots. This is an independent hobbyist project,
not affiliated with any AI lab - trained end-to-end on a single rented GPU.
Architecturally this is a standard Llama-style model (RMSNorm, RoPE, SwiGLU,
tied embeddings), so it loads directly with `transformers`:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MLVXN/microllm")
model = AutoModelForCausalLM.from_pretrained("MLVXN/microllm")
prompt = "<|user|>What's your name?<|assistant|>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8, top_k=50)
print(tok.decode(out[0]))
```
## Chat format
This model was fine-tuned on a simple turn structure, not a full chat
template - wrap each user message like this:
```
<|user|>{message}<|assistant|>
```
Generation should stop at `<|end|>` (id 50259).
## Checkpoint info
- Fine-tuning phase reached: `no_robots`
- Fine-tuning global step: `4845`
- seq_len: 1024
## Known limitations
This is a ~150M-parameter model trained on a modest compute budget. Expect
coherent grammar and conversational fluency, but unreliable facts and no real
multi-step reasoning - that's the honest ceiling for this size/budget, not a
bug. It reliably knows its own identity (name/creator) because that was
explicitly trained in, separately from general knowledge quality.
## Running in LM Studio / llama.cpp / Ollama
If a `.gguf` file is included in this repo, download it directly in LM Studio
via its Hugging Face search, or point llama.cpp / Ollama at the file. If no
`.gguf` is present, convert it yourself:
```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/microllm --outfile microllm.gguf --outtype f16
# optional: quantize for a smaller file
./llama-quantize microllm.gguf microllm.Q4_K_M.gguf Q4_K_M
```
|