README.md

---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- bash
- shell
- linux
- cli
- code
- small-language-model
pipeline_tag: text-generation
model-index:
- name: Dwarf-15M-Instruct
  results: []
---

# Dwarf-15M-Instruct

A **15.54M parameter** shell/bash specialist language model that translates natural language into Linux commands.

## Quick Start

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)

prompt = "<|user|>\nFind all Python files modified in the last 3 days\n<|end|>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
# → find . -name '*.py' -mtime -3
```

## What It Does

Dwarf-15M takes natural language descriptions of tasks and produces the corresponding Linux/bash command:

| Prompt | Output |
|---|---|
| Show current date | `date` |
| List files | `ls` |
| Kill process 1234 | `kill 1234` |
| Delete all .tmp files in current directory | `rm ./*.tmp` |
| Compress the /home/user/project folder | `tar -czf project.tar.gz /home/user/project/` |
| Check if port 8080 is in use | `ss -tlnp \| grep 8080` |
| Restart the nginx service | `sudo systemctl restart nginx` |
| Find all files containing TODO | `grep -rl 'TODO' .` |
| Change owner of /var/www to www-data | `sudo chown -R www-data:www-data /var/www` |
| Run script.sh in background and log output | `nohup ./script.sh > log.txt 2>&1 &` |
| Find and replace foo with bar in config.txt | `sed -i 's/foo/bar/g' config.txt` |
| What does chmod 755 do? | chmod 755 sets read+write+execute for owner, read+execute for group and others. |
| Write a bash function that counts lines | `count_lines() { wc -l < "$1"; }` |

## Architecture

| Parameter | Value |
|---|---|
| Parameters | 15.54M |
| Layers | 12 |
| Hidden dim | 320 |
| Attention | GQA (5 query, 1 KV head) |
| FFN | SwiGLU (d_ff=864) |
| Normalization | RMSNorm |
| Positional | RoPE (θ=10000) |
| Vocabulary | 8,202 (DwarfGoToken) |
| Max sequence | 2,048 |
| Weight tying | Yes (embed ↔ lm_head) |

## Training

**Pretraining:** 21.6B tokens (ratio 1,390:1) on 11 datasets:
- Shell/bash: The Stack (shell, batchfile), GunA-SD/bash_code — 38.5%
- Code: The Stack (Python, C), CodeFeedback — 39.1%
- Instructions: ShellLife (52K NL→command), rlvr-code-data-bash (133K problems) — 11%
- English: helpful-instructions, FineWeb — 10.3%
- CoT: Magpie-Reasoning — 1.1%

**SFT:** 557 curated Linux command pairs, 5 epochs, lr=4e-5. Training time: 19 seconds.

**Tokenizer:** [DwarfGoToken](https://huggingface.co/ThingAI/DwarfGoToken) — 8,202 token BPE with syntax-aware pre-tokenization for shell operators (2>&1, &&, >>).

## Chat Template

```
<|user|>
Your question here
<|end|>
<|assistant|>
Model response here
<|end|>
```

## Intended Use

Dwarf-15M is designed as a **CLI assistant** that suggests commands for user review before execution. It is NOT a general-purpose chatbot. Best results on:
- Simple to medium Linux commands (file ops, process management, networking)
- Bash one-liners and short functions
- Command explanations ("What does chmod 755 do?")

## Limitations

- 15M parameters — cannot handle complex multi-step reasoning
- May produce incorrect commands for unusual or very specific requests
- Should **always** be used with human review before executing any suggested command
- English only
- Trained primarily on Ubuntu/Debian commands

## Hardware

- Pretrained on RTX 3070 (8GB VRAM) at ~127K tokens/sec
- Inference: runs on any hardware including CPU, ~30MB model size

## License

Apache 2.0

## Citation

```bibtex
@misc{dwarf15m2026,
  title={Dwarf-15M-Instruct: A Shell Specialist Language Model},
  author={ThingsAI},
  year={2026},
  url={https://huggingface.co/ThingAI/Dwarf-15M-Instruct}
}
```