Dwarf-15M-Instruct

A 15.54M parameter shell/bash specialist language model that translates natural language into Linux commands.

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ThingAI/Dwarf-15M-Instruct", trust_remote_code=True)

prompt = "<|user|>\nFind all Python files modified in the last 3 days\n<|end|>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
# → find . -name '*.py' -mtime -3

What It Does

Dwarf-15M takes natural language descriptions of tasks and produces the corresponding Linux/bash command:

Prompt	Output
Show current date	date
List files	ls
Kill process 1234	kill 1234
Delete all .tmp files in current directory	rm ./*.tmp
Compress the /home/user/project folder	tar -czf project.tar.gz /home/user/project/
Check if port 8080 is in use	ss -tlnp | grep 8080
Restart the nginx service	sudo systemctl restart nginx
Find all files containing TODO	grep -rl 'TODO' .
Change owner of /var/www to www-data	sudo chown -R www-data:www-data /var/www
Run script.sh in background and log output	nohup ./script.sh > log.txt 2>&1 &
Find and replace foo with bar in config.txt	sed -i 's/foo/bar/g' config.txt
What does chmod 755 do?	chmod 755 sets read+write+execute for owner, read+execute for group and others.
Write a bash function that counts lines	count_lines() { wc -l < "$1"; }

Architecture

Parameter	Value
Parameters	15.54M
Layers	12
Hidden dim	320
Attention	GQA (5 query, 1 KV head)
FFN	SwiGLU (d_ff=864)
Normalization	RMSNorm
Positional	RoPE (θ=10000)
Vocabulary	8,202 (DwarfGoToken)
Max sequence	2,048
Weight tying	Yes (embed ↔ lm_head)

Training

Pretraining: 21.6B tokens (ratio 1,390:1) on 11 datasets:

• Shell/bash: The Stack (shell, batchfile), GunA-SD/bash_code — 38.5%
• Code: The Stack (Python, C), CodeFeedback — 39.1%
• Instructions: ShellLife (52K NL→command), rlvr-code-data-bash (133K problems) — 11%
• English: helpful-instructions, FineWeb — 10.3%
• CoT: Magpie-Reasoning — 1.1%

SFT: 557 curated Linux command pairs, 5 epochs, lr=4e-5. Training time: 19 seconds.

Tokenizer: DwarfGoToken — 8,202 token BPE with syntax-aware pre-tokenization for shell operators (2>&1, &&, >>).

Chat Template

<|user|>
Your question here
<|end|>
<|assistant|>
Model response here
<|end|>

Intended Use

Dwarf-15M is designed as a CLI assistant that suggests commands for user review before execution. It is NOT a general-purpose chatbot. Best results on:

• Simple to medium Linux commands (file ops, process management, networking)
• Bash one-liners and short functions
• Command explanations ("What does chmod 755 do?")

Limitations

• 15M parameters — cannot handle complex multi-step reasoning
• May produce incorrect commands for unusual or very specific requests
• Should always be used with human review before executing any suggested command
• English only
• Trained primarily on Ubuntu/Debian commands

Hardware

• Pretrained on RTX 3070 (8GB VRAM) at ~127K tokens/sec
• Inference: runs on any hardware including CPU, ~30MB model size

License

Apache 2.0

Citation

@misc{dwarf15m2026,
  title={Dwarf-15M-Instruct: A Shell Specialist Language Model},
  author={ThingsAI},
  year={2026},
  url={https://huggingface.co/ThingAI/Dwarf-15M-Instruct}
}