README.md

---
license: gemma
base_model: micrictor/gemma-3-270m-it-ft-bash
tags:
- gemma3
- text-generation
- sft
- bash
- shell
- cli
- conversational
- distilled
- trl
language:
- en
pipeline_tag: text-generation
library_name: transformers
---

# shellminator-270m — a tiny natural-language → bash command model

**shellminator** is a terminal-native bash command assistant: type `sm "copy jpgs to backup"` and a 270M model suggests a single bash command; press **Enter** to run it in your shell, `e` to edit, `r` to refine, `c` to cancel. This repo is the model behind it — a Gemma-3-270M fine-tuned to translate a short natural-language request into **one** bash command.

> **Status: work-in-progress (pre-final checkpoint).** The current weights were trained on the 30K qwen-distilled dataset. A combined dataset (emirkaan 6K + 30K qwen ≈ 36K) is the intended next checkpoint — see [Training data](#training-data) and [Evaluation](#evaluation).

## Intended use

Suggesting a single bash command from a short natural-language request, with a human in the loop:

```
$ sm "kill the process listening on port 8080"
> kill -9 $(lsof -t -i :8080)
  [Enter] run  [e] edit  [r] refine  [c] cancel
```

The model **only suggests** — the `sm` UI always shows the command for review before it runs. It is not an autonomous agent.

## How to use

For the `sm` tool the model is served as a **GGUF** via `llama.cpp` (a pre-quantized ~250MB Q4_K_M GGUF is shipped; end users never need torch). With `transformers`:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok  = AutoTokenizer.from_pretrained("ISB369/shellminator-270m-bash-distilled")
model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-270m-bash-distilled")
SYS = ("You are a helpful assistant that translates natural language to bash commands.\n"
       "Context: cwd=/home/user, system=Linux x86_64, shell=bash.\n"
       "Reply with a single bash command only. No explanation, no markdown fences.")
msgs = [{"role":"system","content":SYS},
        {"role":"user","content":"Generate single Bash command: list the 10 biggest files in cwd"}]
out = model.generate(**tok.apply_chat_template(msgs, tokenize=True, return_tensors="pt"),
                     do_sample=False, max_new_tokens=128)
print(tok.decode(out[0], skip_special_tokens=True).split("assistant\n")[-1].strip())
```

Use **greedy** decoding (`do_sample=False` / `temperature=0`) — the task is deterministic; sampling hurts it.

## Training

- **Base:** [`micrictor/gemma-3-270m-it-ft-bash`](https://huggingface.co/micrictor/gemma-3-270m-it-ft-bash) (Gemma-3 270M, instruction-tuned + bash fine-tune).
- **Method:** full supervised fine-tune with [TRL](https://github.com/huggingface/trl) `SFTTrainer`; fp16-AMP over fp32; 3 epochs; per-epoch held-out eval with `load_best_model_at_end` (ships the best-generalizing epoch, not the most-trained); effective batch 16; lr 1e-4, cosine. Trained on a free Google Colab T4.
- **Orphan-token fix:** the base ships an orphan `<image_soft_token>` (id 262144) with no embedding row; training calls `resize_token_embeddings(len(tokenizer))` so it gets a real trained embedding — otherwise the GGUF convert fails on the embedding-contract violation (`max(token_id) < vocab_size`).

## Training data

The **shellminator distillation pipeline**: a strong cloud teacher (`qwen3.5:397b-cloud` via ollama `/api/chat`, `think:false`) generates varied natural-language requests, labels each with a single bash command, then filters with `bash -n` (syntax) + an LLM judge (correctness) and dedupes by `(nl, cmd)`. 22 dev-tool categories: file ops, text processing, processes, networking, git (basic + advanced), docker, kubernetes, build tools, package managers, systemd, tmux, editing, cloud CLI, monitoring, permissions, disk, archives, ssh, system info, pipes/xargs, scheduling.

Datasets:
- [`ISB369/shellminator-bash-dataset`](https://huggingface.co/datasets/ISB369/shellminator-bash-dataset) — the 30K qwen-distilled (parts 000+001).
- [`ISB369/shellminator-bash-clean`](https://huggingface.co/datasets/ISB369/shellminator-bash-clean) — 10K combined (emirkaan 6K + qwen 4K).
- [`ISB369/shellminator-bash-combined`](https://huggingface.co/datasets/ISB369/shellminator-bash-combined) — **36K combined** (emirkaan 6K + 30K qwen, deduped, `bash -n`-filtered). **Intended for the final retrain.**

> The current checkpoint was trained on the 30K qwen **alone** (no emirkaan). The combined-36K retrain adds emirkaan's "echo the user's literal, specific real command" style back — see Evaluation.

## Evaluation

25 held-out prompts across all 22 categories, judged by **5x majority vote** of the cloud teacher (`qwen3.5:397b-cloud`, greedy) + `bash -n` validity. (Single-vote judging was too noisy; majority smooths the flip-flopping.)

| checkpoint | data | correctness (judge 5x) | validity (`bash -n`) |
|---|---|---|---|
| Aug 11 | 10K clean (emirkaan + qwen 4K) | 12/25 (48%) | 100% |
| **Aug 15 (current)** | **30K qwen-only** | **11/25 (44%)** | **92%** |
| next | 36K combined | TBD | TBD |

**What the 30K learned (win):** `kubectl get pods -n production` (previously a `ps` pipeline — the kubernetes category was absorbed); `unzip backup.zip -d restore`.

**Regressions vs the 10K-clean model:** the 30K qwen data leans generic/placeholder, so the model now emits e.g. `ssh-copy-id user@remote_host` instead of echoing the user's literal `192.168.1.10`, and is sloppier on precise tasks. This is the *data style > scale* lesson — more coverage, but the style shift cost precision. **The combined-36K retrain is designed to fix this** (emirkaan restores literal-echoing while keeping the dev-tool coverage).

**Capacity ceiling (not a data issue):** the model sometimes mangles complex syntax (e.g. `awk` with nested quotes, an unbalanced paren). The dataset has **zero** invalid-syntax labels, so these are 270M *generation* limits, not bad data. A bigger model or constrained decoding (llama.cpp GBNF grammars) is the fix there.

## Limitations

- **270M capacity** — strong on common single-line commands; mangles complex multi-arg/nested syntax; no multi-step reasoning.
- **Can be wrong or destructive** — may suggest an incorrect or dangerous command (wrong flags, `rm`/`kill` with wrong targets). **Always review before running.**
- **English requests, Linux/bash, x86_64** only.
- **Literal-precision** is the current (30K) checkpoint's weak spot (placeholders); the combined retrain targets this.

## Safety / ethics

The model suggests shell commands that can modify or delete data. The `sm` tool is built so a command is **never** run without a human pressing Enter (shown for review; `e` to edit, `c` to cancel). Do **not** wire this model into an autonomous executor. Treat every suggestion as untrusted until you have read it.

## Reproduce

`SFTTrainer` on the combined dataset (`HF_DATASET_REPO=ISB369/shellminator-bash-combined`), Colab T4, the config above. Eval: `train/eval.py` (25 prompts, 5x majority judge). Generation pipeline: `scripts/generate_dataset.py` (teacher → NL → bash → `bash -n` + judge → dedup; resumable; HF upload). Merge: `scripts/merge_combined.py`.

---

*Built as a distillation exercise: a big cloud model's behavior becomes a tiny local model's training data, so a ~250MB model can suggest a bash command in under a second on CPU — privately, with no GPU.*