--- license: gemma base_model: micrictor/gemma-3-270m-it-ft-bash tags: - gemma3 - text-generation - sft - bash - shell - cli - conversational - distilled - trl language: - en pipeline_tag: text-generation library_name: transformers --- # shellminator-270m — a tiny natural-language → bash command model **shellminator** is a terminal-native bash command assistant: type `sm "copy jpgs to backup"` and a 270M model suggests a single bash command; press **Enter** to run it in your shell, `e` to edit, `r` to refine, `c` to cancel. This repo is the model behind it — a Gemma-3-270M fine-tuned to translate a short natural-language request into **one** bash command. > **Status: work-in-progress (pre-final checkpoint).** The current weights were trained on the 30K qwen-distilled dataset. A combined dataset (emirkaan 6K + 30K qwen ≈ 36K) is the intended next checkpoint — see [Training data](#training-data) and [Evaluation](#evaluation). ## Intended use Suggesting a single bash command from a short natural-language request, with a human in the loop: ``` $ sm "kill the process listening on port 8080" > kill -9 $(lsof -t -i :8080) [Enter] run [e] edit [r] refine [c] cancel ``` The model **only suggests** — the `sm` UI always shows the command for review before it runs. It is not an autonomous agent. ## How to use For the `sm` tool the model is served as a **GGUF** via `llama.cpp` (a pre-quantized ~250MB Q4_K_M GGUF is shipped; end users never need torch). With `transformers`: ```python from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("ISB369/shellminator-270m-bash-distilled") model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-270m-bash-distilled") SYS = ("You are a helpful assistant that translates natural language to bash commands.\n" "Context: cwd=/home/user, system=Linux x86_64, shell=bash.\n" "Reply with a single bash command only. No explanation, no markdown fences.") msgs = [{"role":"system","content":SYS}, {"role":"user","content":"Generate single Bash command: list the 10 biggest files in cwd"}] out = model.generate(**tok.apply_chat_template(msgs, tokenize=True, return_tensors="pt"), do_sample=False, max_new_tokens=128) print(tok.decode(out[0], skip_special_tokens=True).split("assistant\n")[-1].strip()) ``` Use **greedy** decoding (`do_sample=False` / `temperature=0`) — the task is deterministic; sampling hurts it. ## Training - **Base:** [`micrictor/gemma-3-270m-it-ft-bash`](https://huggingface.co/micrictor/gemma-3-270m-it-ft-bash) (Gemma-3 270M, instruction-tuned + bash fine-tune). - **Method:** full supervised fine-tune with [TRL](https://github.com/huggingface/trl) `SFTTrainer`; fp16-AMP over fp32; 3 epochs; per-epoch held-out eval with `load_best_model_at_end` (ships the best-generalizing epoch, not the most-trained); effective batch 16; lr 1e-4, cosine. Trained on a free Google Colab T4. - **Orphan-token fix:** the base ships an orphan `` (id 262144) with no embedding row; training calls `resize_token_embeddings(len(tokenizer))` so it gets a real trained embedding — otherwise the GGUF convert fails on the embedding-contract violation (`max(token_id) < vocab_size`). ## Training data The **shellminator distillation pipeline**: a strong cloud teacher (`qwen3.5:397b-cloud` via ollama `/api/chat`, `think:false`) generates varied natural-language requests, labels each with a single bash command, then filters with `bash -n` (syntax) + an LLM judge (correctness) and dedupes by `(nl, cmd)`. 22 dev-tool categories: file ops, text processing, processes, networking, git (basic + advanced), docker, kubernetes, build tools, package managers, systemd, tmux, editing, cloud CLI, monitoring, permissions, disk, archives, ssh, system info, pipes/xargs, scheduling. Datasets: - [`ISB369/shellminator-bash-dataset`](https://huggingface.co/datasets/ISB369/shellminator-bash-dataset) — the 30K qwen-distilled (parts 000+001). - [`ISB369/shellminator-bash-clean`](https://huggingface.co/datasets/ISB369/shellminator-bash-clean) — 10K combined (emirkaan 6K + qwen 4K). - [`ISB369/shellminator-bash-combined`](https://huggingface.co/datasets/ISB369/shellminator-bash-combined) — **36K combined** (emirkaan 6K + 30K qwen, deduped, `bash -n`-filtered). **Intended for the final retrain.** > The current checkpoint was trained on the 30K qwen **alone** (no emirkaan). The combined-36K retrain adds emirkaan's "echo the user's literal, specific real command" style back — see Evaluation. ## Evaluation 25 held-out prompts across all 22 categories, judged by **5x majority vote** of the cloud teacher (`qwen3.5:397b-cloud`, greedy) + `bash -n` validity. (Single-vote judging was too noisy; majority smooths the flip-flopping.) | checkpoint | data | correctness (judge 5x) | validity (`bash -n`) | |---|---|---|---| | Aug 11 | 10K clean (emirkaan + qwen 4K) | 12/25 (48%) | 100% | | **Aug 15 (current)** | **30K qwen-only** | **11/25 (44%)** | **92%** | | next | 36K combined | TBD | TBD | **What the 30K learned (win):** `kubectl get pods -n production` (previously a `ps` pipeline — the kubernetes category was absorbed); `unzip backup.zip -d restore`. **Regressions vs the 10K-clean model:** the 30K qwen data leans generic/placeholder, so the model now emits e.g. `ssh-copy-id user@remote_host` instead of echoing the user's literal `192.168.1.10`, and is sloppier on precise tasks. This is the *data style > scale* lesson — more coverage, but the style shift cost precision. **The combined-36K retrain is designed to fix this** (emirkaan restores literal-echoing while keeping the dev-tool coverage). **Capacity ceiling (not a data issue):** the model sometimes mangles complex syntax (e.g. `awk` with nested quotes, an unbalanced paren). The dataset has **zero** invalid-syntax labels, so these are 270M *generation* limits, not bad data. A bigger model or constrained decoding (llama.cpp GBNF grammars) is the fix there. ## Limitations - **270M capacity** — strong on common single-line commands; mangles complex multi-arg/nested syntax; no multi-step reasoning. - **Can be wrong or destructive** — may suggest an incorrect or dangerous command (wrong flags, `rm`/`kill` with wrong targets). **Always review before running.** - **English requests, Linux/bash, x86_64** only. - **Literal-precision** is the current (30K) checkpoint's weak spot (placeholders); the combined retrain targets this. ## Safety / ethics The model suggests shell commands that can modify or delete data. The `sm` tool is built so a command is **never** run without a human pressing Enter (shown for review; `e` to edit, `c` to cancel). Do **not** wire this model into an autonomous executor. Treat every suggestion as untrusted until you have read it. ## Reproduce `SFTTrainer` on the combined dataset (`HF_DATASET_REPO=ISB369/shellminator-bash-combined`), Colab T4, the config above. Eval: `train/eval.py` (25 prompts, 5x majority judge). Generation pipeline: `scripts/generate_dataset.py` (teacher → NL → bash → `bash -n` + judge → dedup; resumable; HF upload). Merge: `scripts/merge_combined.py`. --- *Built as a distillation exercise: a big cloud model's behavior becomes a tiny local model's training data, so a ~250MB model can suggest a bash command in under a second on CPU — privately, with no GPU.*