Instructions to use ISB369/shellminator-270m-bash-distilled with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ISB369/shellminator-270m-bash-distilled with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ISB369/shellminator-270m-bash-distilled")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ISB369/shellminator-270m-bash-distilled")
model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-270m-bash-distilled", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ISB369/shellminator-270m-bash-distilled with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ISB369/shellminator-270m-bash-distilled"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISB369/shellminator-270m-bash-distilled",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ISB369/shellminator-270m-bash-distilled

SGLang

How to use ISB369/shellminator-270m-bash-distilled with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ISB369/shellminator-270m-bash-distilled" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISB369/shellminator-270m-bash-distilled",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ISB369/shellminator-270m-bash-distilled" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISB369/shellminator-270m-bash-distilled",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ISB369/shellminator-270m-bash-distilled with Docker Model Runner:
```
docker model run hf.co/ISB369/shellminator-270m-bash-distilled
```

shellminator-270m-bash-distilled / README.md

ISB369

Upload Gemma3ForCausalLM

5d9876a verified 4 days ago

preview code

Raw

History Blame Contribute Delete

7.43 kB

	---
	license: gemma
	base_model: micrictor/gemma-3-270m-it-ft-bash
	tags:
	- gemma3
	- text-generation
	- sft
	- bash
	- shell
	- cli
	- conversational
	- distilled
	- trl
	language:
	- en
	pipeline_tag: text-generation
	library_name: transformers
	---

	# shellminator-270m — a tiny natural-language → bash command model

	shellminator is a terminal-native bash command assistant: type `sm "copy jpgs to backup"` and a 270M model suggests a single bash command; press Enter to run it in your shell, `e` to edit, `r` to refine, `c` to cancel. This repo is the model behind it — a Gemma-3-270M fine-tuned to translate a short natural-language request into one bash command.

	> Status: work-in-progress (pre-final checkpoint). The current weights were trained on the 30K qwen-distilled dataset. A combined dataset (emirkaan 6K + 30K qwen ≈ 36K) is the intended next checkpoint — see [Training data](#training-data) and [Evaluation](#evaluation).

	## Intended use

	Suggesting a single bash command from a short natural-language request, with a human in the loop:

	```
	$ sm "kill the process listening on port 8080"
	> kill -9 $(lsof -t -i :8080)
	[Enter] run [e] edit [r] refine [c] cancel
	```

	The model only suggests — the `sm` UI always shows the command for review before it runs. It is not an autonomous agent.

	## How to use

	For the `sm` tool the model is served as a GGUF via `llama.cpp` (a pre-quantized ~250MB Q4_K_M GGUF is shipped; end users never need torch). With `transformers`:

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer
	tok = AutoTokenizer.from_pretrained("ISB369/shellminator-270m-bash-distilled")
	model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-270m-bash-distilled")
	SYS = ("You are a helpful assistant that translates natural language to bash commands.\n"
	"Context: cwd=/home/user, system=Linux x86_64, shell=bash.\n"
	"Reply with a single bash command only. No explanation, no markdown fences.")
	msgs = [{"role":"system","content":SYS},
	{"role":"user","content":"Generate single Bash command: list the 10 biggest files in cwd"}]
	out = model.generate(**tok.apply_chat_template(msgs, tokenize=True, return_tensors="pt"),
	do_sample=False, max_new_tokens=128)
	print(tok.decode(out[0], skip_special_tokens=True).split("assistant\n")[-1].strip())
	```

	Use greedy decoding (`do_sample=False` / `temperature=0`) — the task is deterministic; sampling hurts it.

	## Training

	- Base: [`micrictor/gemma-3-270m-it-ft-bash`](https://huggingface.co/micrictor/gemma-3-270m-it-ft-bash) (Gemma-3 270M, instruction-tuned + bash fine-tune).
	- Method: full supervised fine-tune with [TRL](https://github.com/huggingface/trl) `SFTTrainer`; fp16-AMP over fp32; 3 epochs; per-epoch held-out eval with `load_best_model_at_end` (ships the best-generalizing epoch, not the most-trained); effective batch 16; lr 1e-4, cosine. Trained on a free Google Colab T4.
	- Orphan-token fix: the base ships an orphan `<image_soft_token>` (id 262144) with no embedding row; training calls `resize_token_embeddings(len(tokenizer))` so it gets a real trained embedding — otherwise the GGUF convert fails on the embedding-contract violation (`max(token_id) < vocab_size`).

	## Training data

	The shellminator distillation pipeline: a strong cloud teacher (`qwen3.5:397b-cloud` via ollama `/api/chat`, `think:false`) generates varied natural-language requests, labels each with a single bash command, then filters with `bash -n` (syntax) + an LLM judge (correctness) and dedupes by `(nl, cmd)`. 22 dev-tool categories: file ops, text processing, processes, networking, git (basic + advanced), docker, kubernetes, build tools, package managers, systemd, tmux, editing, cloud CLI, monitoring, permissions, disk, archives, ssh, system info, pipes/xargs, scheduling.

	Datasets:
	- [`ISB369/shellminator-bash-dataset`](https://huggingface.co/datasets/ISB369/shellminator-bash-dataset) — the 30K qwen-distilled (parts 000+001).
	- [`ISB369/shellminator-bash-clean`](https://huggingface.co/datasets/ISB369/shellminator-bash-clean) — 10K combined (emirkaan 6K + qwen 4K).
	- [`ISB369/shellminator-bash-combined`](https://huggingface.co/datasets/ISB369/shellminator-bash-combined) — 36K combined (emirkaan 6K + 30K qwen, deduped, `bash -n`-filtered). Intended for the final retrain.

	> The current checkpoint was trained on the 30K qwen alone (no emirkaan). The combined-36K retrain adds emirkaan's "echo the user's literal, specific real command" style back — see Evaluation.

	## Evaluation

	25 held-out prompts across all 22 categories, judged by 5x majority vote of the cloud teacher (`qwen3.5:397b-cloud`, greedy) + `bash -n` validity. (Single-vote judging was too noisy; majority smooths the flip-flopping.)

	\| checkpoint \| data \| correctness (judge 5x) \| validity (`bash -n`) \|
	\|---\|---\|---\|---\|
	\| Aug 11 \| 10K clean (emirkaan + qwen 4K) \| 12/25 (48%) \| 100% \|
	\| Aug 15 (current) \| 30K qwen-only \| 11/25 (44%) \| 92% \|
	\| next \| 36K combined \| TBD \| TBD \|

	What the 30K learned (win): `kubectl get pods -n production` (previously a `ps` pipeline — the kubernetes category was absorbed); `unzip backup.zip -d restore`.

	Regressions vs the 10K-clean model: the 30K qwen data leans generic/placeholder, so the model now emits e.g. `ssh-copy-id user@remote_host` instead of echoing the user's literal `192.168.1.10`, and is sloppier on precise tasks. This is the data style > scale lesson — more coverage, but the style shift cost precision. The combined-36K retrain is designed to fix this (emirkaan restores literal-echoing while keeping the dev-tool coverage).

	Capacity ceiling (not a data issue): the model sometimes mangles complex syntax (e.g. `awk` with nested quotes, an unbalanced paren). The dataset has zero invalid-syntax labels, so these are 270M generation limits, not bad data. A bigger model or constrained decoding (llama.cpp GBNF grammars) is the fix there.

	## Limitations

	- 270M capacity — strong on common single-line commands; mangles complex multi-arg/nested syntax; no multi-step reasoning.
	- Can be wrong or destructive — may suggest an incorrect or dangerous command (wrong flags, `rm`/`kill` with wrong targets). Always review before running.
	- English requests, Linux/bash, x86_64 only.
	- Literal-precision is the current (30K) checkpoint's weak spot (placeholders); the combined retrain targets this.

	## Safety / ethics

	The model suggests shell commands that can modify or delete data. The `sm` tool is built so a command is never run without a human pressing Enter (shown for review; `e` to edit, `c` to cancel). Do not wire this model into an autonomous executor. Treat every suggestion as untrusted until you have read it.

	## Reproduce

	`SFTTrainer` on the combined dataset (`HF_DATASET_REPO=ISB369/shellminator-bash-combined`), Colab T4, the config above. Eval: `train/eval.py` (25 prompts, 5x majority judge). Generation pipeline: `scripts/generate_dataset.py` (teacher → NL → bash → `bash -n` + judge → dedup; resumable; HF upload). Merge: `scripts/merge_combined.py`.

	---

	Built as a distillation exercise: a big cloud model's behavior becomes a tiny local model's training data, so a ~250MB model can suggest a bash command in under a second on CPU — privately, with no GPU.