How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mohammed-usmani/clipwright-4b:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf mohammed-usmani/clipwright-4b:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mohammed-usmani/clipwright-4b:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf mohammed-usmani/clipwright-4b:Q5_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf mohammed-usmani/clipwright-4b:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf mohammed-usmani/clipwright-4b:Q5_K_M
Use Docker
docker model run hf.co/mohammed-usmani/clipwright-4b:Q5_K_M
Quick Links

Clipwright 4B (GRPO v7)

A 4B model that does media jobs end to end. It runs ffmpeg, ffprobe, ImageMagick, SoX and ExifTool through one run(cmd, foreach) tool in a sandbox, reads the results, checks its output against the request, fixes its own mistakes, and replies with the output files. It also answers questions about files, asks when a request is ambiguous, and says when something can't be done.

Code, environment and evals: github.com/mohammed-usmani/clipwright.

How it was trained

LoRA (r32, alpha 64, all linear layers) on Qwen3.5-4B, in four stages. Every training example was executed on real files and its output checked against a hidden spec before it was kept, and the RL reward is the same check.

Stage What Revision
V3 SFT on 12,095 verified examples covering 46 kinds of media jobs (Colab L4, 1 epoch) tag v3
V4 V3 + 100 steps of GRPO: 8 tasks ร— 8 attempts per step, reward 1 for a verified pass (minus a little for extra steps), 0 otherwise tag v4
SFT v7b V4 + SFT on 2,318 new rows (2 epochs) that teach a check step after writing ("does the output match the request?") and recovery from a failed command, plus 1,500 replay rows so it doesn't forget, LR 3e-5 โ€“
GRPO v7 SFT v7b + 100 GRPO steps: 16 tasks ร— 8 attempts, a pool weighted toward the weakest job types, tasks it already solves every time skipped, LR 1e-5, no KL term (Colab A100) this revision

No test files or test requests were used in training.

Files

File Use
clipwright-4b-Q8_0.gguf LoRA merged into the base, Q8_0 (4.6 GB). Best quality on a Mac / 8 GB+ GPU.
clipwright-4b-Q5_K_M.gguf Q5_K_M (3.2 GB), for smaller GPUs. Scored as well as Q8_0 in our check (below).
adapter_model.safetensors + configs The PEFT adapter, to load on Qwen/Qwen3.5-4B.
system_prompt.txt The system prompt it was trained with. Use it as is.
tool.json The run tool schema (OpenAI function format).

Quick start (llama.cpp)

llama-server -hf mohammed-usmani/clipwright-4b:Q8_0 --jinja -c 32768

Or use the web app in the GitHub repo (app/run.sh), which runs the whole loop for you.

Send the system prompt and the run tool. The user message should list the input files under Files:, the way it looked in training:

make a 10 second preview gif of talk.mp4

Files:
- in/talk.mp4 โ€” video h264 1920x1080 30fps, audio aac stereo 48kHz, 132.1s, 11.8 MB

Then execute each run call yourself with inputs in in/ and outputs written to out/. Return the result as a tool message, as JSON with exit_code, plus stdout or error, and a short summary of the files in out/. Loop until the model stops calling the tool. Run it in a sandbox (no network, read-only inputs): these are real commands.

Evaluation

All scores come from running the model's commands and checking the output files: duration, dimensions, codecs, loudness, frames at given times, pitch, silence, sync, metadata, page counts, or the facts in a written answer. Greedy decoding, one run per task, up to 15 commands. Every task's reference solution passes its check and an empty output fails it.

test_v3 (368 tasks over the 46 job types, on a third set of held-out media, requests reworded by a different model; models compared in the same session):

Model Test (230) Dev (138) Total Stopped for looping
V4 191 114 305 (83%) 16
SFT v7b 204 117 321 (87%) 15
GRPO v7 208 121 329 (89%) 8

GRPO v7 solves 35 tasks V4 fails and fails 11 that V4 solves (paired exact test, p = 0.0005).

test_v2 (154 hand-written tasks, held-out media, the set used for earlier versions):

Job group GRPO v7 V4 (same harness)
Editing 35/42 33/42
Social and delivery 20/24 19/24
Audio 20/23 19/23
Repair 18/18 17/18
Images and PDFs 18/18 17/18
Metadata and privacy 8/9 7/9
Questions about files 8/10 8/10
Mistakes, ambiguity, limits 10/10 9/10
Total 137/154, 89% 129/154, 84%

12 gained, 4 lost (p โ‰ˆ 0.08: a lead, not yet significant on this smaller set). For reference, the 27B teacher that wrote the original training data scored 119/154 on test_v2 (it was not run on test_v3).

Quantization (test_v3 dev, 138 tasks, llama.cpp on a Mac): Q5_K_M 126, Q8_0 125, Q4_K_M 118. Q4_K_M is not published because it loses about 5 points.

Limitations

  • Breadth beyond the 46 job types is the main gap. On requests built from held-out Stack Exchange questions it passes 135 of 172 (78%), and only 16 of 30 (53%) when the job needs a filter that is rare in its training data. In hand tests it failed a GIF built from specific frames, a robot-voice effect and vague "clean up the noise" requests.
  • It can say "Done" over an output that doesn't fully match the request. Check important outputs yourself.
  • The test files are synthetic, so real footage with odd codecs or damage may do worse.
  • Trained on ffmpeg 8 to 9 syntax and ImageMagick 7 (magick).
  • Earlier versions: v3 (SFT only, 115/154 test_v2) and v4 (first GRPO, 128/154) are tagged revisions.
Downloads last month
322
GGUF
Model size
4B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

5-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mohammed-usmani/clipwright-4b

Finetuned
Qwen/Qwen3.5-4B
Adapter
(1,219)
this model