forge v0.2: one-command tune->NPU pipeline (183s proven), forge CLI, snag ledger, frozen known-good stack

README.md

@@ -7,88 +7,83 @@ tags:
 - xdna2
 - fastflowlm
 - strix-halo
+- lora
+- fine-tuning
 - tools
 pipeline_tag: text-generation
 ---
 
-# NPU-Forge 🔥 — keep your custom models alive on AMD Ryzen AI NPUs
+# NPU-Forge 🔥 — fine-tune a model and put it on your AMD Ryzen AI NPU in ~3 minutes
 
-**The problem nobody tells you about:** you convert a fine-tuned model to
-[FastFlowLM](https://github.com/FastFlowLM/FastFlowLM)'s Q4NX format, register
-it in `model_list.json`, it runs beautifully on your NPU… and then an FLM
-update ships and your custom model **silently disappears**. The model files
-survive on disk — but `model_list.json` lives in `C:\Program Files\flm\` and
-every update resets it, de-registering every custom entry you ever added.
+**Measured, on a Strix Halo (Ryzen AI MAX+ 395), June 2026:** a LoRA fine-tune of
+Llama-3.2-1B on 300+ real chat exchanges — trained, merged, behavior-verified,
+converted to GGUF, re-quantized to FastFlowLM's Q4NX, NPU-ready — in
+**183 seconds of cloud time** (≈ $0.10 on a rented T4):
 
-We hit this on a Strix Halo box: a fully converted Llama-3 fine-tune
-(Stheno 8B), complete with compiled `.xclbin` NPU kernels, sitting in
-`Documents\flm\models\` — invisible to `flm list` because an update had
-eaten its registration.
+```
+forge tune my-chats.jsonl --name grandma
+  ├─ LoRA fine-tune (cloud GPU)        122 s
+  ├─ merge                               3 s
+  ├─ voice proof (model speaks first!)   5 s
+  ├─ HF -> GGUF (q8_0)                  29 s
+  └─ GGUF -> Q4NX (NPU format)          24 s
+forge register        (one UAC click)
+flm run grandma-forge:1b
+```
 
-**NPU-Forge fixes it permanently:**
+The "voice proof" stage generates a sample from the merged model *inside the
+training job*, before any conversion — so you know the tune actually took.
+Ours came back with the persona's exact ritual phrases after 2 minutes of
+training. That's the bar.
 
-- **`registry.json`** — YOUR registry, in user space, owned by you. Every
-  custom model entry lives here forever. FLM updates can't touch it.
-- **`bin/register.js`** — idempotently merges your registry into FLM's
-  `model_list.json`. Run it any time a model "disappears" — takes one second,
-  backs up FLM's list (timestamped) before every write.
-- **`register-admin.bat`** — double-click version that self-elevates (the
-  one part that needs admin is writing into Program Files).
+## What's in this repo
 
-## Usage
+- **`forge.js` / `forge.bat`** — the CLI: `tune`, `convert`, `register`,
+  `list`, `doctor`, `serve`
+- **`modal/tune_npu.py`** — the whole tune→NPU pipeline as one
+  [Modal](https://modal.com) job (bring your own Modal account; T4 is plenty)
+- **`modal/convert_q4nx.py`** — just the GGUF→Q4NX stage (65 s for a 1B)
+- **`bin/assemble.js`** — downloads results and stages the FLM model folder
+- **`bin/register.js` + `register-admin.bat`** — the permanent custom-model
+  registry that survives FLM updates (see below)
+- **`registry.example.json`** — entry template
 
-1. Convert your fine-tune to Q4NX with the official
-   [FLM_Q4NX_Converter](https://github.com/FastFlowLM/FLM_Q4NX_Converter)
-   (GGUF in — any quant — Q4NX out). Supported families: LLaMA, Qwen 2/2.5/3/3.5,
-   Gemma 3, Phi-4, LFM2, GPT-OSS, and more.
-2. Put the model folder in `C:\Users\<you>\Documents\flm\models\<YourModel>\`
-   with the standard four files: `config.json`, `model.q4nx`,
-   `tokenizer.json`, `tokenizer_config.json`. (No admin needed — model
-   storage is user-space.)
-3. Add an entry to `registry.json` (copy `registry.example.json` and edit —
-   easiest is mirroring the entry of the same-architecture stock model from
-   FLM's own `model_list.json`, changing `name` to your folder name and
-   blanking `url`).
-4. Double-click `register-admin.bat`. Done: `flm run yourmodel-forge:8b`.
-5. FLM updated and your model vanished again? Double-click again. That's it.
+Chat data format: one JSON per line, `{"messages":[{"role":"user","content":...},{"role":"assistant","content":...}]}`.
 
-```json
-// registry.example.json — one custom llama3-family 8B model
-{
-  "models": {
-    "mymodel-forge": {
-      "8b": {
-        "name": "MyModel-8B-NPU2",
-        "url": "",
-        "file_url": "",
-        "default_context_length": 8192,
-        "max_prefill_len": 4096,
-        "files": ["config.json", "model.q4nx", "tokenizer.json", "tokenizer_config.json"],
-        "details": { "family": "llama3", "parameter_size": "8B", "quantization_level": "Q4_1" },
-        "footprint": 4.7
-      }
-    }
-  }
-}
-```
+## The registry problem (why `forge register` exists)
+
+FLM's `model_list.json` lives in `C:\Program Files\flm\` and **every FLM update
+resets it**, silently de-registering all your custom models. Your model files
+survive (they're in `Documents\flm\models\`) but they vanish from `flm list`.
+Forge keeps its own user-space `registry.json` forever and re-merges with one
+click. `forge doctor` tells you when an update has eaten your registrations.
+
+## The snag ledger — six walls we hit so you don't
 
-## Facts we verified the hard way (Strix Halo, FLM v0.9.43)
+1. **The Q4NX converter's `convert.py` CLI is broken at HEAD** (uncommented
+   debug `sys.argv` override hijacks every invocation). Call the module API:
+   `from q4nx import create_converter; create_converter(gguf, "").convert(q4nx_path=out, weights_type="language")`
+2. Converter needs `einops` and `tqdm` beyond its README list, and **must run
+   with cwd = its repo root** (relative `configs/<arch>.json` loads).
+3. **Llama-3.2 tokenizers need `transformers>=4.46`** — the error
+   `untagged enum ModelWrapper` is that wall exactly.
+4. **`transformers 4.46` needs `accelerate>=1.0`** — the error
+   `'AdamW' object has no attribute 'train'` at step 0 is that skew.
+5. **T4 + Llama-3.2's 128k vocab OOMs at batch 4** (loss-logits blowup).
+   Floor: batch 1 × grad-accum 8 + gradient checkpointing.
+6. **NPU driver minimum for current FLM: `32.0.203.304`** (`.311`
+   recommended). `flm validate` will tell you; so will `forge doctor`.
 
-- Model **files** are user-space (`Documents\flm\models\`) — no admin needed.
-- The **registry** (`model_list.json`) is Program Files — admin only, and
-  reset by updates.
-- A user-level `Documents\flm\model_list.json` is **ignored** (we tested).
-- Unregistered model folders can't be run by folder name (we tested) —
-  registration is mandatory.
-- FLM compiles per-model NPU kernels (`.xclbin`) on first run; they live in
-  the model folder and survive alongside it.
+**Frozen known-good stack** (the whole point — never debug this again):
+`torch 2.4.1 · transformers 4.46.3 · trl 0.9.6 · peft 0.12.0 ·
+accelerate 1.1.1 · datasets 2.21.0 · gguf · amd-quark · einops · tqdm · protobuf`
 
-## Where this is going
+## Requirements
 
-NPU-Forge is the first piece of a larger goal: **one-command fine-tune → NPU**.
-The full pipeline (LoRA fine-tune in the cloud → GGUF → Q4NX → registered and
-serving on your NPU, single command) is in active development, along with
-encoder-model (BERT-class) NPU deployment. Watch this repo.
+- AMD Ryzen AI machine with XDNA2 NPU (Strix, Strix Halo, Kraken…) +
+  [FastFlowLM](https://github.com/FastFlowLM/FastFlowLM)
+- Node.js (the CLI), Python + a [Modal](https://modal.com) account (the cloud legs)
+- NPU driver ≥ 32.0.203.304
 
-Built on a Ryzen AI MAX+ 395 (Strix Halo, XDNA2) as part of an ongoing
-project to make local NPUs a first-class home for personal AI.
+Part of an ongoing project to make local NPUs a first-class home for personal
+AI — voices you own, on silicon you own.

bin/assemble.js

@@ -0,0 +1,75 @@
+#!/usr/bin/env node
+// assemble.js — local leg of forge convert: pull the converted .q4nx from the
+// Modal volume, build the FLM model folder (donor config/tokenizer from the
+// same-architecture stock model), and add the registry.json entry.
+//
+//   node bin/assemble.js --out forge-llama32-1b --donor Llama-3.2-1B-NPU2 \
+//        --tag llama32-forge:1b --donor-tag llama3.2:1b
+'use strict';
+const fs = require('fs');
+const path = require('path');
+const { spawnSync } = require('child_process');
+
+const FORGE = path.join(__dirname, '..');
+const FLM_MODELS = path.join(process.env.USERPROFILE, 'Documents', 'flm', 'models');
+const FLM_LIST = 'C:\\Program Files\\flm\\model_list.json';
+
+function arg(name, def) {
+  const i = process.argv.indexOf('--' + name);
+  return i > 0 ? process.argv[i + 1] : def;
+}
+
+function main() {
+  const outName = arg('out');                       // volume folder name
+  const donor = arg('donor');                       // stock model folder to copy config/tokenizer from
+  const tag = arg('tag');                           // e.g. llama32-forge:1b
+  const donorTag = arg('donor-tag');                // e.g. llama3.2:1b (registry donor entry)
+  if (!outName || !donor || !tag || !donorTag) {
+    console.error('usage: node bin/assemble.js --out <volumeName> --donor <StockFolder> --tag <key:size> --donor-tag <stockKey:size>');
+    process.exit(1);
+  }
+  const forgeName = 'Forge-' + outName.replace(/^forge-/, '') + '-NPU2';
+  const dest = path.join(FLM_MODELS, forgeName);
+  fs.mkdirSync(dest, { recursive: true });
+
+  // 1. download q4nx + tokenizer.json (the converter extracts it from GGUF)
+  console.log('[1/3] downloading from Modal volume npu-forge-out/' + outName);
+  for (const f of ['model.q4nx', 'tokenizer.json']) {
+    const r = spawnSync('modal', ['volume', 'get', 'npu-forge-out', outName + '/' + f, path.join(dest, f), '--force'],
+      { encoding: 'utf8', env: { ...process.env, PYTHONIOENCODING: 'utf-8' }, shell: true, timeout: 900000 });
+    if (!fs.existsSync(path.join(dest, f))) {
+      console.error('download failed for ' + f + ':\n' + (r.stderr || r.stdout || '').slice(-800));
+      process.exit(1);
+    }
+    console.log('  ' + f + ': ' + (fs.statSync(path.join(dest, f)).size / 1e6).toFixed(1) + ' MB');
+  }
+
+  // 2. donor config files (same architecture => identical config works)
+  console.log('[2/3] copying config.json + tokenizer_config.json from donor ' + donor);
+  const donorDir = path.join(FLM_MODELS, donor);
+  for (const f of ['config.json', 'tokenizer_config.json']) {
+    fs.copyFileSync(path.join(donorDir, f), path.join(dest, f));
+  }
+
+  // 3. registry entry from the donor's model_list entry
+  console.log('[3/3] adding registry.json entry ' + tag);
+  const ml = JSON.parse(fs.readFileSync(FLM_LIST, 'utf8').replace(/^/, ''));
+  const [dKey, dSize] = donorTag.split(':');
+  const donorEntry = ml.models[dKey] && ml.models[dKey][dSize];
+  if (!donorEntry) { console.error('donor tag not in model_list: ' + donorTag); process.exit(1); }
+  const entry = JSON.parse(JSON.stringify(donorEntry));
+  entry.name = forgeName; entry.url = ''; entry.file_url = '';
+  entry.modified_at = new Date().toISOString();
+  const regPath = path.join(FORGE, 'registry.json');
+  const reg = fs.existsSync(regPath) ? JSON.parse(fs.readFileSync(regPath, 'utf8').replace(/^/, '')) : { models: {} };
+  const [key, size] = tag.split(':');
+  reg.models[key] = reg.models[key] || {};
+  reg.models[key][size] = entry;
+  fs.writeFileSync(regPath, JSON.stringify(reg, null, 2));
+
+  console.log('\nAssembled: ' + dest);
+  console.log('Registered in forge registry as ' + tag + ' — now double-click register-admin.bat, then:');
+  console.log('  flm run ' + tag);
+}
+
+main();

forge.bat

@@ -0,0 +1,2 @@
+@echo off
+node "%~dp0forge.js" %*

forge.js

@@ -0,0 +1,134 @@
+#!/usr/bin/env node
+// forge — tune and convert models for AMD Ryzen AI NPUs, one command.
+//
+//   forge tune <chats.jsonl> [--name grandma] [--base unsloth/Llama-3.2-1B-Instruct]
+//              [--donor-tag llama3.2:1b] [--epochs 3]
+//   forge convert <hf-repo> <file.gguf> [--name mymodel] [--donor-tag llama3.2:1b]
+//   forge register            merge registry.json into FLM's list (UAC if needed)
+//   forge list                stock models + forge customs with status
+//   forge doctor              NPU stack / registry / port health checks
+//   forge serve <tag>         flm serve passthrough (performance pmode)
+'use strict';
+const fs = require('fs');
+const path = require('path');
+const { spawnSync, spawn } = require('child_process');
+
+const FORGE = __dirname;
+const FLM_LIST = 'C:\\Program Files\\flm\\model_list.json';
+const FLM_MODELS = path.join(process.env.USERPROFILE, 'Documents', 'flm', 'models');
+const ENV = { ...process.env, PYTHONIOENCODING: 'utf-8' };
+
+const argv = process.argv.slice(2);
+const cmd = argv[0];
+function flag(name, def) { const i = argv.indexOf('--' + name); return i > 0 ? argv[i + 1] : def; }
+function run(exe, args, opts = {}) {
+  const r = spawnSync(exe, args, { stdio: 'inherit', env: ENV, shell: true, ...opts });
+  return r.status === 0;
+}
+function readJson(p) { try { return JSON.parse(fs.readFileSync(p, 'utf8').replace(/^/, '')); } catch (_) { return null; } }
+
+function assemble(outName, tag, donorTag, donor) {
+  return run('node', [path.join(FORGE, 'bin', 'assemble.js'), '--out', outName,
+    '--donor', donor, '--tag', tag, '--donor-tag', donorTag]);
+}
+
+function donorFolderFor(donorTag) {
+  const ml = readJson(FLM_LIST) || { models: {} };
+  const [k, s] = (donorTag || 'llama3.2:1b').split(':');
+  const e = ml.models[k] && ml.models[k][s];
+  if (!e) { console.error('donor tag not found in FLM list: ' + donorTag); process.exit(1); }
+  return e.name;
+}
+
+if (cmd === 'tune') {
+  const data = argv[1];
+  if (!data || !fs.existsSync(data)) { console.error('forge tune <chats.jsonl> — file not found'); process.exit(1); }
+  const name = flag('name', path.basename(data, path.extname(data)));
+  const base = flag('base', 'unsloth/Llama-3.2-1B-Instruct');
+  const donorTag = flag('donor-tag', 'llama3.2:1b');
+  const epochs = flag('epochs', '3');
+  const outName = 'forge-' + name.toLowerCase().replace(/[^a-z0-9]+/g, '-');
+  // stage data where the modal mount expects it
+  const stage = path.join(FORGE, 'scratch', 'tune-data');
+  fs.mkdirSync(stage, { recursive: true });
+  fs.copyFileSync(data, path.join(stage, 'data.jsonl'));
+  console.log(`forge tune: ${data} -> ${base} (${epochs} epochs) -> NPU as ${name}-forge`);
+  if (!run('modal', ['run', path.join(FORGE, 'modal', 'tune_npu.py'),
+    '--base', base, '--data-file', 'data.jsonl', '--out-name', outName, '--epochs', epochs])) {
+    console.error('tune job failed — see output above'); process.exit(1);
+  }
+  if (!assemble(outName, name.toLowerCase() + '-forge:1b', donorTag, donorFolderFor(donorTag))) process.exit(1);
+  console.log('\nDone. Run: forge register   then: flm run ' + name.toLowerCase() + '-forge:1b');
+}
+
+else if (cmd === 'convert') {
+  const repo = argv[1], file = argv[2];
+  if (!repo || !file) { console.error('forge convert <hf-repo> <file.gguf>'); process.exit(1); }
+  const name = flag('name', file.replace(/\.gguf$/i, '').toLowerCase().replace(/[^a-z0-9]+/g, '-').slice(0, 30));
+  const donorTag = flag('donor-tag', 'llama3.2:1b');
+  const outName = 'forge-' + name;
+  console.log(`forge convert: ${repo}/${file} -> NPU as ${name}-forge`);
+  if (!run('modal', ['run', path.join(FORGE, 'modal', 'convert_q4nx.py'),
+    '--repo-id', repo, '--filename', file, '--out-name', outName])) {
+    console.error('convert job failed'); process.exit(1);
+  }
+  const size = donorTag.split(':')[1] || '1b';
+  if (!assemble(outName, name + '-forge:' + size, donorTag, donorFolderFor(donorTag))) process.exit(1);
+  console.log('\nDone. Run: forge register   then: flm run ' + name + '-forge:' + size);
+}
+
+else if (cmd === 'register') {
+  const r = spawnSync('node', [path.join(FORGE, 'bin', 'register.js')], { encoding: 'utf8', env: ENV, shell: true });
+  process.stdout.write(r.stdout || '');
+  if (r.status !== 0) {
+    console.log('(needs admin — launching UAC prompt via register-admin.bat)');
+    spawn('cmd', ['/c', 'start', '', path.join(FORGE, 'register-admin.bat')], { detached: true, shell: false });
+  }
+}
+
+else if (cmd === 'list') {
+  run('flm', ['list', '--filter', 'installed']);
+  const reg = readJson(path.join(FORGE, 'registry.json')) || { models: {} };
+  const ml = readJson(FLM_LIST) || { models: {} };
+  console.log('\nforge customs:');
+  for (const [k, sizes] of Object.entries(reg.models)) {
+    for (const [s, e] of Object.entries(sizes)) {
+      const registered = !!(ml.models[k] && ml.models[k][s]);
+      const folder = fs.existsSync(path.join(FLM_MODELS, e.name));
+      console.log(`  ${k}:${s}  files:${folder ? 'OK' : 'MISSING'}  registered:${registered ? 'YES' : 'no — run forge register'}`);
+    }
+  }
+}
+
+else if (cmd === 'doctor') {
+  console.log('— NPU device —');
+  run('powershell', ['-NoProfile', '-Command', '"Get-PnpDevice -Class ComputeAccelerator | Select-Object Status, FriendlyName | Format-Table -HideTableHeaders"']);
+  console.log('— NPU driver version (FLM needs >= 32.0.203.304, recommends .311) —');
+  const dv = spawnSync('powershell', ['-NoProfile', '-Command', '"Get-PnpDevice -Class ComputeAccelerator | ForEach-Object { (Get-PnpDeviceProperty -InstanceId $_.InstanceId -KeyName DEVPKEY_Device_DriverVersion).Data }"'], { encoding: 'utf8', shell: true });
+  const ver = (dv.stdout || '').trim().split('\n')[0] || 'unknown';
+  const last = parseInt((ver.match(/(\d+)$/) || [])[1] || '0', 10);
+  console.log(`  installed: ${ver}` + (last && last < 304 ? '  -> TOO OLD: update the AMD NPU driver (AMD Software / amd.com Ryzen AI driver), then re-run doctor' : last ? '  -> OK' : ''));
+  console.log('— flm validate —');
+  run('flm', ['validate']);
+  console.log('— registry drift (customs lost to an flm update?) —');
+  const reg = readJson(path.join(FORGE, 'registry.json')) || { models: {} };
+  const ml = readJson(FLM_LIST) || { models: {} };
+  let drift = 0;
+  for (const [k, sizes] of Object.entries(reg.models)) for (const s of Object.keys(sizes)) {
+    if (!(ml.models[k] && ml.models[k][s])) { console.log(`  LOST: ${k}:${s} — run forge register`); drift++; }
+  }
+  if (!drift) console.log('  all forge customs registered');
+  console.log('— port 52625 —');
+  const ns = spawnSync('netstat', ['-ano'], { encoding: 'utf8' });
+  console.log((ns.stdout || '').split('\n').some(l => l.includes(':52625') && l.includes('LISTENING')) ? '  flm serve is UP' : '  free (flm not serving)');
+}
+
+else if (cmd === 'serve') {
+  const tag = argv[1];
+  if (!tag) { console.error('forge serve <tag>'); process.exit(1); }
+  run('flm', ['serve', tag, '--pmode', 'performance']);
+}
+
+else {
+  console.log(fs.readFileSync(__filename, 'utf8').split('\n').slice(1, 11).map(l => l.replace(/^\/\/ ?/, '')).join('\n'));
+}

modal/convert_q4nx.py

@@ -0,0 +1,56 @@
+"""npu-forge-convert: GGUF -> Q4NX on Modal (the remote leg of forge convert).
+Clones the official FLM_Q4NX_Converter, pulls a GGUF from HF, converts, and
+parks the .q4nx on a volume for the local box to download.
+
+  modal run convert_q4nx.py --repo-id bartowski/Llama-3.2-1B-Instruct-GGUF \
+      --filename Llama-3.2-1B-Instruct-Q4_K_M.gguf --out-name forge-llama32-1b
+"""
+import modal
+
+app = modal.App("npu-forge-convert")
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .apt_install("git")
+    .pip_install("torch==2.4.1", "transformers==4.44.2", "numpy", "gguf", "amd-quark", "huggingface_hub", "safetensors", "sentencepiece", "einops", "tqdm")
+    .run_commands(
+        "git clone --depth 1 https://github.com/FastFlowLM/FLM_Q4NX_Converter /converter",
+        "if [ -f /converter/requirements.txt ]; then pip install -r /converter/requirements.txt; fi",
+    )
+)
+vol = modal.Volume.from_name("npu-forge-out", create_if_missing=True)
+
+
+@app.function(image=image, cpu=8, memory=16384, timeout=3600, volumes={"/out": vol})
+def convert(repo_id: str, filename: str, out_name: str, family: str = ""):
+    import subprocess, os, time
+    from huggingface_hub import hf_hub_download
+
+    t0 = time.time()
+    gguf = hf_hub_download(repo_id=repo_id, filename=filename)
+    t1 = time.time()
+    print(f"[download] {filename} in {t1-t0:.0f}s ({os.path.getsize(gguf)/1e9:.2f} GB)")
+
+    outdir = f"/out/{out_name}"
+    os.makedirs(outdir, exist_ok=True)
+    # call the converter's Python API directly — upstream convert.py HEAD has
+    # an uncommented debug sys.argv override that hijacks every CLI invocation
+    import sys
+    sys.path.insert(0, "/converter")
+    os.chdir("/converter")            # converter loads configs/<arch>.json relative to its repo root
+    from q4nx import create_converter
+    print(f"[convert] create_converter({gguf}, family={family!r})")
+    model = create_converter(gguf, family or "")
+    model.convert(q4nx_path=outdir, weights_type="language")
+    t2 = time.time()
+    files = {f: os.path.getsize(os.path.join(outdir, f)) for f in os.listdir(outdir)}
+    vol.commit()
+    return {"convert_seconds": round(t2 - t1), "download_seconds": round(t1 - t0), "outputs": files}
+
+
+@app.local_entrypoint()
+def main(repo_id: str = "bartowski/Llama-3.2-1B-Instruct-GGUF",
+         filename: str = "Llama-3.2-1B-Instruct-Q4_K_M.gguf",
+         out_name: str = "forge-llama32-1b",
+         family: str = ""):
+    import json
+    print(json.dumps(convert.remote(repo_id, filename, out_name, family), indent=2))

modal/tune_npu.py

@@ -0,0 +1,108 @@
+"""npu-forge-tune: THE pipeline — LoRA fine-tune -> merge -> in-job voice
+proof -> GGUF -> Q4NX, one container, one command. Output lands on the
+npu-forge-out volume ready for bin/assemble.js on the local box.
+
+  modal run tune_npu.py                 (defaults: Llama-3.2-1B + grandma.jsonl)
+"""
+import modal
+
+app = modal.App("npu-forge-tune")
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .apt_install("git")
+    .pip_install(
+        "torch==2.4.1", "transformers==4.46.3", "peft==0.12.0", "trl==0.9.6",
+        "datasets==2.21.0", "accelerate==1.1.1", "numpy", "gguf", "amd-quark",
+        "huggingface_hub", "safetensors", "sentencepiece", "einops", "tqdm", "protobuf",
+    )
+    .run_commands(
+        "git clone --depth 1 https://github.com/FastFlowLM/FLM_Q4NX_Converter /converter",
+        "git clone --depth 1 https://github.com/ggml-org/llama.cpp /llamacpp",
+    )
+    .add_local_dir("C:/Users/Forgemind/Desktop/npu-forge/scratch/tune-data", remote_path="/data")
+)
+vol = modal.Volume.from_name("npu-forge-out", create_if_missing=True)
+
+
+@app.function(image=image, gpu="T4", cpu=8, memory=32768, timeout=5400, volumes={"/out": vol})
+def tune(base: str, data_file: str, out_name: str, epochs: int = 3, probe: str = "Good morning grandma, how is the garden today?"):
+    import json, os, subprocess, time
+    import torch
+    from datasets import load_dataset
+    from transformers import AutoTokenizer, AutoModelForCausalLM
+    from peft import LoraConfig
+    from trl import SFTTrainer, SFTConfig
+
+    timings = {}
+    t = time.time()
+
+    tok = AutoTokenizer.from_pretrained(base)
+    if tok.pad_token is None:
+        tok.pad_token = tok.eos_token
+    model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.float16, device_map="cuda")
+
+    ds = load_dataset("json", data_files=f"/data/{data_file}", split="train")
+    ds = ds.map(lambda r: {"text": tok.apply_chat_template(r["messages"], tokenize=False)})
+    print(f"[data] {len(ds)} examples")
+
+    trainer = SFTTrainer(
+        model=model,
+        train_dataset=ds,
+        peft_config=LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, target_modules=["q_proj", "k_proj", "v_proj", "o_proj"]),
+        args=SFTConfig(
+            output_dir="/tmp/sft", num_train_epochs=epochs, per_device_train_batch_size=1,
+            gradient_accumulation_steps=8, learning_rate=2e-4, logging_steps=20,
+            max_seq_length=1024, dataset_text_field="text", report_to=[], save_strategy="no",
+            fp16=True, gradient_checkpointing=True,
+        ),
+    )
+    trainer.train()
+    timings["train_s"] = round(time.time() - t); t = time.time()
+
+    merged = trainer.model.merge_and_unload()
+    merged.save_pretrained("/tmp/merged", safe_serialization=True)
+    tok.save_pretrained("/tmp/merged")
+    timings["merge_s"] = round(time.time() - t); t = time.time()
+
+    # in-job voice proof: does the merged model actually speak the tune?
+    msgs = [{"role": "user", "content": probe}]
+    inp = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to("cuda")
+    with torch.no_grad():
+        gen = merged.generate(inp, max_new_tokens=160, temperature=0.8, do_sample=True, pad_token_id=tok.eos_token_id)
+    sample = tok.decode(gen[0][inp.shape[1]:], skip_special_tokens=True)
+    print("[voice proof] " + sample[:600])
+    timings["sample_s"] = round(time.time() - t); t = time.time()
+
+    # HF -> GGUF (q8_0 keeps quality into the Q4NX requant)
+    r = subprocess.run(["python", "/llamacpp/convert_hf_to_gguf.py", "/tmp/merged",
+                        "--outfile", "/tmp/model-q8.gguf", "--outtype", "q8_0"],
+                       capture_output=True, text=True)
+    if r.returncode != 0:
+        print(r.stdout[-2000:]); print("STDERR:", r.stderr[-3000:])
+        raise RuntimeError("convert_hf_to_gguf failed")
+    timings["gguf_s"] = round(time.time() - t); t = time.time()
+
+    # GGUF -> Q4NX (proven stage: module API, cwd=/converter)
+    import sys
+    sys.path.insert(0, "/converter")
+    os.chdir("/converter")
+    from q4nx import create_converter
+    outdir = f"/out/{out_name}"
+    os.makedirs(outdir, exist_ok=True)
+    create_converter("/tmp/model-q8.gguf", "").convert(q4nx_path=outdir, weights_type="language")
+    timings["q4nx_s"] = round(time.time() - t)
+
+    files = {f: os.path.getsize(os.path.join(outdir, f)) for f in os.listdir(outdir)}
+    with open(os.path.join(outdir, "tune-report.json"), "w") as fh:
+        json.dump({"base": base, "data": data_file, "epochs": epochs, "sample": sample, "timings": timings}, fh, indent=2)
+    vol.commit()
+    return {"timings": timings, "outputs": files, "voice_sample": sample[:400]}
+
+
+@app.local_entrypoint()
+def main(base: str = "unsloth/Llama-3.2-1B-Instruct",
+         data_file: str = "grandma.jsonl",
+         out_name: str = "forge-grandma-1b",
+         epochs: int = 3):
+    import json
+    print(json.dumps(tune.remote(base, data_file, out_name, epochs), indent=2))