Complete examples/ suite parity with Thanatos-27B (adapted to Janus)

Ports the sibling's remaining runnable clients, adapted to Janus:
- examples/ollama_chat.py — chat/stream/tools/OpenAI-compat, MODEL=janus
- examples/transformers_quickstart.py — Heretic safetensors, 4-bit ~20 GB /
bf16 ~70 GB, MoE all-experts-resident note
- examples/llama_cpp_quickstart.py — GGUF via llama-cpp-python, no daemon
- examples/README.md — index rewritten for Janus's direct-script
flow (no Makefile/heal-hf; ./scripts/*.sh,
tag janus, Q4_K_M ~19 GB, mmproj ~903 MB)

README 'What's here' row consolidated to examples/; inline 'Inference
examples' now points at examples/. Supersedes the earlier
'text examples not duplicated' CHANGELOG note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -55,19 +55,26 @@ track the **tooling and documentation**, not the underlying base model.
   dense sibling's exact wording and drops the ambiguous `262k` rounding
   (262 144 = 256 × 1024, not 262 000).
 
-### Added (vision tooling — parity with Thanatos-27B)
-- **`scripts/fetch_vision.sh`, `examples/llama_cpp_vision.py`, and a `## Vision`
-  README section.** Mirrors the dense sibling's vision path, adapted to Janus:
-  bundle `Janus-35B-A3B.Q4_K_M.gguf`, arch `qwen35moe`, projector
-  `Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf` (~903 MB, BF16 only)
-  from `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`, with the
+### Added (examples/ suite + vision tooling — parity with Thanatos-27B)
+- **`examples/` directory, `scripts/fetch_vision.sh`, and a `## Vision`
+  README section.** Ports the dense sibling's runnable clients and vision path,
+  adapted to Janus: bundle `Janus-35B-A3B.Q4_K_M.gguf`, arch `qwen35moe`,
+  projector `Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf` (~903 MB,
+  BF16 only) from `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`, with the
   `unsloth/Qwen3.6-35B-A3B-GGUF` `mmproj-F16.gguf` as the F16/F32 fallback.
-  `fetch_vision.sh` pulls the projector; `llama_cpp_vision.py` runs
-  image-text-to-text via llama-cpp-python. The Vision section documents the
-  llama.cpp loader path and the Ollama-vision-broken caveat (`unknown model
-  architecture: 'qwen35moe'`, ollama/ollama#15898). The sibling's **text**
-  example scripts were deliberately not duplicated — Janus already covers those
-  inline under "Inference examples". "What's here" table updated.
+  - `examples/ollama_chat.py` (chat / streaming / tool round-trip /
+    OpenAI-compat, default `MODEL=janus`), `examples/transformers_quickstart.py`
+    (Heretic safetensors, 4-bit ~20 GB / bf16 ~70 GB, MoE keeps all experts
+    resident), `examples/llama_cpp_quickstart.py` (GGUF, no daemon),
+    `examples/llama_cpp_vision.py` (image-text-to-text via llama-cpp-python),
+    and `examples/README.md`. The index is adapted to Janus's direct-script
+    flow (`./scripts/build.sh`, `./scripts/load_bundle.sh`,
+    `./scripts/fetch_vision.sh`) — Janus has no Makefile and, being already
+    `qwen35moe`-stamped, no heal/rebadge step.
+  - `fetch_vision.sh` pulls the projector; the Vision section documents the
+    llama.cpp loader path and the Ollama-vision-broken caveat (`unknown model
+    architecture: 'qwen35moe'`, ollama/ollama#15898).
+  - "What's here" table + "Inference examples" pointer updated.
 
 ### Changed (HF repo rename → FoolDev/Janus-35B-HERETIC)
 - **HF repository slug renamed `FoolDev/Janus-35B` → `FoolDev/Janus-35B-HERETIC`.**

README.md

@@ -103,7 +103,7 @@ and `thinking` under Capabilities. Hardware: ~38 GB RAM at default
 | `scripts/bench.sh` | Measures tok/s from Ollama's `eval_count` / `eval_duration` over a short/medium/long prompt mix (with a discarded warmup). Defaults to `MODEL=janus`. |
 | `scripts/load_bundle.sh` | Loads the bundled `Janus-35B-A3B.Q4_K_M.gguf` into Ollama as a local `janus` tag without an upstream pull (smudges the LFS pointer via `hf download` if needed, checks the arch is `qwen35moe`). |
 | `scripts/fetch_vision.sh` | Downloads the vision projector (`Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf`) from the Heretic GGUF repo for llama.cpp image input (Ollama vision is broken upstream — see [Vision](#vision)). |
-| `examples/llama_cpp_vision.py` | Minimal image-text-to-text example via `llama-cpp-python`; the projector attaches here, since Ollama can't attach it yet. |
+| `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python (text, tools, and vision — see `examples/README.md`) |
 
 GGUF-only release. Pull the Heretic safetensors from [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) if you need the `transformers` tree (or the vanilla pre-Heretic base from [`Qwen/Qwen3.6-35B-A3B`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)).
 
@@ -150,7 +150,7 @@ After either path, `ollama show janus` should list `completion`, `tools`, and `t
 
 ### Inference examples
 
-Once the model is loaded (via `ollama run janus`, `lms server`, or `llama-server`), all the standard OpenAI-compatible clients work. Examples assume the loader is listening on `http://localhost:11434` (Ollama default) — adjust the port for LM Studio (`:1234`) or llama.cpp (`:8080`).
+Once the model is loaded (via `ollama run janus`, `lms server`, or `llama-server`), all the standard OpenAI-compatible clients work. Examples assume the loader is listening on `http://localhost:11434` (Ollama default) — adjust the port for LM Studio (`:1234`) or llama.cpp (`:8080`). Runnable versions of everything below live in [`examples/`](examples/README.md).
 
 #### curl
 

examples/README.md

@@ -0,0 +1,91 @@
+# Janus-35B examples
+
+Four minimal entry points. Pick the one that matches how you run models.
+
+| File | Backend | When to use |
+|---|---|---|
+| `ollama_chat.py` | Ollama HTTP API | You already have `ollama serve` running and the `janus` model created from the project `Modelfile`. **Text + tool calling** — vision via Ollama is broken upstream for this arch. |
+| `transformers_quickstart.py` | Hugging Face Transformers | You want to run the Heretic safetensors (`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`) on GPU, optionally in 4-bit via bitsandbytes. |
+| `llama_cpp_quickstart.py` | llama-cpp-python | You want to invoke a local GGUF directly without a daemon (CI, batch jobs, scripts). Text only. |
+| `llama_cpp_vision.py` | llama-cpp-python + mmproj | **Image input.** Loads a text GGUF + `Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf` and answers questions about an image. The only working vision path right now. |
+
+All four apply the same Janus system prompt and sampling defaults
+(`temp=0.6, top_p=0.95, top_k=20, repeat_penalty=1.05`) so behavior should
+be consistent across backends modulo quantization noise. The three
+non-Ollama scripts set them explicitly; `ollama_chat.py` inherits them
+from the `Modelfile` / bridge files.
+
+## Setup
+
+### Ollama
+
+Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
+root-level `template` / `system` / `params` files via HF's Ollama
+bridge):
+
+```bash
+ollama pull hf.co/FoolDev/Janus-35B-HERETIC              # ~19 GB Q4_K_M (only bundled quant)
+pip install requests
+MODEL=hf.co/FoolDev/Janus-35B-HERETIC python ollama_chat.py
+```
+
+Or build a local `janus` tag from this repo's bundled GGUF without going
+through the HF pull:
+
+```bash
+cd ..  &&  ./scripts/load_bundle.sh  &&  cd examples
+python ollama_chat.py                                   # MODEL defaults to `janus`
+```
+
+For a non-bundled quant (e.g. Q3_K_M ~14 GB, Q5_K_M ~24 GB),
+`./scripts/build.sh` downloads it from
+`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`, patches the `Modelfile`
+`FROM` line into a temp copy, and creates the local `janus` tag:
+
+```bash
+cd ..  &&  QUANT=Q5_K_M ./scripts/build.sh  &&  cd examples
+python ollama_chat.py
+```
+
+### Transformers (safetensors)
+
+```bash
+pip install --upgrade "transformers>=4.45" accelerate sentencepiece bitsandbytes
+python transformers_quickstart.py            # 4-bit, ~20 GB VRAM
+python transformers_quickstart.py --no-4bit  # bf16, ~70 GB VRAM
+```
+
+### llama-cpp-python (GGUF, no daemon)
+
+```bash
+pip install llama-cpp-python  # CPU-only build
+python llama_cpp_quickstart.py /path/to/Janus-35B-A3B.Q4_K_M.gguf --gpu-layers 99
+```
+
+For GPU offload, rebuild llama-cpp-python with the matching backend — see
+the script header for `CMAKE_ARGS` recipes (CUDA, Metal, ROCm/HIP).
+
+### Vision (image input)
+
+```bash
+# Pull the projector once (~903 MB) into the repo root:
+cd ..  &&  ./scripts/fetch_vision.sh  &&  cd examples
+
+pip install llama-cpp-python pillow
+python llama_cpp_vision.py \
+  --gguf ../Janus-35B-A3B.Q4_K_M.gguf \
+  --mmproj ../Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \
+  --image /path/to/photo.jpg \
+  --prompt "Describe this image."
+```
+
+Why not Ollama? Ollama's Go engine has the `qwen35` / `qwen35moe`
+arch entries (text inference works in 0.24+), but the C++ llama.cpp
+fallback that Ollama switches to when an mmproj is attached still
+lacks them. `ollama create` accepts the dual-`FROM` and `ollama show`
+reports `vision` capability, but the first inference call fails with
+`error loading model architecture: unknown model architecture:
+'qwen35moe'`. Tracked in
+[ollama/ollama#15898](https://github.com/ollama/ollama/issues/15898).
+Until that's fixed, llama.cpp / llama-cpp-python is the working path
+for vision.

examples/llama_cpp_quickstart.py

@@ -0,0 +1,84 @@
+#!/usr/bin/env python3
+"""
+Janus-35B — llama-cpp-python quickstart.
+
+Skip Ollama entirely and call the GGUF directly through llama-cpp-python.
+Useful for batch jobs, CI, or environments where you don't want a daemon.
+
+Install:
+    pip install llama-cpp-python
+
+For GPU offload (CUDA / Metal / ROCm), install with the matching extras:
+    CMAKE_ARGS="-DGGML_CUDA=on"  pip install llama-cpp-python --no-binary :all:
+    CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python --no-binary :all:
+    CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install llama-cpp-python --no-binary :all:
+
+Usage:
+    python llama_cpp_quickstart.py /path/to/Janus-35B-A3B.Q4_K_M.gguf
+    python llama_cpp_quickstart.py /path/to/file.gguf --gpu-layers 99
+    python llama_cpp_quickstart.py /path/to/file.gguf --prompt "..."
+"""
+from __future__ import annotations
+
+import argparse
+import sys
+
+try:
+    from llama_cpp import Llama
+except ImportError:  # pragma: no cover
+    sys.exit("Missing llama-cpp-python. Install with: pip install llama-cpp-python")
+
+
+JANUS_SYSTEM = (
+    "You are Janus, a precise and capable assistant for reasoning, writing, "
+    "coding, and long-form dialogue.\n\n"
+    "Behavior rules:\n"
+    "- Answer the user's actual request directly.\n"
+    "- Be accurate, complete, and structured.\n"
+    "- Think before answering, but do not get stuck in repetitive loops.\n"
+    "- If the request is ambiguous, state what is missing and make the smallest "
+    "reasonable assumption needed to continue.\n"
+    "- Finish with a usable answer, not just planning."
+)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("gguf", help="Path to Janus-35B-A3B GGUF (e.g. Q4_K_M).")
+    ap.add_argument(
+        "--prompt",
+        default="Explain the Burrows-Wheeler transform in 200 words.",
+    )
+    ap.add_argument("--ctx", type=int, default=16384, help="Context window.")
+    ap.add_argument(
+        "--gpu-layers",
+        type=int,
+        default=0,
+        help="Layers to offload to GPU (-1 or 99 = all).",
+    )
+    ap.add_argument("--max-tokens", type=int, default=512)
+    args = ap.parse_args()
+
+    llm = Llama(
+        model_path=args.gguf,
+        n_ctx=args.ctx,
+        n_gpu_layers=args.gpu_layers,
+        verbose=False,
+    )
+
+    out = llm.create_chat_completion(
+        messages=[
+            {"role": "system", "content": JANUS_SYSTEM},
+            {"role": "user", "content": args.prompt},
+        ],
+        temperature=0.6,
+        top_p=0.95,
+        top_k=20,
+        repeat_penalty=1.05,
+        max_tokens=args.max_tokens,
+    )
+    print(out["choices"][0]["message"]["content"])
+
+
+if __name__ == "__main__":
+    main()

examples/ollama_chat.py

@@ -0,0 +1,214 @@
+#!/usr/bin/env python3
+"""
+Janus-35B — Ollama chat examples.
+
+Prerequisites (pick one):
+
+    A. From the bundled GGUF (default flow):
+        $ ../scripts/load_bundle.sh      # local `janus` tag from the bundled Q4_K_M
+        # or build another quant:
+        $ ../scripts/build.sh            # default Q4_K_M -> `ollama create janus`
+        # or straight from the Modelfile:
+        $ ollama create janus -f ../Modelfile
+
+    B. Pull straight from HF (Q4_K_M is the only bundled quant):
+        $ ollama run hf.co/FoolDev/Janus-35B-HERETIC
+        # then set MODEL=hf.co/FoolDev/Janus-35B-HERETIC below
+
+Then:
+    $ ollama serve         # usually already running
+    $ python ollama_chat.py
+
+The model emits <think>...</think> reasoning blocks before its answer.
+Current Ollama (0.24, especially with `OLLAMA_NEW_ENGINE=1`) returns the
+reasoning in a separate `message.thinking` field and keeps `content`
+clean. Older builds put the whole `<think>...</think>` block inside
+`content`. The demo below reads `message.thinking` first and falls
+back to parsing `<think>` tags out of `content` so it works against
+either path.
+
+Endpoints used:
+    - Native Ollama:  http://localhost:11434/api/chat
+    - OpenAI-compat:  http://localhost:11434/v1/chat/completions
+"""
+from __future__ import annotations
+
+import json
+import os
+import re
+import sys
+from typing import Any, Iterator
+
+import requests
+
+MODEL = os.environ.get("MODEL", "janus")
+HOST = os.environ.get("HOST", "http://localhost:11434")
+
+_THINK_RE = re.compile(r"<think>.*?</think>\s*", re.DOTALL)
+
+
+def split_thinking(content: str) -> tuple[str, str]:
+    """Return (thinking, final_answer) from a content string."""
+    parts = re.findall(r"<think>(.*?)</think>", content, re.DOTALL)
+    thinking = "\n".join(p.strip() for p in parts).strip()
+    answer = _THINK_RE.sub("", content).strip()
+    return thinking, answer
+
+
+# ---------- 1. Simple chat ----------
+
+def chat(prompt: str, system: str | None = None) -> dict[str, Any]:
+    msgs: list[dict[str, Any]] = []
+    if system:
+        msgs.append({"role": "system", "content": system})
+    msgs.append({"role": "user", "content": prompt})
+    r = requests.post(
+        f"{HOST}/api/chat",
+        json={"model": MODEL, "messages": msgs, "stream": False},
+        timeout=600,
+    )
+    r.raise_for_status()
+    return r.json()
+
+
+# ---------- 2. Streaming ----------
+
+def chat_stream(prompt: str) -> Iterator[str]:
+    """Yield content tokens as they arrive."""
+    with requests.post(
+        f"{HOST}/api/chat",
+        json={
+            "model": MODEL,
+            "messages": [{"role": "user", "content": prompt}],
+            "stream": True,
+        },
+        stream=True,
+        timeout=600,
+    ) as r:
+        r.raise_for_status()
+        for line in r.iter_lines():
+            if not line:
+                continue
+            chunk = json.loads(line)
+            if "message" in chunk and "content" in chunk["message"]:
+                yield chunk["message"]["content"]
+            if chunk.get("done"):
+                break
+
+
+# ---------- 3. Tool calling ----------
+
+WEATHER_TOOL = {
+    "type": "function",
+    "function": {
+        "name": "get_current_weather",
+        "description": "Get the current weather in a given city",
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "city": {"type": "string", "description": "City name"},
+                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
+            },
+            "required": ["city", "unit"],
+        },
+    },
+}
+
+
+def fake_weather(city: str, unit: str) -> str:
+    """Stand-in tool implementation."""
+    return json.dumps(
+        {"city": city, "temperature": 14, "unit": unit, "conditions": "light rain"}
+    )
+
+
+def tool_round_trip(prompt: str) -> str:
+    """Single-shot tool call: model -> tool -> model -> final answer."""
+    history: list[dict[str, Any]] = [{"role": "user", "content": prompt}]
+    r = requests.post(
+        f"{HOST}/api/chat",
+        json={
+            "model": MODEL,
+            "messages": history,
+            "tools": [WEATHER_TOOL],
+            "stream": False,
+        },
+        timeout=600,
+    )
+    r.raise_for_status()
+    msg = r.json()["message"]
+
+    if not msg.get("tool_calls"):
+        return msg["content"]
+
+    history.append({"role": "assistant", "tool_calls": msg["tool_calls"]})
+    for tc in msg["tool_calls"]:
+        fn = tc["function"]
+        if fn["name"] == "get_current_weather":
+            result = fake_weather(**fn["arguments"])
+        else:
+            result = json.dumps({"error": f"unknown tool {fn['name']}"})
+        history.append({"role": "tool", "tool_name": fn["name"], "content": result})
+
+    r = requests.post(
+        f"{HOST}/api/chat",
+        json={
+            "model": MODEL,
+            "messages": history,
+            "tools": [WEATHER_TOOL],
+            "stream": False,
+        },
+        timeout=600,
+    )
+    r.raise_for_status()
+    return r.json()["message"]["content"]
+
+
+# ---------- 4. OpenAI-compatible endpoint ----------
+
+def openai_chat(prompt: str) -> str:
+    r = requests.post(
+        f"{HOST}/v1/chat/completions",
+        json={
+            "model": MODEL,
+            "messages": [{"role": "user", "content": prompt}],
+            "temperature": 0.6,
+        },
+        timeout=600,
+    )
+    r.raise_for_status()
+    return r.json()["choices"][0]["message"]["content"]
+
+
+# ---------- demo ----------
+
+def _demo() -> None:
+    print("=== 1. simple chat ===")
+    resp = chat("What is 84 * 3 / 2?")
+    msg = resp["message"]
+    # Prefer the dedicated `thinking` field (Ollama 0.24+ / new engine);
+    # fall back to extracting <think>...</think> from `content` for
+    # older builds that inline the reasoning.
+    thinking = (msg.get("thinking") or "").strip()
+    answer = msg.get("content", "")
+    if not thinking:
+        thinking, answer = split_thinking(answer)
+    if thinking:
+        print(f"[thinking] {thinking[:200]}...")
+    print(f"[answer]   {answer}")
+
+    print("\n=== 2. streaming ===")
+    for tok in chat_stream("Count from 1 to 5 in one line."):
+        sys.stdout.write(tok)
+        sys.stdout.flush()
+    print()
+
+    print("\n=== 3. tool round-trip ===")
+    print(tool_round_trip("What is the weather in Paris in celsius?"))
+
+    print("\n=== 4. OpenAI-compat ===")
+    print(openai_chat("Say 'OpenAI endpoint OK' and nothing else."))
+
+
+if __name__ == "__main__":
+    _demo()

examples/transformers_quickstart.py

@@ -0,0 +1,127 @@
+#!/usr/bin/env python3
+"""
+Janus-35B — Hugging Face Transformers quickstart.
+
+Loads the Heretic Qwen 3.6 35B-A3B safetensors directly and runs a single
+chat turn using its embedded chat template. Janus-35B is a *wrapper*
+around that base, so for the transformers route there is nothing to
+download from this repo — point at llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
+and apply the same system prompt the Modelfile uses.
+
+Set MODEL_ID = "Qwen/Qwen3.6-35B-A3B" to bypass the Heretic abliteration and
+load the vanilla upstream base instead.
+
+Note: this is a mixture-of-experts model — 35B total / 3B active per token.
+Transformers keeps all 256 experts resident, so memory tracks the 35B total,
+not the 3B active count.
+
+Requirements:
+    pip install --upgrade "transformers>=4.45" accelerate sentencepiece bitsandbytes
+
+Memory:
+    - bf16 full precision: ~70 GB VRAM (multi-GPU or CPU offload).
+    - 4-bit (bitsandbytes nf4): ~20 GB VRAM, fits a 24 GB card (3090/4090)
+      at modest context.
+    - Fall back to device_map="auto" + bnb_4bit on consumer GPUs.
+
+Usage:
+    python transformers_quickstart.py
+    python transformers_quickstart.py --no-4bit         # bf16, needs ~70 GB VRAM
+    python transformers_quickstart.py --prompt "..."    # custom prompt
+"""
+from __future__ import annotations
+
+import argparse
+import sys
+
+try:
+    import torch
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+except ImportError as e:  # pragma: no cover
+    sys.exit(
+        f"Missing dependency: {e.name}. Install with:\n"
+        "  pip install --upgrade 'transformers>=4.45' accelerate sentencepiece bitsandbytes"
+    )
+
+
+MODEL_ID = "llmfan46/Qwen3.6-35B-A3B-uncensored-heretic"
+
+JANUS_SYSTEM = (
+    "You are Janus, a precise and capable assistant for reasoning, writing, "
+    "coding, and long-form dialogue.\n\n"
+    "Behavior rules:\n"
+    "- Answer the user's actual request directly.\n"
+    "- Be accurate, complete, and structured.\n"
+    "- Think before answering, but do not get stuck in repetitive loops or "
+    "meta-commentary.\n"
+    "- If the request is ambiguous or incomplete, state what is missing and "
+    "make the smallest reasonable assumption needed to continue.\n"
+    "- If the user wants creative writing, preserve tone, continuity, and "
+    "character consistency.\n"
+    "- If the user wants analysis or technical help, prefer concrete steps, "
+    "examples, and decisions over fluff.\n"
+    "- Finish with a usable answer, not just planning."
+)
+
+
+def load(use_4bit: bool):
+    kwargs: dict = {"device_map": "auto", "torch_dtype": torch.bfloat16}
+    if use_4bit:
+        from transformers import BitsAndBytesConfig
+        kwargs["quantization_config"] = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_compute_dtype=torch.bfloat16,
+            bnb_4bit_use_double_quant=True,
+        )
+        kwargs.pop("torch_dtype", None)
+
+    tok = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
+    model = AutoModelForCausalLM.from_pretrained(MODEL_ID, trust_remote_code=True, **kwargs)
+    return tok, model
+
+
+def generate(tok, model, prompt: str, max_new_tokens: int = 512) -> str:
+    messages = [
+        {"role": "system", "content": JANUS_SYSTEM},
+        {"role": "user", "content": prompt},
+    ]
+    inputs = tok.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        return_tensors="pt",
+    ).to(model.device)
+
+    out = model.generate(
+        inputs,
+        max_new_tokens=max_new_tokens,
+        do_sample=True,
+        temperature=0.6,
+        top_p=0.95,
+        top_k=20,
+        repetition_penalty=1.05,
+    )
+    return tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--prompt", default="Explain the Burrows-Wheeler transform in 200 words.")
+    ap.add_argument(
+        "--no-4bit",
+        action="store_true",
+        help="Disable 4-bit quantization (requires ~70 GB VRAM in bf16).",
+    )
+    ap.add_argument("--max-new-tokens", type=int, default=512)
+    args = ap.parse_args()
+
+    print(f"[load] {MODEL_ID} (4bit={'no' if args.no_4bit else 'yes'})")
+    tok, model = load(use_4bit=not args.no_4bit)
+
+    print(f"[gen]  prompt: {args.prompt!r}")
+    print()
+    print(generate(tok, model, args.prompt, args.max_new_tokens))
+
+
+if __name__ == "__main__":
+    main()