---
license: agpl-3.0
language:
- en
library_name: transformers
tags:
- qwen
- qwen3
- qwen3.6
- moe
- distillation
- chain-of-thought
- agentic
- claude-fable-5
- claude-opus-4.7
- tool-use
- chained-distill
- qwable
- qwable-v2
pipeline_tag: text-generation
base_model:
- lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled
datasets:
- lordx64/fable-sft-combined-v2
- lordx64/agentic-distill-fable-5-sft
- lordx64/fable-tool-use-sft
---

# Qwable-v2

> **Qwen + Fable, second iteration** · An open-weights agentic coding model.
> 35B Mixture-of-Experts (3B active), built by layering Claude Fable-5 agentic tool-use behavior on top of a Claude Opus 4.7 reasoning distill of Qwen3.6-35B-A3B. Trained with **4× the LoRA capacity** and **2× the SFT data** of [Qwable-v1](https://huggingface.co/lordx64/Qwable-v1).

[![v1 model](https://img.shields.io/badge/🤗_Previous-Qwable--v1-blue)](https://huggingface.co/lordx64/Qwable-v1)
[![Base model](https://img.shields.io/badge/🤗_Base-Qwen3.6--35B--A3B--Claude--4.7--Opus--Reasoning--Distilled-blue)](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled)
[![Dataset](https://img.shields.io/badge/🤗_SFT-fable--sft--combined--v2-orange)](https://huggingface.co/datasets/lordx64/fable-sft-combined-v2)
[![License](https://img.shields.io/badge/license-AGPL_3.0-blue)](./LICENSE)

## TL;DR

Qwable-v2 is the second iteration of the [Qwable lineage](https://huggingface.co/lordx64/Qwable-v1) — same chained-distill structure (vanilla Qwen3.6 → Opus 4.7 reasoning → Fable-5 agentic), but with a beefier training recipe targeting the dominant v1 failure mode (early-stop-mid-tool_use, ~48% of completed SWE-bench Lite runs in v1).

- **Thinks** in explicit `<think>…</think>` chains-of-thought (inherited from the Opus 4.7 prior)
- **Acts** like a Claude-Code-style agent when prompted as one — emits `<tool_use>` XML for file edits, shell, reads. v2 uses **real Claude Code tool names** (`Read`, `Edit`, `Bash`) with **correct field signatures** (`file_path`, `old_string`, `new_string`) where v1 was inventing variants (`read_file`, `Replace`, `regex`/`replace`).
- The XML format is still **system-prompt-conditional**: it appears reliably with an agent-style system prompt or a preceding `<tool_result>` turn. Bare prompts fall back to text-style explanations.
- Runs on a single H200 / 2× A100-80GB at bf16, or any 24+ GB consumer GPU at IQ4_XS quantization.

## What changed vs v1

Single-purpose comparison — same lineage, same base, same training infrastructure. Only these knobs moved:

| Setting | v1 (shipped 2026-06-15) | **v2 (shipped 2026-06-27)** | Why |
|---|---|---|---|
| **SFT dataset** | `agentic-distill-fable-5-sft` (4,659 rows) | **`fable-sft-combined-v2` (9,842 rows)** = v1 ∪ `fable-tool-use-sft` (5,183 rows), 0% SHA-256 overlap on user content | 2× the unique training rows; mixes with-`<think>` (46%) and without-`<think>` (54%) for conditional reasoning |
| **LoRA rank** | 16 | **64** (4× v1) | Target the closing-tag pattern that v1 LoRA capacity couldn't fully learn — 145/300 SWE-bench Lite empties were unclosed `<tool_use>` blocks |
| **Sequence length** | 4,096 | **8,192** (2× v1) | Full agentic conversations fit; closing tags can't get clipped by seq budget |
| **Epochs** | 2 | **3** | More reps over the larger corpus |
| **Learning rate** | 2e-5 | **1.5e-5** | Gentler updates over the longer run; protects the Opus 4.7 reasoning prior from catastrophic forgetting |
| **Target modules** | attention-only (`q/k/v/o`) | same | Adding MLP touches MoE expert weights, risks re-triggering the unsloth-zoo shape bug we fixed for v1 |
| Effective batch | 16 (1 × grad-accum 16) | same | Stable from v1 |
| Base (warm-start) | `lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` | same | Same A/B is cleaner |
| Wall-clock | 14.1h on Seoul H200 | **37.5h** on us-east-2 H200 (1 first attempt lost to an HF payment hold mid-run; 2nd run completed clean) | r=64 + seq=8192 + 2× steps = ~5× the per-run compute |
| Cost | ~$70 | **~$190** for the successful 2nd run (+$200 for the lost first run = $390 total v2 spend) | Same $5/hr H200 rate |
| Final loss | 0.7956 (last-20 avg) | **TBD** (run completed cleanly; computing final-window avg) | — |

## Spot-check (post-training, 2026-06-27)

Three identical probe variants against v1 and v2 (port-change task, real Fable-5-style prompt):

| Variant | v1 output | v2 output |
|---|---|---|
| Bare prompt, generic system | Markdown bash/python explanation (Opus prior dominates) | `<tool_code>...</tool_code>` Gemini-style block (still no `<tool_use>` XML, but at least a tool-call-shaped envelope) |
| Agent system prompt | ✅ `<tool_use name="read_file" id="toolu_01abc">` — **invented tool name**, made-up id | ✅ `<tool_use name="Read" id="toolu_01V21JqR711Vf45JL6o2Y351">` — **real Claude Code tool name**, real Anthropic-format id |
| Multi-turn with prior `<tool_result>` | ✅ `<tool_use name="Replace">{"regex":..., "replace":...}` — invented Replace tool with synthesized fields | ✅ `<tool_use name="Edit">{"file_path":..., "old_string":..., "new_string":...}` — **real Claude Code Edit signature with correct field names** |

**The structural improvement is real and measurable.** v2 has internalized the actual Claude Code tool surface (names + field schemas), where v1 was inventing plausible-sounding variants. The system-prompt-conditionality wasn't fully fixed — bare prompts still fall back — but the fallback is now a different XML envelope (`<tool_code>`) rather than plain markdown.

**Definitive proof of the early-stop-mid-tool_use fix** (the dominant v1 ceiling on SWE-bench Lite) still needs the full SWE-bench Lite re-run — that's pending and the number will land in the [Evaluation](#evaluation) section when it does.

## Honest scope

Same fundamentals as v1, just stronger on the things that already worked:

```
Qwen3.6-35B-A3B (vanilla, Apache 2.0)
  └─SFT─▶ Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled
           └─SFT─▶ Qwable-v1
                     ─── (same warm-start, not chained)
           └─SFT─▶ Qwable-v2  ← you are here
```

The new SFT corpus is the **union** of v1's data + a tool-use-only re-pack of `Glint-Research/Complete-FABLE.5-traces-2M` (=our [`fable-tool-use-sft`](https://huggingface.co/datasets/lordx64/fable-tool-use-sft)). Both upstream sources are signature-verified real Fable-5 captures, with the SFT-side text re-rendered for Qwen chat template + tool envelope serialization.

**For pure reasoning** (math, science, general Q&A): omit the agent system prompt. The Opus 4.7 distill underneath is what's doing the work. v2 should match v1 here (the lower LR was designed to protect the prior from forgetting).
**For agentic coding** (edit-this-file, run-this-test, scroll-this-codebase): supply an agent system prompt naming `<tool_use>` XML. v2 produces structurally cleaner tool calls than v1 (real Claude Code names + fields).
**For chat / general assistant**: works; persona may drift toward Claude voice (double Anthropic SFT stacking, same as v1).

## Training recipe (concrete reference)

| | |
|---|---|
| Base (warm-start) | `lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` |
| SFT dataset | [`lordx64/fable-sft-combined-v2`](https://huggingface.co/datasets/lordx64/fable-sft-combined-v2) — 9,842 unique rows, ~24.5M Qwen tokens |
| Library | [Unsloth](https://github.com/unslothai/unsloth) `FastLanguageModel` + TRL `SFTTrainer` |
| LoRA | r=64, alpha=64, attention-only (`q_proj, k_proj, v_proj, o_proj`), dropout 0.0 |
| Loss masking | `train_on_responses_only` (gradients only flow through assistant turns, including `<think>` block) |
| Sequence length | 8,192 |
| Epochs | 3 |
| Effective batch size | 16 (per-device 1 × grad-accum 16) |
| Optimizer | AdamW 8-bit, cosine LR, 3% warmup, weight decay 0.01 |
| Learning rate | 1.5e-5 |
| Precision | bf16 forward + LoRA params |
| Random seed | 3407 |
| Hardware | 1× nvidia-h200 x1 (141 GB) on AWS us-east-2 via HF Inference Endpoints |
| Total optimizer steps | 1,839 (9,842 rows × 3 epochs ÷ effective batch 16; small drop from prep for label-all-masked rows) |
| Wall-clock | 37.5h (one prior 40h attempt lost to a billing hold ~98% through training) |
| Cost | ~$190 (successful run only) + ~$200 (lost attempt) = ~$390 total v2 spend |
| Final save | `merged_16bit` via Unsloth |

The training script is `training/train.py` in the [source repo](https://github.com/lordx64/distillation); the submitter is `training/endpoint/deploy_fable.py --v2`.

## Evaluation

> 🚧 **Evals are in progress.** This table will fill in as each suite completes; nothing here is published until verified.

| Benchmark | Setup | v1 score | v2 score | Status |
|---|---|---:|---:|---|
| **GSM8K-CoT** | 8-shot multi-turn, limit 300 | _pending_ | _pending_ | 🚧 in progress |
| **MMLU-Pro** | 5-shot multi-turn, limit 500 | _pending_ | _pending_ | 🚧 in progress |
| **GPQA Diamond** | 0-shot CoT | _pending_ | _pending_ | 🚧 in progress |
| **MATH-500** | 0-shot, `math_verify` metric | _pending_ | _pending_ | 🚧 in progress |
| **AIME 2024 / 2025** | 0-shot CoT | _pending_ | _pending_ | 🚧 in progress |
| **HumanEval / MBPP** | pass@1 / pass@10 | _pending_ | _pending_ | 🚧 in progress |
| **IFEval** | 0-shot | _pending_ | _pending_ | 🚧 in progress |
| **SWE-bench Lite** (hand-rolled harness, non-empty patches) | 300 instances, no Docker test execution | **109/254 = 42.9%** of valid runs | _pending_ | 🚧 v2 re-run pending |
| **SWE-bench Lite — Resolved %** | Official Docker eval on generated patches | _pending_ | _pending_ | 🚧 Docker harness setup pending |

Standing rule on this project: numbers stay blank until verified. If a benchmark hits a known extraction bug, we omit it rather than publish a misleading score.

## Usage

### Transformers (full bf16, ~70 GB)

**Important**: Qwable-v2 emits `<tool_use>` XML reliably only when prompted as an agent. Same recipe as v1 — use a system prompt that explicitly requests the XML format:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("lordx64/Qwable-v2")
model = AutoModelForCausalLM.from_pretrained(
    "lordx64/Qwable-v2",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM = (
    "You are a coding agent. When you need to read, write, edit, or run code, "
    "emit XML tool calls in this exact format:\n"
    '<tool_use name="X" id="toolu_01abc">\n{"...": "..."}\n</tool_use>\n'
    "Do NOT respond with markdown code blocks. Always use <tool_use> XML."
)
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": "Read /tmp/server.py and tell me what port it listens on."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
                                  return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=2048, temperature=0.6, top_p=0.9)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=False))
```

For pure reasoning use (math, science, general Q&A), omit the system prompt or use the generic `"You are a helpful AI assistant."` — the model will produce reasoning + a text answer like the underlying Opus 4.7 distill.

### vLLM serving

```bash
vllm serve lordx64/Qwable-v2 \
    --max-model-len 16384 \
    --tensor-parallel-size 2 \
    --trust-remote-code
```

### llama.cpp / LM Studio (GGUF)

```bash
# Quants build pending; same flow as v1's lordx64/Qwable-v1-GGUF (IQ4_XS / Q4_K_M / Q5_K_M / Q8_0)
llama-cli -m Qwable-v2-IQ4_XS.gguf -p "..."
```

## Tool-use format

Same as v1 — custom `<tool_use>` XML envelope. v2's key improvement is **fidelity to the real Claude Code tool inventory**:

```
<think>
The user wants to change the port. I should Read the file first, then Edit it.
</think>

<tool_use name="Read" id="toolu_01ABC...">
{
  "file_path": "/tmp/server.py"
}
</tool_use>
```

Then on the next turn the user supplies a `<tool_result>`:

```
<tool_result id="toolu_01ABC..." is_error="false">
from flask import Flask
...
app.run(port=8000)
</tool_result>
```

And v2 produces:

```
<tool_use name="Edit" id="toolu_01XYZ...">
{
  "file_path": "/tmp/server.py",
  "old_string": "    app.run(port=8000)",
  "new_string": "    app.run(port=8080)"
}
</tool_use>
```

Versus v1 which invented `<tool_use name="Replace">{"regex": ..., "replace": ...}`. The v2 names + field signatures match Anthropic's published Claude Code tool definitions.

## Limitations

- **Tool-use format is still system-prompt-conditional.** Improved over v1 but not fully fixed. Bare prompts produce a `<tool_code>` Gemini-style envelope instead of `<tool_use>` XML; agent system prompts produce the right format reliably. Same operational guidance as v1: run inside a harness that supplies a tool-use system prompt + tool registry.
- **Narrow training distribution** — even with the 2× corpus, both source datasets ultimately derive from the same hundreds of Glint-captured Claude Code sessions. Out-of-distribution agent tasks (DevOps, data science, security workflows) may still be hit-or-miss.
- **Custom tool envelope.** `<tool_use>` XML doesn't slot into vLLM's tool-calling API automatically; need a parser wrapper.
- **Persona drift** — two SFT rounds against Anthropic-style outputs, now reinforced over more steps. May produce a model that occasionally self-identifies as Claude in chat.
- **Reasoning is still from Opus 4.7, not Fable-5.** Don't expect v2 to outperform the underlying Opus 4.7 distill on pure-reasoning benchmarks. The lower LR (1.5e-5 vs v1's 2e-5) was deliberately chosen to protect the reasoning prior; verifying that worked is pending the lm-eval suite.
- **No formal evals at v2 ship time.** Same standing rule as v1 — pending.
- **Two paid runs to ship.** First v2 attempt completed ~98% of training before an HF billing hold paused the endpoint; the second clean run cost ~$190 on top of the ~$200 lost first attempt. v3 will add checkpoint+resume to mitigate this.

## License & terms

Inherits **AGPL-3.0** from the upstream `Glint-Research/Fable-5-traces` + `Glint-Research/Complete-FABLE.5-traces-2M` datasets. Downstream users running Qwable-v2 in a network-accessible service must comply with AGPL §13 (source disclosure for network use).

The underlying Fable-5 thinking traces are derivative content from Anthropic's `claude-fable-5` preview model (suspended globally 2026-06-22 under U.S. export-control directives). Downstream users should verify compliance with [Anthropic's usage policies](https://www.anthropic.com/legal/usage-policy) for their specific use case.

The Qwen3.6-35B-A3B base is Apache 2.0; the Opus 4.7 distill (intermediate base) is Apache 2.0. Qwable-v2's AGPL designation supersedes those due to the Fable-5 data's AGPL upstream.

## Citation

```bibtex
@misc{lordx64_qwable_v2_2026,
  title  = {Qwable-v2: Agentic coding distillation from Claude Fable-5 onto Qwen3.6-35B-A3B with LoRA r=64 + combined corpus},
  author = {lordx64},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/lordx64/Qwable-v2}},
}
```

## Acknowledgements

Same lineage as v1, with the added v2 contributions:

- **`Glint-Research`** for both the original `Fable-5-traces` (v1's source) and `Complete-FABLE.5-traces-2M` (the upstream for `fable-tool-use-sft`, v2's added corpus).
- **`1EYE4ALL`** and **`Crownelius`** for the intermediate hosts in the `Complete-FABLE.5-traces-2M` provenance chain.
- **`TeichAI`** for the upstream collection that all of this ultimately traces to.
- **Anthropic** for the Claude Fable-5 preview model (briefly available 2026-06-10 to 2026-06-22) and the prior Opus 4.7 / Opus 4.6 work this lineage is built on.
- **Qwen team** for releasing Qwen3.6-35B-A3B under Apache 2.0.
- **[Unsloth](https://github.com/unslothai/unsloth)** for 2× faster LoRA training and the MoE+LoRA shape fix in unsloth-zoo PR [#601](https://github.com/unslothai/unsloth-zoo/pull/601).
- **HuggingFace** for the Inference Endpoint H200 fleet where the training actually ran.