| --- |
| base_model: thinkingmachines/Inkling-Small |
| library_name: peft |
| tags: |
| - tinker |
| - peft |
| - lora |
| - base_model:adapter:thinkingmachines/Inkling-Small |
| tinker_path: tinker://0e3342b7-6f34-5a67-9009-f94aad6ae787:train:0/sampler_weights/claude-code-r32 |
| --- |
| |
| # Inkling-Small LoRA: claude-code |
|
|
| Rank-32 LoRA adapter for [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small), |
| trained on Tinker from [`deburky/gpt-oss-claude-code`](https://huggingface.co/datasets/deburky/gpt-oss-claude-code). |
|
|
| ## Training |
|
|
| | | | |
| | --- | --- | |
| | Base | `thinkingmachines/Inkling-Small` (276B total / 12B active MoE) | |
| | Method | LoRA, rank 32, alpha 32, `target_modules: all-linear` | |
| | Modules | attention + MoE expert MLPs (`train_mlp=True`) | |
| | Optimizer | Adam, lr 1e-4, 2 epochs, batch 8 | |
| | Data | 284 train / 71 validation chat records | |
| | Platform | Tinker | |
|
|
| Held-out loss went from **1.5374** (base) to **0.7486**. |
|
|
| The source dataset is in gpt-oss harmony format with tool calls encoded as JSON |
| inside the final channel. It was parsed back to structured messages and |
| re-rendered with Inkling's renderer, converting those to native tool calls. |
|
|
| ## Usage |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM |
| |
| base = AutoModelForCausalLM.from_pretrained("thinkingmachines/Inkling-Small") |
| model = PeftModel.from_pretrained(base, "deburky/inkling-small-claude-code") |
| ``` |
|
|
| Serving with vLLM, which supports MoE expert LoRA via `FusedMoEWithLoRA`: |
|
|
| ```bash |
| vllm serve thinkingmachines/Inkling-Small-NVFP4 \ |
| --enable-lora \ |
| --lora-modules claude-code=deburky/inkling-small-claude-code \ |
| --max-lora-rank 32 |
| ``` |
|
|
| The adapter uses the "shared-outer" MoE layout: `w1`/`w3` `lora_A` and `w2` |
| `lora_B` are stored once with expert-dim 1, the rest per-expert across 256 |
| experts. |
|
|
| ## Honest evaluation |
|
|
| Held-out loss more than halved, but **this adapter does not beat the base model |
| on behaviour**, and it is not recommended over base for agentic use. |
|
|
| Scored in real Claude Code sessions (3 tasks x 6 repeats, tool access, checkable |
| ground truth): |
|
|
| | Model | Held-out loss | Agentic (n=18) | |
| | --- | --- | --- | |
| | base `Inkling-Small` | 1.5374 | 18/18 | |
| | this adapter | 0.7486 | 18/18 | |
| | attention-only variant | 0.8175 | 12/18 | |
|
|
| On a harder multi-step task (discover files, read several, judge a bug) the |
| adapter needed 2-3x the turns and wall-clock of base, dropped the `file:line` |
| citations base gives consistently, and confabulated a path outside the repo in |
| one run of three. |
|
|
| The base model already handled tool calling and prose style well, so most of the |
| loss reduction reflects fitting the corpus's phrasing rather than new capability. |
|
|
| ## Size |
|
|
| 7.9 GB in F32, because `train_mlp=True` puts a LoRA pair on all 256 experts per |
| layer: 96% of the bytes are expert MLPs, 2% attention. An attention-only variant |
| is 185 MB but scores worse (12/18 above). Casting to bf16 would roughly halve |
| this file. |
|
|
| Note the asymmetry: the adapter is 8 GB, but the base it attaches to is 532 GB in |
| BF16. Serving realistically means a quantized base (`Inkling-Small-NVFP4` at |
| 171 GB) on multi-GPU. |
|
|
