| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.6-27B |
| tags: |
| - coding |
| - agent |
| - structured-cot |
| - sft |
| - merged-lora |
| - r2e-gym |
| datasets: |
| - andthattoo/etpi-sft |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| |
| # etpi-phase1 — Structured-CoT SFT seed for Qwen 3.6 27B |
|
|
| `etpi-phase1` is **Qwen 3.6 27B** with one round of supervised fine-tuning |
| on grammar-constrained, structured-CoT coding-agent trajectories. The |
| adapter (LoRA rank 128, `target_modules=all-linear`) has been **merged into |
| the base weights** — this is a normal HuggingFace model checkpoint with |
| no PEFT runtime dependency. |
|
|
| This is **phase 1** of a multi-phase training pipeline. Phase 2 is RLVR |
| (GRPO) against R2E-Gym, starting from this checkpoint. |
|
|
| ## What's different about this model |
|
|
| The base Qwen 3.6 27B reasons in long free-form chains-of-thought |
| (hundreds-to-thousands of tokens per turn). This SFT seed teaches the |
| model a **compact 3-slot IR** inside `<think>...</think>`: |
|
|
| ``` |
| <think> |
| STATE: <one short line: current workspace state> |
| ACTION: <one short line: what to do next and why> |
| EXPECT: <one short line: what the observation should look like> |
| </think> |
| <tool_call>...</tool_call> # or whatever the harness expects |
| ``` |
|
|
| Typical thinking-token spend per turn drops from **~1000+ → ~50-100** |
| while preserving multi-turn coherence and tool-use ability. |
|
|
| ## Intended use |
|
|
| Drive a multi-turn coding agent — e.g., R2E-Gym / SWE-bench / |
| Terminal-Bench style sandboxes — where the model: |
|
|
| 1. Reads a task instruction (GitHub issue, terminal prompt, etc.) |
| 2. Issues tool calls in a loop (bash, file editor, search) |
| 3. Observes results and iterates |
| 4. Submits when verified |
|
|
| The IR structure compresses the *thinking phase*; tool calls and answers |
| are left unconstrained. |
|
|
| ## Training details |
|
|
| | | | |
| |---|---| |
| | Base model | `Qwen/Qwen3.6-27B` | |
| | Training data | [`andthattoo/etpi-sft`](https://huggingface.co/datasets/andthattoo/etpi-sft) (318 verified-successful multi-turn trajectories) | |
| | Method | LoRA r=128, α=256, dropout 0.05, target_modules=all-linear, merged at end of training | |
| | Optimizer | AdamW (8-bit), lr 2e-4, cosine schedule, warmup 0.03 | |
| | Epochs | 2 | |
| | Batch | per-device 1, grad accum 8 (effective batch 8) | |
| | Sequence length | 8192 | |
| | Loss masking | assistant tokens only (observations and user messages masked) | |
| | Other | gradient checkpointing on, bf16, Liger fused CE loss | |
| | Hardware | 1× H100 80GB (~40 min) | |
| |
| Final metrics: |
| - train_loss (final): 0.096 |
| - mean_token_accuracy: 95.87% |
|
|
| ## Data provenance |
|
|
| Trajectories were generated by running grammar-constrained |
| Qwen 3.6 27B as a multi-turn agent against **R2E-Gym-Lite** (real GitHub |
| issue tasks). The grammar enforced the IR structure inside `<think>` |
| while leaving tool calls free. Each task was rolled out 4 times (K=4 |
| best-of-N). Only `reward=1.0` trajectories (passed the R2E unit-test |
| verifier) were kept; among those, the **shortest** trajectory per task |
| was selected as the SFT target — encoding a brevity-given-correctness |
| preference directly into the data. |
|
|
| See [`andthattoo/etpi-sft`](https://huggingface.co/datasets/andthattoo/etpi-sft) |
| for the full dataset including system prompt, task instructions, and |
| per-turn messages. |
|
|
| ## Limitations |
|
|
| - **Small training set (318 examples).** SFT-seed scale, not full SFT. |
| Expected to generalize the IR format rather than acquire new |
| capabilities. The phase-2 RL run is where capability climbs. |
| - **Trained on one task distribution** (R2E-Gym, Python SWE-style |
| issues). Performance on other languages or task types is untested. |
| - **R2E paper authors use the Terminus 2 scaffold with an 80k-token- |
| per-turn budget** to report 77.2% on SWE-bench Verified for the base |
| Qwen 3.6 27B. This model is intentionally trained under a tighter |
| scaffold (bash-only loop, IR-constrained thinking) with a different |
| efficiency objective — direct numerical comparison is not apples to |
| apples. |
| - **Phase 1 only.** No RL has been applied. The expected pass@1 lift on |
| R2E-Gym from SFT alone is modest; the real lift comes from phase 2. |
|
|
| ## Recommended inference |
|
|
| The model has internalized the IR format. You can drive it with or |
| without grammar enforcement at inference time: |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| m = AutoModelForCausalLM.from_pretrained("andthattoo/etpi-phase1", torch_dtype="bfloat16", device_map="auto") |
| t = AutoTokenizer.from_pretrained("andthattoo/etpi-phase1") |
| |
| messages = [ |
| {"role": "system", "content": "You are a software-engineering agent. ..."}, |
| {"role": "user", "content": "<github-issue text>"}, |
| ] |
| prompt = t.apply_chat_template(messages, add_generation_prompt=True, tokenize=False) |
| ids = t(prompt, return_tensors="pt").to(m.device) |
| out = m.generate(**ids, max_new_tokens=512, temperature=0.0) |
| print(t.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| For maximum compression-safety in production, apply a GBNF grammar that |
| enforces the IR shape on the `<think>` block. |
|
|
| ## License |
|
|
| Apache 2.0 (matches base Qwen 3.6 27B license). |
|
|
| ## Acknowledgements |
|
|
| - Base model: [Qwen team](https://qwen.ai) |
| - Training environment: [R2E-Gym](https://github.com/R2E-Gym/R2E-Gym) |
| - Inference: [SGLang](https://github.com/sgl-project/sglang) |
| - Tooling: TRL, PEFT, Liger Kernel, bitsandbytes |
|
|
