EdwinS9/qwen3-8b-sdft-robot-planner

Qwen3-8B · SDFT Robot Skill Planner (LoRA adapter)

A LoRA adapter for Qwen/Qwen3-8B, trained with Self-Distillation Fine-Tuning (SDFT) to turn a natural-language instruction plus a JSON scene description into a sequence of robot skill calls (navigate_to, pick, place, open, close, push, detect, done).

This is a research reproduction of the method in Self-Distillation Enables Continual Learning (Shenfeld, Damani, Hübotter & Agrawal, arXiv:2601.19897). Code: https://github.com/EdwinS9/qwench

Method (brief)

SDFT distills a demonstration-conditioned teacher (the model shown the example's own gold demonstration in context) into the zero-shot student (no demonstration), using an analytic per-token forward KL — KL(teacher ‖ student) — computed on the student's own on-policy sampled plans (pure temperature-1.0 sampling), with an EMA-of-the-student teacher (α=0.01/step). The aim is to absorb in-context skill into the weights while staying close to the base model's distribution. Note: the paper's Eq. 1 writes the reverse KL, but the official implementation's README clarifies the published results used the per-token forward KL, which is what this repo implements.

Provenance note (read this)

Checkpoints published before 2026-07-03 were trained with a pre-audit variant that diverges from the paper in two ways: the in-context demonstration was a different same-family example (not the example's own demonstration), and the loss was the per-token reverse KL. An audit against the official implementation (github.com/idanshen/Self-Distillation) identified both divergences; see SPEC.md ("Corrections") in the code repo. The evaluation numbers below describe the variant checkpoint. A retrain with the corrected, paper-faithful recipe is planned; until then treat this adapter as an SDFT variant, not a faithful reproduction.

Intended use

Input: a system prompt (see the repo's qwench/prompts.py), the instruction, and a JSON scene state. Output: a single JSON object {"thinking": ..., "plan": [{"skill": ..., "args": {...}}]} ending in a done step. Qwen3 "thinking" mode should be disabled (enable_thinking=False).

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "EdwinS9/qwen3-8b-sdft-robot-planner")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

Training data

Procedurally generated, fully verified (every gold plan is executed and must reach its goal): ~668 train / 132 held-out examples across four task families — PickAndPlace, Open/Close articulated, Push, Stack. No human labels, no API. See the repo for the generator.

Evaluation

Graded by a symbolic world-model executor (parse → schema → execution → goal):

• Plan-success on the full 132-example held-out split: ≈61% (this checkpoint).
• MMLU (general capability): ≈0.71, statistically unchanged from base Qwen3-8B (≈0.71) — i.e. no measurable forgetting under this LoRA setup (see caveat below).

The full 132-example split is the headline number; the 64-example in-training eval subset is easier and scores higher. This checkpoint is saved early (at the first in-training peak), so it under-fits the harder task families such as multi-step stacking, where most failures land.

Limitations (read before relying on this)

• Symbolic grader, not a physics simulator. Plans are validated against a symbolic world model, not ManiSkill or a real robot. Physical executability is not demonstrated.
• Templated task. Instructions/scenes are templated; the task is comparatively easy once the output format is learned.
• Forgetting not demonstrated vs. SFT. Both SFT and SDFT leave MMLU essentially unchanged here — expected, because LoRA trains ~1% of parameters and is inherently forgetting-resistant. SDFT's anti-forgetting advantage is a full fine-tuning phenomenon and is not shown by this LoRA adapter. Treat this as a method reproduction, not a SOTA claim.
• Vocabulary is limited to the 8 skills above and the four task families.

Citation

@misc{shenfeld2026selfdistillation,
  title         = {Self-Distillation Enables Continual Learning},
  author        = {Shenfeld, Idan and Damani, Mehul and H\"ubotter, Jonas and Agrawal, Pulkit},
  year          = {2026},
  eprint        = {2601.19897},
  archivePrefix = {arXiv}
}