Papers
arxiv:2611.26510

AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher

Published on Nov 19
ยท Submitted by
Guoxin Chen
on Nov 20
Authors:
,
,
,
,
,

Abstract

Task success alone does not specify which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. We propose Skill-FT (Skill-Guided Fine-Tuning), which uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on the resulting trajectories. We introduce AutoMaster, an autonomous post-training system in which an LLM agent develops these interventions through trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation. Across Terminal, Search, and software engineering tasks with 4B and 9B target models, AutoMaster improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42-8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed. Together with the ablations and trajectory analyses, these results suggest that behavioral shaping can make a fixed model a more effective teacher. We further find that retaining skill instructions during SFT yields higher mean performance than removing them across all six settings, despite their absence at evaluation and identical supervised outputs. These findings suggest that preserving skill context as part of a training demonstration can support better behavioral internalization.

Community

Paper submitter

Task success does not guarantee good teaching. Even successful agent trajectories may contain undesirable behaviors that SFT can inadvertently reinforce. We introduce AutoMaster, an autonomous post-training framework that makes the same model a better teacher through Skill-FT. AutoMaster autonomously analyzes behavioral patterns, designs and refines natural-language skills to shape teacher rollouts, and selects valuable demonstrations through iterative feedback. Across Terminal, Search, and SWE benchmarks with 4B and 9B models, AutoMaster outperforms Vanilla SFT by 1.42โ€“8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed.

Our analysis reveals two key insights:
(1) skill guidance improves demonstrated behaviors even when both guided and unguided trajectories succeed;
(2) retaining skill instructions during SFT improves performance by 2.15โ€“4.01 points over removing them, highlighting the importance of skill context for behavioral internalization. Better post-training requires shaping what teachers demonstrate, not merely selecting successful trajectories.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.26510
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.26510 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.26510 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.26510 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.