simonlqy commited on
Commit
58ad1f2
·
verified ·
1 Parent(s): 5502bc9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +66 -0
README.md ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: qwen
4
+ license_link: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE
5
+ base_model: Qwen/Qwen3.5-9B
6
+ pipeline_tag: text-generation
7
+ library_name: transformers
8
+ tags:
9
+ - agent
10
+ - tool-use
11
+ - skill-selection
12
+ - reinforcement-learning
13
+ - grpo
14
+ ---
15
+
16
+ # SkillGate-9B
17
+
18
+ Policy from **"SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents."**
19
+
20
+ Agent frameworks expose *skills* — instruction files with a name, a one-line
21
+ description and a body — by progressive disclosure: the agent sees only names and
22
+ descriptions and must open a file to learn what is inside. With thousands of skills in
23
+ a library, *which* one to read becomes a decision the policy makes mid-episode, and
24
+ outcome-rewarded RL cannot teach it: the tokens naming the chosen skill carry a median
25
+ 0.14% of their trajectory's loss weight, and two in five of them receive a *negative*
26
+ advantage because execution afterwards failed.
27
+
28
+ SkillGate partitions one trajectory's token support into two disjoint credit channels:
29
+ outcome credit reaches only execution tokens (the whole skill-read call is removed from
30
+ the task loss), while an action-local advantage reaches exactly the skill-naming
31
+ tokens, positive only when the trajectory's single read is the correct skill.
32
+
33
+ ## Model
34
+
35
+ | | |
36
+ |---|---|
37
+ | Base | Qwen3.5-9B |
38
+ | Training | 100 steps on-policy GRPO, 491 tasks, 8 rollouts/prompt, global batch 128, lr 1e-6, KL 3e-5, selector coefficient 0.20 |
39
+ | Checkpoint | `iter_0000099`, the final step (`selection_role: final`) |
40
+ | Architecture | `Qwen3_5ForConditionalGeneration` |
41
+
42
+ ## Results (385-trial protocol, 5 agentic benchmarks, 16-candidate slate)
43
+
44
+ | Method | Overall | Oracle read | Misleading read |
45
+ |---|---:|---:|---:|
46
+ | SFT (RL init) | 40.8 | 37.9 | 61.8 |
47
+ | SkillRL (outcome reward only) | 47.0 | 54.3 | 69.6 |
48
+ | **SkillGate** | **53.2** | **83.9** | **21.8** |
49
+
50
+ Same initialisation, data, steps and hyperparameters as the outcome-only row; the only
51
+ difference is which tokens the gradient reaches.
52
+
53
+ ## Intended use
54
+
55
+ Research on agentic skill/tool selection. The model expects the OpenClaw-style prompt
56
+ profile and tool schema used in the paper; see the repository for the exact system
57
+ prompt and the frozen skill slates.
58
+
59
+ ## Links
60
+
61
+ - Paper: (arXiv link to follow)
62
+ - Code: https://github.com/DeepExperience/SkillGate
63
+
64
+ ## License
65
+
66
+ Derived from Qwen3.5-9B and distributed under the Qwen license; see `license_link`.