dolev31/tabagent-skillsbench-skill-selection

A pretrained TabAgent head that picks which skills an agent should load for a task. It scores all 195 skills of the SkillsBench catalog in one CatBoost pass and returns a ranked shortlist, so the agent receives a few skills instead of the whole library. It is the skill-selection decision of the paper TabAgent: Distilling LLM Agent Components into Cheap Classifiers, trained with the released code.

Quick start

pip install "tabagent[embed,hub]"

import tabagent
from tabagent import datasets

head = tabagent.load("hf://dolev31/tabagent-skillsbench-skill-selection")  # hashes verified, no pickle
catalog = datasets.load("skillsbench/skill-select", download=True).df[["skill_name", "skill_desc"]].drop_duplicates()
head.decide(catalog.assign(task_text="Recover the missing cells of a damaged Excel workbook."), k=5)

decide returns one row per shortlisted skill with its score and rank (0 is the best). The catalog comes from the pinned SkillsBench release, rebuilt and SHA-256 verified by tabagent.datasets (3.5 MB). The e5-small-v2 encoder behind one feature is downloaded from the Hub on first use (about 128 MB). Pin a revision for a reproducible deployment: hf://dolev31/tabagent-skillsbench-skill-selection@<commit>.

Try it without installing anything: the Colab quickstart.

What it decides

Decision	skill selection: set retrieval (ranking) over a fixed catalog
Input	the task instruction (SkillsBench task.md) and the catalog: each skill's name and the first 800 characters of its SKILL.md
Output	per skill, the probability that it belongs to the task's skill set; decide(rows, k=5) returns the top 5
Candidates	the 195 skills of SkillsBench v1.1
Label it learned	the skill is in the task's curated skill bundle (SkillsBench's own target)

The files are a standard, pickle-free TabAgent artifact: card.yaml (the feature card), fitted.json and fitted.npz (fitted statistics), featurizer.json, model.cbm (CatBoost's native format) and manifest.json with the SHA-256 of every file; tabagent.load refuses a modified file. Loading runs no code from the repository.

Results

The paper evaluates this recipe with 5-fold cross-validation grouped by task, so every score comes from tasks the model did not see. This checkpoint was then fitted on all 87 tasks, so it has no held-out score of its own, and none is reported: scoring it on SkillsBench tasks would measure its training data.

Source	Protocol	P@R	Recall@9
Paper, Table 1	5-fold task-level CV, mean of three seeds	80	91
Released code (docs/reproducing.md)	the same protocol, recomputed with python -m experiments.reproduce --cells skill-selection	79.6 [74.0, 85.4]	90.5

Percentages, macro-averaged over tasks. P@R is R-precision: the share of a task's R relevant skills found in its top R. Recall@9 is the share found in the top 9. The released code orders tied scores uniformly at random and reports the expected value; its interval is a 95% bootstrap interval over tasks.

How it was trained

• Data. SkillsBench v1.1, the official release: the audited run index of the leaderboard at benchflow/skillsbench-leaderboard@f104580363 and the task packages of benchflow-ai/skillsbench@27738384b1, rebuilt by tabagent.datasets.load("skillsbench/skill-select"). 87 tasks × 195 skills, 16,965 rows.
• Features (the bundled card skillsbench_skill_select_noprior, the card of the paper's row): the task instruction, the skill's name and the opening of its SKILL.md, the skill name's length, a centered e5-small-v2 cosine between task and skill (centering fitted on the training rows), and their lexical overlap.
• Head. CatBoost TabHead with the paper's settings (300 iterations, learning rate 0.08, depth 6), seed 20260720 (the first of the paper's three seeds), real rows only (no TabSynth, as in the paper).
• Fit. Once, on all 87 tasks, with no held-out fold, using the released code at commit a26e803 (tabagent 0.1.0.dev0).

Limitations and intended use

From the paper's scope and limitations:

• In-distribution only. TabAgent learns a decision from an existing pool of examples and does not zero-shot to a new domain. This head ranks the 195 SkillsBench skills for tasks like SkillsBench's; skills outside that catalog, or very different tasks, are out of distribution.
• Tied to the agent's architecture and decision. The features and the boundary are those of one decision in one setting; another agent or catalog needs its own head (the library onboards one from logs).
• Inherits its training labels. A distilled head reproduces the choices it was trained on, biases included. Here the labels are SkillsBench's curated skill bundles.

License

The model is released under Apache-2.0, the license of the TabAgent code. It was trained on SkillsBench data, released under Apache-2.0 by its authors; the e5-small-v2 encoder it calls is MIT-licensed.

Citation

If you use this model, please cite the paper.

@misc{levy2026tabagent,
  title  = {TabAgent: Distilling LLM Agent Components into Cheap Classifiers},
  author = {Levy, Ido and Shlomov, Segev and Yaeli, Avi and Mashkif, Nir and Adi, Asaf and
            Goldstein, Yinon and Shapira, Eilam and Choshen, Leshem},
  year   = {2026},
  eprint = {2602.16429},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi    = {10.48550/arXiv.2602.16429},
  url    = {https://arxiv.org/abs/2602.16429}
}

Authors: Ido Levy (IBM; Weizmann Institute of Science), Segev Shlomov, Avi Yaeli, Nir Mashkif, Asaf Adi, Yinon Goldstein, Eilam Shapira (IBM), Leshem Choshen (IBM; Weizmann Institute of Science).

Links

• Paper: arXiv:2602.16429
• Code: github.com/dolev31/TabAgent
• Docs: dolev31.github.io/TabAgent
• Try it: Colab quickstart
• Install: pip install tabagent (PyPI)