---
license: cc-by-4.0
language:
  - en
tags:
  - research-paper
  - tool-calling
  - function-calling
  - agentic
  - mixture-of-experts
  - diagnosis
---

# Tool-Name Binding in Small Language Models Is Generative, Not Copy-Grounded: A Counterfactual Diagnosis and a Data-Side Fix

**Wei Ciao Wu** (independent researcher), with Claude (Anthropic) as AI research assistant.

📄 **[Read the paper (PDF)](binding-diagnosis.pdf)** · LaTeX source included · 13 pages

Preprint, 2026-07. Part of the **circus-0.2** research line (1B-parameter from-scratch bilingual MoE).

## TL;DR

Reliable tool calling requires **binding**: emitting a tool name byte-identical to an entry in the in-context schema. A 1B from-scratch MoE (circus-0.2) plateaus at ~59% held-out binding despite a 2.93B-token agentic mid-train, while emission is healthy. Counterfactual probes localize the failure completely:

- Given **gold reasoning**, the model copies the correct name 98% (125/128), fixing 25/25 previously failed cases; in free rollouts the wrong name always appears **first in the model's own chain-of-thought** (25/25) and is then copied faithfully.
- Failed names are semantic paraphrases (`check_email_validity` for `email_validate_regex`): the model **generates names from tool semantics and parametric memory instead of copying from the schema**. An architecture control rules out the NoPE/SWA attention stack (91% exact copy under worst-case distractors).
- A renamed-schema evaluation decomposes the base 59% into ~47% genuine copying + ~12% *convention collision* (invented canonical-style names that happen to match).
- **Data-side fix — name randomization**: consistently renaming a fraction of training tools to semantically unpredictable identifiers makes generation-from-semantics score zero, forcing the copy circuit. Renamed-schema binding rises 47% → 62% → 69% with dose (control 53%), then **saturates**: a 5× token scale-up holds at 69%. The ceiling is **emission-bound, not copy-bound** — valid-tool rate is capped by ~80% call-emission, while copy discipline *conditioned on emission* reaches ~86% (control 76%). Out-of-distribution zero-shot binding is the one dose-responsive axis (renamed-OOD 30% → 66% on a 100-prompt 4-domain follow-up), climbing sharply then plateauing.

## Citation

```bibtex
@misc{wu2026bindingdiagnosis,
  title  = {Tool-Name Binding in Small Language Models Is Generative, Not
            Copy-Grounded: A Counterfactual Diagnosis and a Data-Side Fix},
  author = {Wei Ciao Wu},
  year   = {2026},
  note   = {Preprint. https://huggingface.co/wcamon/circus-0.2-binding-diagnosis}
}
```

## Related

- Companion paper: [Memory as an Additive Side-Path (STEM niches, combine mechanisms, learned depth allocation)](https://huggingface.co/wcamon/circus-0.2-stem-architecture)