Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use

Excited to share our new work on the knowing–doing gap in LLM tool use.

Most prior work in LLM adaptive tool use treats “tool necessity” as fixed and model-agnostic. But models have different capabilities. What GPT-5 can solve without a tool may require tools for another model.
So we introduce model-adaptive tool necessity, grounded in each model’s empirical capability.

Across arithmetic + factual QA tasks, we compare:
✅ When models actually need tools vs.🔧 When models actually use tools
We find major mismatches — up to 54% disagreement between necessity and behavior.

Models frequently:

• use tools unnecessarily
• skip tools when needed

To understand why, we model tool use as a two-stage process:

• 🧠 Cognition: recognizing a tool is needed
• ⚡ Execution: actually invoking the tool
  This distinction turns out to matter a lot.

By probing hidden states, we found:

• Tool necessity signals are often decodable
• Tool-call execution signals are also decodable
• But at late tokens in late layers, the two signals become nearly orthogonal

Meaning:
The representation of necessity and action are decoupled.

By tracing the trajectory of samples in the two-stage process, we further discover that the majority of mismatch is concentrated in the cognition-to-action transition, not in cognition itself. This shows models often internally know whether they need a tool, but fail to translate that cognition into the matching tool-call or direct-answer action.
We call this:
👉 the knowing–doing gap in LLM tool use.

Takeaway: improving tool-use reliability requires not only better recognition of when tools are needed, but also better translation of that recognition into action.

Curious to hear your thoughts:

• Have you observed similar knowing–doing gaps in agentic systems?
• What mechanisms might better align internal recognition with external action?