docs/architecture/system-registry.md

System Registry & Wiring Architecture

Status: Active / Canonical Last Updated: 2025-12-06

This document serves as the Source of Truth for the architectural wiring of the agent framework. It defines the strict rules for decorators, protocol markers, and the tool registry to prevent regression and ensure correct system behavior.

---

1. Decorator Registry

The agent framework relies on a strict decorator stack to inject functionality into ChatClient implementations. The order of application is critical for correct behavior.

Standard Stack (Bottom-Up Order)

Order	Decorator	Purpose	Source	Critical Notes
1 (Inner)	@use_chat_middleware	Handles request/response middleware processing (e.g. logging, filtering).	agent_framework._middleware	Must be closest to the class.
2	@use_observability	Injects tracing and metrics (OpenTelemetry/logging).	agent_framework.observability	Wraps the middleware-enhanced client.
3 (Outer)	@use_function_invocation	CRITICAL: Intercepts FunctionCallContent in responses, executes the Python function, and recursively calls the LLM with the result.	agent_framework._tools	MUST NOT be used if __function_invoking_chat_client__ = True is set (see Markers).

Correct Usage Example

@use_function_invocation  # <--- 3. Handles tool execution loop
@use_observability        # <--- 2. Adds tracing
@use_chat_middleware      # <--- 1. Adds middleware support
class MyChatClient(BaseChatClient):
    ...

---

2. Protocol Markers

Special class attributes (dunder methods/variables) that control framework behavior.

Marker	Value	Purpose	Set By	Read By	Impact of Misuse
__function_invoking_chat_client__	bool	Signals that this client natively handles the tool execution loop internally.	ChatClient Class Body	@use_function_invocation	CRITICAL BUG: If set to True but the client doesn't execute tools, tool calls will be generated by the LLM but never executed. The agent will hang or hallucinate results.

Wiring Rules

• Default Clients (OpenAI/HuggingFace): Should generally NOT set this marker. Rely on @use_function_invocation to handle execution.
• Special Clients: Only set to True if you are implementing a custom loop that executes tools and feeds results back without the framework's help.

Setting Responsibility

• Default: Do not set __function_invoking_chat_client__ in the class body. The @use_function_invocation decorator sets it automatically after wrapping.
• Custom Loop: Only set to True if you have implemented a custom tool execution loop that does not rely on the framework's decorator.

---

3. Tool Inventory

3.1 AI Functions (Agent-Callable Tools)

These are the @ai_function decorated functions that agents can invoke. The framework executes these via @use_function_invocation.

Function Name	File Path	Description
search_pubmed	src/agents/tools.py:20	Searches PubMed for biomedical literature
search_clinical_trials	src/agents/tools.py:80	Searches ClinicalTrials.gov for clinical studies
search_preprints	src/agents/tools.py:120	Searches Europe PMC for preprints and papers
get_bibliography	src/agents/tools.py:160	Returns collected references for final report
search_web	src/agents/retrieval_agent.py:17	Searches web using DuckDuckGo
execute_python_code	src/agents/code_executor_agent.py	REMOVED in PR #130 (Modal deleted)

3.2 Tool Classes (Internal Wrappers)

These are internal implementation wrappers used by the AI Functions. They are NOT directly callable by agents.

Class	File Path	Used By
PubMedTool	src/tools/pubmed.py	search_pubmed
ClinicalTrialsTool	src/tools/clinicaltrials.py	search_clinical_trials
EuropePMCTool	src/tools/europepmc.py	search_preprints
OpenAlexTool	src/tools/openalex.py	OpenAlex search (used in SearchHandler)
WebSearchTool	src/tools/web_search.py	search_web (DuckDuckGo)
SearchHandler	src/tools/search_handler.py	Orchestrates parallel searches
RateLimiter	src/tools/rate_limiter.py	Rate limiting via limits library
BaseTool	src/tools/base.py	Abstract base class for tools
ModalCodeExecutor	src/tools/code_execution.py	REMOVED in PR #130

---

4. Client Implementation Guide

When adding a new LLM provider, follow this strict pattern:

A. The "Native Execution" Fallacy

Do not assume that because an API supports "function calling" (parsing JSON), the client supports "function execution" (running Python code).

• Function Calling: LLM -> JSON (Client responsibility)
• Function Execution: JSON -> Python Result -> LLM (Framework responsibility via @use_function_invocation)

B. Reference Implementation

from agent_framework import BaseChatClient
from agent_framework._tools import use_function_invocation
from agent_framework.observability import use_observability
from agent_framework._middleware import use_chat_middleware

# 1. Apply decorators in this EXACT order
@use_function_invocation
@use_observability
@use_chat_middleware
class NewProviderChatClient(BaseChatClient):
    
    # 2. DO NOT set this unless you know what you are doing
    # __function_invoking_chat_client__ = True  <-- DELETE THIS
    
    async def _inner_get_response(self, ...):
        # 3. Parse API response -> FunctionCallContent
        # 4. Return ChatResponse with contents=[FunctionCallContent(...)]
        pass
        
    async def _inner_get_streaming_response(self, ...):
        # 5. Yield FunctionCallContent when tool calls are detected
        pass

---

5. Known Issues & Gotchas

• P1 Bug - Premature Marker Setting (FIXED): The HuggingFaceChatClient previously set __function_invoking_chat_client__ = True in the class body, which caused @use_function_invocation to skip wrapping. Resolution: Marker removed; decorator now sets it correctly.
• HuggingFace Provider Routing: Large models (70B+) may be routed to third-party inference providers (Novita, Hyperbolic) instead of native HF infrastructure. See CLAUDE.md for current model recommendations.
• Model Hallucination: If tool execution fails (due to incorrect wiring), models like Qwen2.5-7B will often hallucinate fake tool results as text. Always verify AgentRunResponse contains actual FunctionResultContent.

---

6. Verification Checklist

When adding or modifying a ChatClient:

• Decorators applied in correct order: @use_function_invocation → @use_observability → @use_chat_middleware
• __function_invoking_chat_client__ is NOT set in class body (unless implementing custom execution loop)
• Verify @use_function_invocation decorator actually wraps methods (check __wrapped__ attribute at runtime)
• Tool calls parsed into FunctionCallContent objects
• Streaming yields FunctionCallContent at end of stream
• Run make check to verify all tests pass