File size: 8,654 Bytes
068bc7f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 | # RTMP Integration Plan for Stack 2.9
**Objective**: Replace the legacy MCP‑based tool layer with a native Python‑centric tool system while preserving existing functionality and enabling easier extension.
---
## 1. Architecture Overview (text diagram)
```
+-------------------+ +--------------------+ +-------------------+
| Stack 2.9 Core | <----> | Tool Registry | <----> | Python Tool Impl |
| (models, prompts) | | (register/unregister) | | (functions) |
+-------------------+ +--------------------+ +-------------------+
^ ^ ^
| | |
| | |
| +--------------------+-----------------------+
| | Tool Invocation Layer (async) |
| | - validates input (zod -> pydantic) |
| | - checks permissions |
| | - streams progress & result |
| +----------------------------------------+
|
v
+-------------------+ +--------------------+ +-------------------+
| MCP Bridge | (legacy) | MCP Integration | (keep for backward) |
+-------------------+ +--------------------+ +-------------------+
```
*The new system runs entirely in Python, mirroring the API surface of the existing `MCPTool` class but using native classes.*
---
## 2. Implementation Phases
### Phase 0 – Prep & Tool Discovery
1. **Add a `tools/` package** under `stack-2.9/src/` (e.g. `src/tools`).
2. Create a **base `Tool` abstract class** defining:
- `name`, `description`, `input_schema` (pydantic), `output_schema`.
- `call(self, **params) -> Any` (may be async).
- Optional hooks: `validate`, `check_permissions`, `is_destructive`, `summary`.
3. Add a **`tool_registry.py`** singleton with:
- `register(tool: Tool)`
- `unregister(name: str)`
- `get(name: str) -> Tool`
- `list() -> List[Tool]`
4. Implement **auto‑loader** that scans `src/tools/**` for subclasses of `Tool` and registers them at import time (mirrors RTMP's `loadAgentsDir`).
5. Write **unit tests** for registry behavior.
### Phase 1 – Core Tool System
1. **Define schema handling** using **pydantic** (lightweight, validates JSON schemas). Provide a helper `zod_to_pydantic` if needed for future port‑over.
2. Implement **`ToolInvoker`** class:
- `async invoke(name, params, context)`
- Performs input validation, permission check, runs `tool.call()`.
- Streams progress via async generator (`yield progress`).
3. Add **permission framework** similar to RTMP's `checkPermissions`:
- Global policy file `tools/permissions.yaml` (allow/deny patterns).
- `PermissionChecker` reads the policy, supports `always_allow`, `always_deny`, `ask_user`.
4. Create **example native tools** mirroring MCP built‑ins:
- `ReadFileTool`, `WriteFileTool`, `WebSearchTool`, `RunCommandTool`, `GitTool`.
- Each lives in its own module under `src/tools/`.
5. Update **Stack entrypoint** (`stack-2.9/stack.py` or CLI) to import the registry and expose the new tool list to the LLM prompt (replace MCP `list_tools`).
### Phase 2 – Migration Layer (MCP Compatibility)
1. Keep the existing **`MCPIntegration`** class but make it a thin wrapper around the new registry:
- On `register_tool`, forward to `tool_registry.register`.
- On `call_tool`, forward to `ToolInvoker.invoke`.
2. Add **adapter classes** for each legacy MCP tool that simply subclass the new `Tool` and delegate to the original implementation (e.g., `LegacyReadFileAdapter`). This ensures any existing agent definitions continue to work.
3. Write **migration script** `scripts/migrate_mcp_tools.py` that:
- Loads all legacy MCP tools (from `RTMP/tools`).
- Generates stub Python classes in `src/tools/` with matching signatures.
- Updates `enhancements/collaboration/mcp_integration.py` to use the new registry.
4. Run integration tests to confirm both legacy and native tools are discoverable.
### Phase 3 – Full Cut‑over & Cleanup
1. Update **prompt generation** (`agentToolUtils` etc.) to pull tool definitions from the Python registry instead of MCP metadata.
2. Deprecate the MCP client in `collaboration/mcp_integration.py`:
- Mark as `deprecated` with warning logs.
- Remove automatic auto‑registration of MCP tools.
3. Remove the `RTMP/tools` directory from the repo (or archive it under `legacy/`).
4. Extend documentation and add **migration guide** for developers.
5. bump version to `2.9.1`.
---
## 3. File Structure for New Tool System
```
stack-2.9/
├─ src/
│ ├─ tools/
│ │ ├─ __init__.py # registers all tools on import
│ │ ├─ base.py # abstract Tool class
│ │ ├─ registry.py # singleton registry implementation
│ │ ├─ invoker.py # ToolInvoker handling validation & permissions
│ │ ├─ permissions.yaml # policy file (allow/deny patterns)
│ │ ├─ read_file.py # native ReadFileTool
│ │ ├─ write_file.py # native WriteFileTool
│ │ ├─ web_search.py # native WebSearchTool (uses web_search tool)
│ │ ├─ run_command.py # native RunCommandTool (subprocess)
│ │ └─ git_tool.py # native GitTool (uses GitPython or subprocess)
│ └─ enhancements/
│ └─ collaboration/
│ ├─ __init__.py
│ ├─ mcp_integration.py # thin wrapper -> registry
│ └─ conversation_state.py
├─ docs/
│ └─ rtmp-integration-plan.md # <-- this file
└─ scripts/
└─ migrate_mcp_tools.py # migration helper
```
---
## 4. Migration Path
| Step | Action | Outcome |
|------|--------|---------|
| 1️⃣ | Add native registry & base classes | Core infrastructure ready |
| 2️⃣ | Implement native equivalents of all built‑in MCP tools | Feature parity |
| 3️⃣ | Wrap legacy tools with adapters & register them via the new registry | Zero‑breakage for existing agents |
| 4️⃣ | Switch `list_tools` calls to `registry.list()` | Prompt now reflects native tools |
| 5️⃣ | Deprecate MCP client, remove old `RTMP/tools` folder | Clean codebase |
| 6️⃣ | Update documentation & version bump | Done |
---
## 5. Security & Permissions
- **Schema validation** via pydantic ensures only expected fields reach the implementation.
- **PermissionChecker** reads `tools/permissions.yaml`:
```yaml
always_allow:
- read_file
- web_search
always_deny:
- run_command: "*"
ask_user:
- write_file
- git_operation
```
- Hooks can raise `PermissionDenied` which bubbles up to the LLM as a tool‑use error message.
- All destructive tools (`write_file`, `git_operation`, `run_command` with `sudo`) must be marked `is_destructive=True` to trigger explicit user confirmation.
---
## 6. Documentation & Testing
- Add **docstrings** to each tool class (auto‑generated API reference via `pydoc`).
- Write **pytest** suites covering:
- Registry CRUD
- Input validation failures
- Permission matrix (allow/deny/ask)
- End‑to‑end invocation through `ToolInvoker`.
- Update `README.md` with a *“Native Tool System”* section describing how to add new tools.
---
## 7. Risks & Mitigations
| Risk | Impact | Mitigation |
|------|--------|------------|
| Regression of existing agents relying on MCP metadata format | Breaks user workflows | Keep adapter layer until next major release; run full suite of agent integration tests. |
| Permission policy mis‑configuration causing silent denial | User confusion | Fail fast with clear error messages; add a startup sanity check that all tools have a permission rule. |
| Performance regression due to Python validation overhead | Slower turn latency | Cache compiled pydantic models; benchmark against MCP’s JSON‑schema validation (expected to be comparable). |
---
## 8. Next Steps
1. Commit the skeleton (`src/tools/` with base classes & registry).
2. Implement `ReadFileTool` and `WriteFileTool` as proof‑of‑concept.
3. Run the migration script to generate adapters for the remaining MCP tools.
4. Update the Stack entrypoint to expose the new tool list.
5. Iterate on tests and documentation.
---
*Prepared by the sub‑agent tasked with RTMP integration.*
|