# Implementation Plan - Vast.ai SSH Tunnel ## Primary Objective Replace the complex MCP/Cloudflare auto-deploy system with simple SSH tunnel for remote GPU access. ## Current Status - ✅ Requirements documented - ✅ Design documented - ✅ Tasks broken down - ✅ Bugs documented - ⏳ Ready to implement ## Implementation Order ### PHASE 1: Fix Critical Bugs (1-2 hours) These bugs block testing and need to be fixed first: 1. **Fix `deliberate` float parsing bug** - Location: agent_compiler.py council deliberation code generation - Issue: Float values being concatenated without separators - Impact: Blocks council deliberation feature 2. **Implement `to` command** - Location: agent_compiler.py command parser - Issue: Command documented but not implemented - Impact: Can't test individual council members ### PHASE 2: SSH Infrastructure (2-3 hours) Core remote execution system: 1. **Add paramiko to requirements.txt** 2. **Create RemoteGPUExecutor class** (new file: remote_gpu.py) - SSH connection management - Remote Python REPL - Code execution and output capture 3. **Test SSH connection** with real Vast.ai instance ### PHASE 3: Simplify vast rent (3-4 hours) **CRITICAL**: This is quine code - must be extremely careful 1. **Remove complexity from vast rent**: - Delete agent upload code - Delete MCP server setup - Delete Cloudflare tunnel code - Keep only: rent instance + SSH setup 2. **Add SSH tunnel setup**: - Wait for "running" status - Get SSH details - Create RemoteGPUExecutor - Store in TUI state 3. **Test**: Rent GPU, verify SSH connection works ### PHASE 4: Remote Model Loading (2-3 hours) 1. **Modify plug command**: - Check if remote executor exists - If yes: load on remote GPU - If no: load locally 2. **Implement remote loading**: - Generate model loading code - Execute via SSH - Verify success 3. **Test**: Plug BGE-small on remote GPU ### PHASE 5: Remote Inference (2-3 hours) 1. **Modify infer/embed commands**: - Check if model is remote - Execute remotely if yes - Return results to local TUI 2. **Test**: Run embeddings on remote GPU ### PHASE 6: Polish & Documentation (1-2 hours) 1. **Add status display**: Show local vs remote GPU 2. **Add vast stop command**: Destroy instance, close SSH 3. **Update documentation**: relay_center.cmd, FelixBag guides ## Total Estimated Time: 11-17 hours ## Risk Mitigation ### Quine Structure Risk - **Risk**: Breaking quine structure when editing agent_compiler.py - **Mitigation**: - Make small, incremental changes - Test after each change - Keep old code commented out initially - Verify quine hash after compilation ### SSH Compatibility Risk - **Risk**: paramiko might not work on Windows - **Mitigation**: - Test early (Phase 2) - Have fallback plan (use subprocess + ssh.exe) ### Remote Execution Risk - **Risk**: Code serialization/deserialization might fail - **Mitigation**: - Start with simple examples - Test incrementally - Add robust error handling ## Success Criteria 1. ✅ User can rent GPU with `vast rent 1` 2. ✅ SSH connection establishes automatically 3. ✅ User can `plug` model on remote GPU 4. ✅ User can run `embed` using remote GPU 5. ✅ Results return to local TUI correctly 6. ✅ Quine structure preserved (hash matches) 7. ✅ Works on Windows ## Next Steps 1. Start with Phase 1: Fix critical bugs 2. Test fixes with Beholder Council 3. Move to Phase 2: SSH infrastructure 4. Continue through phases sequentially ## Notes - Primary focus: SSH tunnel implementation - Secondary: Fix bugs that block testing - Tertiary: Polish and additional features - Always test after each change - Document any new bugs discovered