IMPLEMENTATION_PLAN.md

Implementation Plan - Vast.ai SSH Tunnel

Primary Objective

Replace the complex MCP/Cloudflare auto-deploy system with simple SSH tunnel for remote GPU access.

Current Status

• ✅ Requirements documented
• ✅ Design documented
• ✅ Tasks broken down
• ✅ Bugs documented
• ⏳ Ready to implement

Implementation Order

PHASE 1: Fix Critical Bugs (1-2 hours)

These bugs block testing and need to be fixed first:

1. Fix deliberate float parsing bug

   • Location: agent_compiler.py council deliberation code generation
   • Issue: Float values being concatenated without separators
   • Impact: Blocks council deliberation feature

2. Implement to command

   • Location: agent_compiler.py command parser
   • Issue: Command documented but not implemented
   • Impact: Can't test individual council members

PHASE 2: SSH Infrastructure (2-3 hours)

Core remote execution system:

1. Add paramiko to requirements.txt
2. Create RemoteGPUExecutor class (new file: remote_gpu.py)
   • SSH connection management
   • Remote Python REPL
   • Code execution and output capture
3. Test SSH connection with real Vast.ai instance

PHASE 3: Simplify vast rent (3-4 hours)

CRITICAL: This is quine code - must be extremely careful

1. Remove complexity from vast rent:

   • Delete agent upload code
   • Delete MCP server setup
   • Delete Cloudflare tunnel code
   • Keep only: rent instance + SSH setup

2. Add SSH tunnel setup:

   • Wait for "running" status
   • Get SSH details
   • Create RemoteGPUExecutor
   • Store in TUI state

3. Test: Rent GPU, verify SSH connection works

PHASE 4: Remote Model Loading (2-3 hours)

1. Modify plug command:

   • Check if remote executor exists
   • If yes: load on remote GPU
   • If no: load locally

2. Implement remote loading:

   • Generate model loading code
   • Execute via SSH
   • Verify success

3. Test: Plug BGE-small on remote GPU

PHASE 5: Remote Inference (2-3 hours)

1. Modify infer/embed commands:

   • Check if model is remote
   • Execute remotely if yes
   • Return results to local TUI

2. Test: Run embeddings on remote GPU

PHASE 6: Polish & Documentation (1-2 hours)

1. Add status display: Show local vs remote GPU
2. Add vast stop command: Destroy instance, close SSH
3. Update documentation: relay_center.cmd, FelixBag guides

Total Estimated Time: 11-17 hours

Risk Mitigation

Quine Structure Risk

• Risk: Breaking quine structure when editing agent_compiler.py
• Mitigation:
  • Make small, incremental changes
  • Test after each change
  • Keep old code commented out initially
  • Verify quine hash after compilation

SSH Compatibility Risk

• Risk: paramiko might not work on Windows
• Mitigation:
  • Test early (Phase 2)
  • Have fallback plan (use subprocess + ssh.exe)

Remote Execution Risk

• Risk: Code serialization/deserialization might fail
• Mitigation:
  • Start with simple examples
  • Test incrementally
  • Add robust error handling

Success Criteria

1. ✅ User can rent GPU with vast rent 1
2. ✅ SSH connection establishes automatically
3. ✅ User can plug model on remote GPU
4. ✅ User can run embed using remote GPU
5. ✅ Results return to local TUI correctly
6. ✅ Quine structure preserved (hash matches)
7. ✅ Works on Windows

Next Steps

1. Start with Phase 1: Fix critical bugs
2. Test fixes with Beholder Council
3. Move to Phase 2: SSH infrastructure
4. Continue through phases sequentially

Notes

• Primary focus: SSH tunnel implementation
• Secondary: Fix bugs that block testing
• Tertiary: Polish and additional features
• Always test after each change
• Document any new bugs discovered