work/Convergence_Engine/integration/implementation_checklist.md

Convergence Engine: Optimization Implementation Checklist

Quick Start Guide - Do This First

---

✅ Pre-Implementation Checklist

• Backup current codebase (git commit)
• Install GPU profiling tools: pip install torch-tb-profiler
• Verify CUDA availability: python -c "import torch; print(torch.cuda.is_available())"
• Document current performance baseline (see Profiling section)
• Read through Part 1 of convergence_optimization_analysis.md

---

🚀 Phase 1: Batched Inference (HIGHEST PRIORITY - 1-2 hours)

Step 1.1: Create vectorized neural network

• Create new file: neural_organism_vectorized.py
• Copy VectorizedOrganismNetwork class from optimization_code_examples.md
• Test import: python -c "from neural_organism_vectorized import VectorizedOrganismNetwork"

Step 1.2: Update simulator loop

• In reality_simulator/simulator.py or main loop:
  • Find: for organism in organisms: organism.action = model(organism.state)
  • Replace with batched version from optimization_code_examples.md
  • Collect all organism states into single tensor
  • Call population_manager.get_population_actions(organism_states)

Step 1.3: Benchmark

import time
import numpy as np

# Record time before change
start = time.time()
for cycle in range(10):
    actions = population_manager.get_population_actions(organism_states)
    # ... rest of cycle
duration_before = (time.time() - start) / 10

# Compare to existing implementation
print(f"Per-cycle time: {duration_before * 1000:.1f} ms")
# Expected: 20-50 ms (vs 700 ms+ before)

Step 1.4: Verify correctness

• Actions from batched version match serial version (within numerical precision)
• Organisms behave identically after refactoring
• No NaN or inf values in network outputs

Expected Result: 10-50× speedup in organism decision-making

---

🚀 Phase 2: GPU Clustering (1-2 hours)

Step 2.1: Install RAPIDS cuML

# Choose one:
pip install cuml  # Recommended
# OR
conda install -c rapidsai -c conda-forge cuml

Step 2.2: Test GPU availability

python -c "
from cuml.cluster import HDBSCAN as cuHDBSCAN
import cupy as cp
print(f'CUDA devices: {cp.cuda.runtime.getDeviceCount()}')
"

• cuML installed successfully
• cupy detects GPU

Step 2.3: Create GPU clustering module

• Create new file: ml_analysis_gpu.py
• Copy GPUClusterer class from optimization_code_examples.md
• Test: python -c "from ml_analysis_gpu import GPUClusterer; c = GPUClusterer(use_gpu=True)"

Step 2.4: Update ML analysis pipeline

• In reality_simulator/ml_analysis.py:
  • Add: from ml_analysis_gpu import MLAnalysisPipeline
  • Replace existing clustering code with:

    pipeline = MLAnalysisPipeline(use_gpu=True)
    result = pipeline.analyze_cycle(cycle, organism_embeddings)
    clusters = result['clusters']
    

Step 2.5: Benchmark

# Profile clustering speed
import time
pipeline = MLAnalysisPipeline(use_gpu=True)
embeddings = ... # 4000 organism embeddings

start = time.time()
result = pipeline.analyze_cycle(0, embeddings)
duration_ms = (time.time() - start) * 1000
print(f"Clustering: {duration_ms:.1f} ms")
# Expected: 30-100 ms (vs 500-2000 ms)

Expected Result: 15-60× speedup in clustering

---

🚀 Phase 3: Prioritized Experience Replay (2-3 hours)

Step 3.1: Install torchrl

pip install torchrl

Step 3.2: Create prioritized DQN agent

• Create new file: dqn_agent_prioritized.py
• Copy DQNAgentPrioritized class from optimization_code_examples.md
• Verify import: python -c "from dqn_agent_prioritized import DQNAgentPrioritized"

Step 3.3: Update training loop

• In your DQN training code:
  • Replace experience storage: agent.store_experience(state, action, reward, next_state, done)
  • Replace sampling: loss = agent.train_step()
  • Remove manual priority sampling logic (now automatic)

Step 3.4: Benchmark convergence

• Train for fixed number of episodes with old and new replay buffer
• Compare episodes-to-convergence
• Expected: 20-40% faster convergence

Expected Result: 3-8× faster sampling, 20-40% better learning efficiency

---

🟡 Phase 4: torch.compile() (EXPERIMENTAL - 2-3 hours)

Step 4.1: Test compilation compatibility

• In test script:

  from neural_organism_vectorized import VectorizedOrganismNetwork
  import torch
  
  model = VectorizedOrganismNetwork()
  test_input = torch.randn(1400, 18)
  
  # Try compiling
  try:
      compiled_model = torch.compile(model, mode='default')
      output = compiled_model(test_input)
      print("✓ Compilation successful")
  except Exception as e:
      print(f"✗ Compilation failed: {e}")
  

• Fix any graph breaks (see troubleshooting in main analysis)

• Document issues found

Step 4.2: Add to network class

• Update VectorizedOrganismNetwork.__init__():

  def __init__(self, ..., use_compile=False):
      # ... existing code ...
      if use_compile:
          self.forward = torch.compile(self.forward, mode='default')
  

Step 4.3: Add config flag

• In config.json, add: "use_torch_compile": false (keep disabled for now)
• Add conditional: if config['use_torch_compile']: model = torch.compile(model)

Step 4.4: Benchmark carefully

• Profile with PyTorch profiler (see profiling_code_examples.md)
• Verify: speedup > recompilation overhead
• Expected: 1.2-2× speedup (if beneficial at all)

Expected Result: Small incremental speedup (~10-20%)

---

🟢 Phase 5: Automatic Mixed Precision (OPTIONAL - 1 hour)

Step 5.1: Add AMP to forward pass

• Update network inference:

  from torch.cuda.amp import autocast
  
  with torch.no_grad():
      with autocast(device_type='cuda', dtype=torch.float16):
          actions, language = model(organism_states)
  

Step 5.2: Test numerical stability

• Verify actions don't contain NaN/inf
• Check if agent behavior changes significantly
• If yes: disable AMP (not compatible with your network)

Step 5.3: Benchmark

• Profile memory usage before/after
• Expected: 10-30% faster, 30-50% less memory

Expected Result: Small speedup + memory savings

---

📊 Testing & Validation Checklist

For each optimization:

Correctness

• Numeric outputs match original within tolerance (1e-5)
• Behavior is identical (same random seed)
• No NaN/inf values
• No GPU out-of-memory errors

Performance

• Measured speedup matches expected range
• Improvement consistent across multiple runs
• No memory leaks (GPU memory stable)
• CPU usage reasonable

Production Readiness

• Code handles edge cases (empty batches, single organism, etc.)
• Error messages are clear
• Configuration can be toggled on/off
• Fallback to CPU versions if GPU unavailable

---

🔧 Configuration Quick Reference

Minimal optimization (conservative)

{
  "optimization": {
    "use_batched_inference": true,
    "use_gpu_clustering": false,
    "use_prioritized_replay": false,
    "use_mixed_precision": false,
    "use_torch_compile": false
  }
}

Moderate optimization (recommended)

{
  "optimization": {
    "use_batched_inference": true,
    "use_gpu_clustering": true,
    "use_prioritized_replay": true,
    "use_mixed_precision": false,
    "use_torch_compile": false
  }
}

Aggressive optimization (experimental)

{
  "optimization": {
    "use_batched_inference": true,
    "use_gpu_clustering": true,
    "use_prioritized_replay": true,
    "use_mixed_precision": true,
    "use_torch_compile": true
  }
}

---

🐛 Quick Troubleshooting

Issue: "CUDA out of memory"

• Reduce batch size (especially for clustering)
• Check GPU memory with nvidia-smi
• Fall back to CPU version

Issue: Actions don't match after batching

• Check shapes: states should be (batch_size, 18)
• Verify padding logic if organisms have variable state sizes
• Compare outputs with original serial code

Issue: Clustering extremely slow

• Verify cuML installed correctly
• Check use_gpu=True in GPUClusterer
• Fall back to CPU HDBSCAN

Issue: torch.compile fails

• Keep it disabled (use_torch_compile: false)
• torch.compile is optional, not critical
• See troubleshooting in main analysis for details

---

📈 Performance Targets

Phase	Component	Target Speedup	Actual Goal
1	Batched inference	10-50×	20×
2	GPU clustering	15-60×	30×
3	Prioritized replay	3-8×	5×
4	torch.compile	1.2-2×	Skip if <1.5×
5	AMP	1.1-1.5×	Nice-to-have
Total	All combined	100×	10-30×

---

📝 Implementation Timeline

Week 1

• Day 1-2: Phase 1 (batched inference)
• Day 3-4: Phase 2 (GPU clustering)
• Day 5: Testing & validation

Week 2

• Day 1-2: Phase 3 (prioritized replay)
• Day 3: Benchmarking & profiling
• Day 4-5: Documentation & refinement

Week 3 (Optional)

• Day 1-2: Phase 4 (torch.compile, experimental)
• Day 3: Phase 5 (AMP, if needed)
• Day 4-5: Production hardening

---

📚 Reference Files

1. convergence_optimization_analysis.md - Main analysis (9 parts)
2. optimization_code_examples.md - Copy-paste ready code
3. THIS FILE - Quick checklist and implementation guide

---

✅ Final Validation

Before considering optimization complete:

• All tests pass
• Performance benchmarks documented
• Configuration properly integrated
• Fallback mechanisms work
• Code is commented and version-controlled
• Teammate/collaborator review completed

---

🎯 Next Action

RIGHT NOW:

1. Read Part 1 of convergence_optimization_analysis.md
2. Run the profiling code from optimization_code_examples.md
3. Start Phase 1 (batched inference) today

THIS WEEK: Complete Phases 1-3

LATER: Evaluate torch.compile() and AMP based on actual measurements

---

Questions? Refer to the main analysis document. It covers all implementation details, troubleshooting, and advanced patterns.