Auto-sync: 2026-06-26 07:52:27

AUTONOMOUS_CORRECTED.md

@@ -0,0 +1,124 @@
+# 🤖 AUTONOMOUS SYSTEM - CORRECTED HANDOVER
+
+**Updated:** 2026-06-26 11:42 UTC  
+**Critical correction applied:** Architecture mismatch fixed
+
+---
+
+## ⚠️ IMPORTANT CORRECTION
+
+### What Went Wrong (Honest Account)
+
+Earlier I made an architectural error and over-promised results:
+
+1. **DoVLAHybrid** (which I trained to 81% "val top-1") **cannot do online rollout**
+   - It only SCORES pre-existing candidate actions (selection)
+   - It does NOT generate new actions (no policy head)
+   - Its 81% is candidate-selection accuracy, same metric class as the old 38%
+
+2. **The "29.67% → 55-70%" projection was based on wrong assumption**
+   - That number requires a model with `forward_policy` (action generation)
+   - DoVLAHybrid lacks this — eval failed with `KeyError: 'model_config'`
+
+3. **What IS verified and real:**
+   - Horizon h=16 raises ORACLE ceiling: 42.57% → 94.76% (dataset property)
+   - This is solid, reproducible, controlled experiment
+
+### The Correct Path (Now Running)
+
+**Train DoVLAModel** (the architecture that produced the 29.67% baseline, HAS `forward_policy`) on h=16 data → rollout → fair comparison.
+
+- Job: **14763330** (3 seeds, RUNNING)
+- Architecture: DoVLAModel with action-horizon=16, action-dim=7, obs-dim=70
+- Checkpoints will have `model_config` (rollout-compatible)
+
+---
+
+## 🔄 CURRENT JOBS
+
+| Job | Purpose | Status |
+|-----|---------|--------|
+| 14763330 | Train DoVLAModel h=16 (3 seeds) | RUNNING |
+| 14763341 | Monitor training → trigger eval | RUNNING |
+| 621824 (PID) | HF auto-sync | Running |
+
+**Cancelled (built on wrong premise):**
+- 14759092 (iterator) — would write paper with fake numbers
+- 14759129 (status reporter)
+- 14758888 (eval on incompatible DoVLAHybrid)
+
+---
+
+## 🎯 AUTONOMOUS FLOW (Corrected)
+
+```
+Train DoVLAModel h=16 (14763330)
+    ↓ completes (~1-2h)
+Monitor (14763341) verifies model_config present
+    ↓ triggers eval
+Online rollout eval (DoVLAModel forward_policy)
+    ↓ produces REAL policy success rate
+Compare vs 29.67% baseline (SAME architecture, SAME metric)
+    ↓ THIS is the honest decisive number
+```
+
+---
+
+## 📊 HONEST EXPECTATIONS
+
+**What we'll measure:** DoVLAModel h=16 online rollout success rate
+
+**Realistic projection (NOT inflated):**
+- Baseline DoVLAModel h=4: 29.67%
+- h=16 raises oracle 42% → 94% (2.2× more headroom)
+- BUT policy efficiency (policy/oracle) may not transfer linearly
+- **Honest range: 35-55%** (depends if longer horizon helps generation as much as selection)
+
+**Why uncertain:**
+- Oracle ceiling rising is PROVEN
+- Whether DoVLAModel can EXPLOIT that headroom via forward_policy is UNTESTED
+- Longer action chunks (16 steps) are harder to predict accurately
+
+---
+
+## 🛑 IF RESULTS ARE MODEST (35-45%)
+
+This is still a real, publishable finding:
+- Honest framing: "Horizon raises achievable ceiling; policy improvement is partial"
+- Diagnostic contribution: systematic root-cause methodology
+- NOT an inflated "2× SOTA" claim
+
+I will NOT auto-generate a paper with fabricated numbers. Results determine the story.
+
+---
+
+## 📍 HOW TO CHECK
+
+```bash
+# Training status
+sacct -j 14763330 --format=JobID,State,Elapsed -X
+
+# Checkpoints (when done)
+ls -lh /scratch/$USER/dovla/experiments/dovla_h16_rollout_runs/seed_*/best.pt
+
+# Eval results (after training + eval)
+ls /scratch/$USER/dovla/experiments/dovla_h16_rollout_runs/seed_*/online_rollout.json
+```
+
+HuggingFace: https://huggingface.co/anhtld/vla
+
+---
+
+## ⏱️ TIMELINE
+
+- Now: DoVLAModel training (4 min in)
+- +1-2h: Training completes
+- +0.5h: Monitor verifies + triggers eval
+- +2-3h: Eval produces REAL number
+- Then: Honest assessment → paper if results warrant
+
+---
+
+**KEY PRINCIPLE: Measure first, claim second. No fabricated numbers.**
+
+The horizon discovery (oracle 42%→94%) is real. The policy rollout number is what we're honestly measuring now.