Upload README.md

README.md

@@ -9,17 +9,17 @@ Modern agent systems waste test-time compute because every tool call, retrieval,
 ## Core Architecture
 
 ```
-┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
-│  Impact Oracle  │────▶│  Credit Ledger  │────▶│ Resource Broker │
-│  (score action) │     │  (earn/spend)   │     │ (allow/deny)    │
-└─────────────────┘     └─────────────────┘     └─────────────────┘
-         │                                              │
-         └──────────────────┬───────────────────────────┘
-                            ▼
-                     ┌──────────────┐
-                     │  GRPO/RL Hook│
-                     │ (reward func) │
-                     └──────────────┘
++-----------------+     +-----------------+     +-----------------+
+|  Impact Oracle  |---->|  Credit Ledger  |---->| Resource Broker |
+|  (score action) |     |  (earn/spend)   |     | (allow/deny)    |
++-----------------+     +-----------------+     +-----------------+
+         |                                              |
+         +------------------+---------------------------+
+                            |
+                     +--------------+
+                     |  GRPO/RL Hook|
+                     | (reward func) |
+                     +--------------+
 ```
 
 ### 1. Impact Oracle (`oracle/`)
@@ -35,7 +35,7 @@ All scores are cost-adjusted: `reward = verified_impact - compute_cost * penalty
 
 - **Non-transferable** credits (laundering prevention)
 - **Exponential decay** on idle credits (hoarding prevention)
-- **Capability-scoped** rights (retrieval credits ≠ file-write credits)
+- **Capability-scoped** rights (retrieval credits != file-write credits)
 - **Full provenance** with oracle hash and reason
 
 ### 3. Resource Broker (`broker/`)
@@ -54,34 +54,36 @@ TRL-compatible reward function wrapping the Impact Oracle. Includes offline poli
 ## Installation
 
 ```bash
+git clone https://huggingface.co/narcolepticchicken/occ-stack
+cd occ-stack
 pip install -e .
+
 # For NLI evidence scoring:
-pip install sentence-transformers
+pip install -e ".[nli]"
+
 # For real LLM inference:
-pip install transformers datasets
-# For GRPO training:
-pip install trl accelerate
+pip install -e ".[train]"
 ```
 
 ## Quick Start
 
 ```bash
 # Run all benchmarks and ablations
-python -m benchmarks.eval_runner
+python eval_runner.py
 
 # Run individual benchmarks
 python -m benchmarks.benchmark_code
 python -m benchmarks.benchmark_retrieval_qa
 python -m benchmarks.benchmark_debate
 
-# Run with real NLI model (requires sentence-transformers)
-python -m benchmarks.benchmark_retrieval_qa_nli
-
 # Adversarial debate benchmark
 python -m benchmarks.benchmark_debate_adversarial
 
 # GRPO offline demonstrator
 python -m rl.grpo_train_demo
+
+# Real LLM code benchmark (requires GPU, ~30 min)
+# See jobs/run_real_llm_standalone_v3.py for a self-contained GPU job
 ```
 
 ## Benchmark Results
@@ -96,33 +98,30 @@ python -m rl.grpo_train_demo
 
 OCC tries cheap agents first, escalates only on failure. At iso-accuracy (0.780 pass@1), it reduces compute by 52%.
 
-### Code Compute Allocation (Real LLM - Qwen2.5-Coder-0.5B)
+### Code Compute Allocation (Real LLM — Qwen2.5-Coder-0.5B)
 
-GPU job running on T4. Script: `jobs/run_real_llm_standalone.py`
+**Status: Attempted but blocked.** The model loads successfully on GPU and generates code, but code extraction heuristics do not yet produce valid runnable Python when concatenated with HumanEval tests. This is a known issue documented in `reports/report.md`. Fixes needed: markdown stripping, AST validation, better body replacement.
 
-### Retrieval QA (with real NLI - cross-encoder/nli-deberta-v3-xsmall)
+### Retrieval QA (Simulated)
 
 | Strategy | Accuracy | ECE | Retrievals |
 |----------|----------|-----|------------|
 | Direct answer | 0.580 | 0.226 | 0 |
 | RAG baseline | 0.750 | 0.167 | 338 |
-| RAG + verifier | 0.790 | 0.151 | 344 |
+| RAG + verifier | **0.790** | 0.151 | 344 |
 | OCC baseline | 0.710 | 0.201 | 227 |
-| **OCC + real NLI** | *needs calibration* | — | 220 |
-
-Note: OCC + NLI shows stronger evidence quality but broker thresholds are too conservative on neutral evidence. Needs tuning for production use.
 
-### Multi-Agent Debate
+Note: OCC does not yet beat RAG+verifier on raw accuracy. OCC's value is compute savings + anti-gaming, not pure accuracy. See `reports/report.md` for analysis.
 
-With 50% adversarial agents:
+### Multi-Agent Debate (50% adversarial agents, v2)
 
-| Strategy | Accuracy | Quality/Compute |
-|----------|----------|-----------------|
-| Equal turns | 0.760 | 0.001275 |
-| Confidence-weighted | **0.560** | 0.000924 |
-| **OCC credit allocation** | **0.760** | **0.001196** |
+| Strategy | Accuracy | Quality/Compute | Bad Agent Containment |
+|----------|----------|-----------------|----------------------|
+| Equal turns | 0.760 | 0.001275 | 0% |
+| Confidence-weighted | **0.560** | 0.000924 | 0% |
+| **OCC credit allocation** | **0.760** | **0.001196** | **100%** |
 
-OCC contains adversarial agents while confidence-weighted voting collapses (bad agents exploit high confidence).
+Confidence-weighted voting **made things worse** — adversarial agents are overconfident, so their wrong answers got amplified. OCC denied turns to adversarial agents entirely after initial wrong proposals.
 
 ### Anti-Gaming
 
@@ -131,6 +130,7 @@ OCC contains adversarial agents while confidence-weighted voting collapses (bad
 | Spam low-value actions | 100% credit exhaustion | Credits = 0 |
 | Hidden-test gaming | 100% oracle detection | Immediate penalty |
 | Over-abstention | 70% oracle penalization | Wrong abstentions punished |
+| Collusion in debate | Credit-based filtering | Adversarial agents excluded |
 
 ## Project Structure
 
@@ -142,16 +142,17 @@ OCC contains adversarial agents while confidence-weighted voting collapses (bad
   /rl            - GRPO reward hooks and offline comparator
   /benchmarks    - Code, QA, and debate benchmarks
   /jobs          - GPU job scripts for real LLM inference
-  /reports       - Evaluation results (JSON)
+  /reports       - Evaluation results and technical report
   /configs       - Configuration files
 ```
 
-## Limitations & Next Steps
+## Limitations & Honest Assessment
 
-1. **Retrieval QA** needs better NLI calibration. Real NLI scores are strong but broker thresholds are too aggressive on neutral evidence.
-2. **All benchmarks use simulated agents** for tractability. Real LLM inference script (`jobs/run_real_llm_standalone.py`) is submitted as a GPU job.
+1. **All benchmarks use simulated agents** for tractability. Real LLM inference was attempted but code extraction heuristics need improvement.
+2. **Retrieval QA** needs better NLI calibration. Real NLI scores are strong but broker thresholds are too aggressive on neutral evidence.
 3. **GRPO training** hook is implemented but not trained on real data. Offline comparator validates the reward design.
 4. **Cost model** is token-count only. Real cost should include model size, latency, and API pricing.
+5. **OCC is a meta-controller, not a direct reasoning improvement.** It wins when there is clear agent/cost differentiation and loses when the baseline already optimizes well.
 
 ## Citation