docs/LOCAL_TESTING.md

AEPO Local Testing Guide — qwen2.5-coder:32b via Ollama

This guide walks you through testing AEPO end-to-end on your local machine using qwen2.5-coder:32b served by Ollama as the agent's LLM backend.

---

Prerequisites

Tool	Version	Install
Python	3.10	Already installed
Ollama	latest	https://ollama.com/download
qwen2.5-coder:32b model	—	ollama pull qwen2.5-coder:32b

---

Step 1 — Pull the model

Open a terminal and run:

ollama pull qwen2.5-coder:32b

Wait for the download to finish. Verify it is available:

ollama list
# Should show: qwen2.5-coder:32b   ...   xx GB

---

Step 2 — Start the Ollama server

Ollama needs to be running before the inference script talks to it.

ollama serve

Leave this terminal open. Ollama listens on http://localhost:11434 by default.

---

Step 3 — Install project dependencies

In a new terminal, from the project root:

cd C:\Users\Umesh Maurya\projects\autonomous-enterprise-payment-orchestrator
pip install -r requirements.txt

---

Step 4 — Start the AEPO FastAPI server

The inference script calls the environment via HTTP, so the server must be up first.

uvicorn server.app:app --host 0.0.0.0 --port 7860

Verify it is healthy:

# In another terminal
curl http://localhost:7860/
# Expected: {"status":"healthy","message":"AEPO is live..."}

# NEW: Contract declaration (Fix 9.4 — verifies 4-tuple bridge is live)
curl http://localhost:7860/contract
# Expected: {"step_tuple":"4-tuple","openenv_compliant":true,...}

Leave this terminal open.

---

Step 5 — Run inference with qwen2.5-coder:32b

Open a third terminal from the project root. Set environment variables to point inference.py at the local server and the Ollama OpenAI-compatible endpoint:

# PowerShell
$env:SPACE_URL        = "http://localhost:7860"
$env:API_BASE_URL     = "http://localhost:11434/v1"
$env:MODEL_NAME       = "qwen2.5-coder:32b"
$env:HF_TOKEN         = "ollama"          # Ollama ignores the token; any non-empty string works
$env:DRY_RUN          = "false"           # Use the real LLM
$env:AGENT_MODE       = "llm"             # Default: use the LLM backend (llm|qtable|heuristic)

python inference.py

# bash / Git Bash equivalent
SPACE_URL="http://localhost:7860" \
API_BASE_URL="http://localhost:11434/v1" \
MODEL_NAME="qwen2.5-coder:32b" \
HF_TOKEN="ollama" \
DRY_RUN="false" \
AGENT_MODE="llm" \
python inference.py

Expected stdout output

[START] task=easy env=aepo model=qwen2.5-coder:32b
[STEP] step=1 action={"risk_decision":0,"crypto_verify":1,"infra_routing":0,"db_retry_policy":0,"settlement_policy":0,"app_priority":2} reward=0.80 done=false error=null
[STEP] step=2 action={"risk_decision":0,"crypto_verify":1,"infra_routing":0,"db_retry_policy":0,"settlement_policy":0,"app_priority":2} reward=0.80 done=false error=null
...
[END] success=true steps=100 score=0.78 rewards=0.80,0.80,...

[START] task=medium env=aepo model=qwen2.5-coder:32b
...
[END] success=false steps=100 score=0.42 rewards=...

[START] task=hard env=aepo model=qwen2.5-coder:32b
...
[END] success=true steps=100 score=0.51 rewards=...

Note on action format: The action JSON now contains 6 integer fields (Phase 10 expansion). The three new fields are db_retry_policy, settlement_policy, and app_priority.

Rich terminal dashboard (stderr)

When rich is installed (pip install rich), each step also renders a colour-coded live status line to stderr showing system health signals and the action taken:

task=easy step= 42 phase=normal   LAG ████████░░░░░░░░░░░░░░░░  3800  POOL ███████████████░░░░░░░░░  61%  rwd=0.800  Reject/Normal
task=easy step= 43 phase=spike    LAG ██████████████░░░░░░░░░░  6100  POOL ████████████████████░░░░  83%  rwd=0.750  Approve/Throttle

• Red bar = signal above 75 % of max (danger zone)
• Yellow bar = 50–75 % (warning zone)
• Green bar = below 50 % (healthy)

These lines go to stderr only — they do not appear in the [STEP] stdout stream and do not affect the OpenEnv grader.

---

Step 6 — Quick smoke-test (no LLM needed)

Three modes are available via AGENT_MODE. Use the qtable mode for exact, reproducible scores:

Mode A — Heuristic agent (legacy dry-run)

$env:SPACE_URL  = "http://localhost:7860"
$env:DRY_RUN    = "true"           # equivalent to AGENT_MODE=heuristic
python inference.py

Mode B — Q-table agent (trained snapshot, 100% reproducible) ← recommended for judges

$env:SPACE_URL   = "http://localhost:7860"
$env:AGENT_MODE  = "qtable"
python inference.py

Requires results/qtable.pkl — run python train.py once to generate it.

Mode C — Heuristic (explicit)

$env:SPACE_URL   = "http://localhost:7860"
$env:AGENT_MODE  = "heuristic"
python inference.py

Expected scores by mode:

Task	Heuristic	Q-table	Threshold	Pass?
easy	~0.76	~0.81	≥ 0.75	✅
medium	~0.39–0.44	~0.52	≥ 0.45	✅
hard	~0.30–0.34	0.6650	≥ 0.30	✅

---

Step 7 — Run the full test suite

pytest tests/ -v
# Expected: 182 passed

Run with coverage:

pip install pytest-cov
pytest tests/ --cov=unified_gateway --cov-report=term-missing
# unified_gateway.py: 97%

---

Step 8 — Train the Q-table agent (optional)

Standard run

python train.py

Runs 500 episodes on the hard task in ~3–4 seconds on CPU. Produces:

• results/reward_curve.png — raw + rolling mean reward curve
• results/reward_staircase.png — phase-coloured staircase chart
• results/qtable.pkl — trained Q-table snapshot for AGENT_MODE=qtable
• results/lag_predictor.pt — LagPredictor weights (univariate world model)
• results/multi_obs_predictor.pt — MultiObsPredictor weights (full 10-dim world model, Fix 10.1)
• ASCII comparison table: Random vs Heuristic vs Trained

Expected key output lines:

[BLIND SPOT #1 DISCOVERED] episode=3 step=42 reward=0.8800 | ...
episode=10/500  recent_mean=0.6234  epsilon=0.990  lag_model_loss=0.012345  world_model_loss=0.023456  ...
hard  0.2507  0.2955  0.6650  0.30  PASS

A/B comparison mode (--compare)

pip install rich       # one-time
python train.py --compare

After training, renders a colour-coded rich table comparing the Heuristic (LLM baseline) agent against the Trained AEPO agent across all three tasks:

                AEPO — A/B Comparison: Heuristic (LLM Baseline) vs Trained Agent
┌──────────┬──────────┬──────────────────────────┬────────────────┬───────────┬────────┐
│ Task     │   Random │ Heuristic (LLM Baseline) │  Trained (AEPO)│ Threshold │  Pass? │
├──────────┼──────────┼──────────────────────────┼────────────────┼───────────┼────────┤
│ EASY     │   0.2134 │                   0.7612 │         0.8103 │      0.75 │  PASS  │
│ MEDIUM   │   0.1987 │                   0.4102 │         0.5240 │      0.45 │  PASS  │
│ HARD     │   0.1543 │                   0.2955 │         0.6650 │      0.30 │  PASS  │
└──────────┴──────────┴──────────────────────────┴────────────────┴───────────┴────────┘

Viewing the generated PNG charts

# Open both charts (Windows)
start results\reward_curve.png
start results\reward_staircase.png

The staircase chart (reward_staircase.png) colour-codes the background:

• Green region = Easy curriculum (level 0)
• Orange region = Medium curriculum (level 1)
• Red region = Hard curriculum (level 2)

The staircase pattern (agent improves → adversary escalates → agent adapts) is the primary visual proof of recursive self-improvement for the pitch demo.

---

Troubleshooting

ConnectionRefusedError on inference.py

The FastAPI server is not running. Start it first (Step 4).

qwen2.5-coder:32b not found by Ollama

Run ollama list to confirm the model name. If it shows qwen2.5-coder:32b but inference fails, ensure MODEL_NAME exactly matches the name shown by ollama list.

LLM returns malformed action

parse_llm_action() in inference.py catches all parse errors and falls back to the safe conservative action (Reject + FullVerify + Normal). You will see this in the step log as the same action repeating. This is expected for smaller models that don't follow the 6-integer output format consistently.

To improve LLM compliance, the system prompt in inference.py already instructs the model to output exactly six space-separated integers. If the model still produces malformed output, try adjusting temperature=0.0 (already set) or using a larger quantisation level in Ollama.

ModuleNotFoundError: No module named 'torch'

pip install torch==2.2.0+cpu --extra-index-url https://download.pytorch.org/whl/cpu

Ollama OpenAI endpoint not working

Ollama exposes an OpenAI-compatible API at /v1/chat/completions. Verify:

curl http://localhost:11434/v1/models
# Should list available models including qwen2.5-coder:32b

---

Local Testing Checklist

☐ ollama serve is running in background terminal
☐ ollama list shows qwen2.5-coder:32b
☐ uvicorn server.app:app --port 7860 is running
☐ curl http://localhost:7860/ returns {"status":"healthy"}
☐ curl http://localhost:7860/contract returns {"step_tuple":"4-tuple","openenv_compliant":true}
☐ AGENT_MODE=heuristic python inference.py → [END] lines for all 3 tasks
☐ AGENT_MODE=qtable python inference.py → hard score=0.67 (requires results/qtable.pkl)
☐ pytest tests/ -v → all tests pass
☐ (optional) python train.py → hard PASS; world_model_loss logged; results/multi_obs_predictor.pt created
☐ (optional) python train.py --compare → coloured rich A/B comparison table