ops/scripts/acceptance_test.md

MVP Acceptance Test Checklist

Pre-requisites

• Docker and Docker Compose installed
• LLM API key configured in .env
• All containers running (docker-compose up -d)

Test Scenarios

Scenario 1: Basic Query Flow

1. Open http://localhost:3000
2. Type: "Show all customers"
3. Verify workflow created message appears
4. Verify graph visualization updates
5. Wait for results (may take 5+ minutes due to pauses)
6. Verify table with customer data appears
7. Verify SQL query is displayed

Scenario 2: Human Intervention

1. Start new query: "Count all orders"
2. During the 5-minute pause (watch agent logs)
3. Open Neo4j Browser: http://localhost:7474
4. Run query: MATCH (i:Instruction {status: 'pending', type: 'generate_sql'}) RETURN i
5. Edit instruction: SET i.parameters = '{"question": "Count orders with status completed"}'
6. Verify agent uses modified query after pause
7. Verify results reflect the change

Scenario 3: Stop Workflow

1. Start query: "Find expensive orders"
2. Click STOP button during execution
3. Verify workflow stops
4. Check Neo4j: MATCH (w:Workflow) RETURN w.status shows 'stopped'

Scenario 4: Graph Visualization

1. Start any query
2. Verify workflow node appears (blue/gray)
3. Verify instruction nodes appear
4. Verify colors change:
   • Gray = pending
   • Yellow = executing
   • Green = complete
5. Click on node shows properties

Scenario 5: Audit Trail

1. After running queries, open Neo4j Browser
2. Run: MATCH (l:Log) RETURN l ORDER BY l.timestamp DESC LIMIT 10
3. Verify all MCP operations are logged
4. Run: MATCH (e:Execution) RETURN e
5. Verify all executions have results

Performance Checks

• Agent processes instructions within 30 seconds of becoming available
• Frontend responds within 1 second
• Graph updates within 10 seconds
• No memory leaks after 10+ queries

Data Validation

Run these queries in Neo4j Browser:

// Check schema discovered
MATCH (t:Table) RETURN t.name

// Check instructions executed
MATCH (i:Instruction)-[:EXECUTED_AS]->(e:Execution)
RETURN i.type, i.status, e.completed_at

// Check successful SQL generation
MATCH (qt:QueryTemplate)
RETURN qt.query, qt.created_at
ORDER BY qt.created_at DESC
LIMIT 5

// Check workflow completion rate
MATCH (w:Workflow)
RETURN w.status, count(w) as count

Sign-off

• All test scenarios pass
• No errors in container logs
• System recovers from agent restart
• Clean startup from docker-compose down && docker-compose up -d

Tester: _________________
Date: _________________
Version: MVP 1.0

---

Final validation commands:

# Complete test sequence
docker-compose down
docker-compose up -d
sleep 10

# Check health
curl -s http://localhost:8000/health && echo "MCP: OK"
curl -s http://localhost:3000 > /dev/null && echo "Frontend: OK"

# Seed data
docker-compose exec mcp python /app/ops/scripts/seed.py

# Run automated validation
docker-compose exec mcp python /app/ops/scripts/validate.py

# Run demo
bash ops/scripts/demo.sh

# In another terminal, watch all logs
docker-compose logs -f

# Manual test in another terminal
curl -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "X-API-Key: dev-key-123" \
  -d '{
    "tool": "write_graph",
    "params": {
      "action": "create_node",
      "label": "Instruction",
      "properties": {
        "id": "final-test",
        "type": "generate_sql",
        "status": "pending",
        "sequence": 999,
        "pause_duration": 30,
        "parameters": "{\"question\": \"What is the total revenue from all orders?\"}"
      }
    }
  }'

# Check execution after ~30 seconds
curl -X POST http://localhost:8000/mcp \
  -H "Content-Type: application/json" \
  -H "X-API-Key: dev-key-123" \
  -d '{
    "tool": "query_graph",
    "params": {
      "query": "MATCH (i:Instruction {id: \"final-test\"})-[:EXECUTED_AS]->(e:Execution) RETURN e.result"
    }
  }'