Spaces:
No application file
No application file
File size: 4,109 Bytes
ead5455 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | # MVP Acceptance Test Checklist
## Pre-requisites
- [ ] Docker and Docker Compose installed
- [ ] LLM API key configured in `.env`
- [ ] All containers running (`docker-compose up -d`)
## Test Scenarios
### Scenario 1: Basic Query Flow
1. [ ] Open http://localhost:3000
2. [ ] Type: "Show all customers"
3. [ ] Verify workflow created message appears
4. [ ] Verify graph visualization updates
5. [ ] Wait for results (may take 5+ minutes due to pauses)
6. [ ] Verify table with customer data appears
7. [ ] Verify SQL query is displayed
### Scenario 2: Human Intervention
1. [ ] Start new query: "Count all orders"
2. [ ] During the 5-minute pause (watch agent logs)
3. [ ] Open Neo4j Browser: http://localhost:7474
4. [ ] Run query: `MATCH (i:Instruction {status: 'pending', type: 'generate_sql'}) RETURN i`
5. [ ] Edit instruction: `SET i.parameters = '{"question": "Count orders with status completed"}'`
6. [ ] Verify agent uses modified query after pause
7. [ ] Verify results reflect the change
### Scenario 3: Stop Workflow
1. [ ] Start query: "Find expensive orders"
2. [ ] Click STOP button during execution
3. [ ] Verify workflow stops
4. [ ] Check Neo4j: `MATCH (w:Workflow) RETURN w.status` shows 'stopped'
### Scenario 4: Graph Visualization
1. [ ] Start any query
2. [ ] Verify workflow node appears (blue/gray)
3. [ ] Verify instruction nodes appear
4. [ ] Verify colors change:
- Gray = pending
- Yellow = executing
- Green = complete
5. [ ] Click on node shows properties
### Scenario 5: Audit Trail
1. [ ] After running queries, open Neo4j Browser
2. [ ] Run: `MATCH (l:Log) RETURN l ORDER BY l.timestamp DESC LIMIT 10`
3. [ ] Verify all MCP operations are logged
4. [ ] Run: `MATCH (e:Execution) RETURN e`
5. [ ] Verify all executions have results
## Performance Checks
- [ ] Agent processes instructions within 30 seconds of becoming available
- [ ] Frontend responds within 1 second
- [ ] Graph updates within 10 seconds
- [ ] No memory leaks after 10+ queries
## Data Validation
Run these queries in Neo4j Browser:
```cypher
// Check schema discovered
MATCH (t:Table) RETURN t.name
// Check instructions executed
MATCH (i:Instruction)-[:EXECUTED_AS]->(e:Execution)
RETURN i.type, i.status, e.completed_at
// Check successful SQL generation
MATCH (qt:QueryTemplate)
RETURN qt.query, qt.created_at
ORDER BY qt.created_at DESC
LIMIT 5
// Check workflow completion rate
MATCH (w:Workflow)
RETURN w.status, count(w) as count
```
## Sign-off
- [ ] All test scenarios pass
- [ ] No errors in container logs
- [ ] System recovers from agent restart
- [ ] Clean startup from `docker-compose down && docker-compose up -d`
**Tester:** _________________
**Date:** _________________
**Version:** MVP 1.0
---
## Final validation commands:
```bash
# Complete test sequence
docker-compose down
docker-compose up -d
sleep 10
# Check health
curl -s http://localhost:8000/health && echo "MCP: OK"
curl -s http://localhost:3000 > /dev/null && echo "Frontend: OK"
# Seed data
docker-compose exec mcp python /app/ops/scripts/seed.py
# Run automated validation
docker-compose exec mcp python /app/ops/scripts/validate.py
# Run demo
bash ops/scripts/demo.sh
# In another terminal, watch all logs
docker-compose logs -f
# Manual test in another terminal
curl -X POST http://localhost:8000/mcp \
-H "Content-Type: application/json" \
-H "X-API-Key: dev-key-123" \
-d '{
"tool": "write_graph",
"params": {
"action": "create_node",
"label": "Instruction",
"properties": {
"id": "final-test",
"type": "generate_sql",
"status": "pending",
"sequence": 999,
"pause_duration": 30,
"parameters": "{\"question\": \"What is the total revenue from all orders?\"}"
}
}
}'
# Check execution after ~30 seconds
curl -X POST http://localhost:8000/mcp \
-H "Content-Type: application/json" \
-H "X-API-Key: dev-key-123" \
-d '{
"tool": "query_graph",
"params": {
"query": "MATCH (i:Instruction {id: \"final-test\"})-[:EXECUTED_AS]->(e:Execution) RETURN e.result"
}
}'
```
|