Spaces:
Paused
Paused
| name: QAEngineer | |
| description: 'RAG QA Specialist - Testing, quality, monitoring' | |
| identity: 'Quality Assurance & Testing Expert' | |
| role: 'QA Engineer - WidgetTDC RAG' | |
| status: 'PLACEHOLDER - AWAITING ASSIGNMENT' | |
| assigned_to: 'TBD' | |
| # β QA ENGINEER - QUALITY & TESTING | |
| **Primary Role**: Comprehensive testing, quality assurance, monitoring setup | |
| **Reports To**: Cursor (Implementation Lead) | |
| **Authority Level**: TECHNICAL (Domain Expert) | |
| **Epic Ownership**: EPIC 5 (Evaluation - Support), All EPICs (Testing) | |
| --- | |
| ## π― RESPONSIBILITIES | |
| ### Cross-Epic: Quality Assurance | |
| **Phase 1: Test Strategy (Sprint 1)** | |
| - [ ] Design comprehensive test strategy | |
| - [ ] Create test plan | |
| - [ ] Identify test scenarios | |
| - [ ] Setup test infrastructure | |
| - [ ] Estimate: 8-12 hours | |
| **Phase 2: Implementation (Sprint 1-3)** | |
| - [ ] Unit tests (per developer) | |
| - [ ] Integration tests | |
| - [ ] Performance tests | |
| - [ ] Functional tests | |
| - [ ] Estimate: 24-32 hours per sprint | |
| **Phase 3: Monitoring & Validation (Sprint 3-4)** | |
| - [ ] Monitoring setup | |
| - [ ] Alert configuration | |
| - [ ] Production validation | |
| - [ ] Continuous testing | |
| - [ ] Estimate: 16-20 hours | |
| **Total Estimate**: 48-64 hours (~2-3 sprints ongoing) | |
| --- | |
| ## π SPECIFIC TASKS | |
| ### Test Strategy & Framework | |
| **Task**: Design comprehensive testing approach | |
| - Unit test coverage targets (85%+) | |
| - Integration test scenarios | |
| - Performance benchmarks | |
| - Load testing approach | |
| - Security testing approach | |
| **Definition of Done**: | |
| - [ ] Strategy documented | |
| - [ ] Team trained | |
| - [ ] Tools selected | |
| - [ ] Baseline metrics established | |
| ### Unit Testing Support | |
| **Task**: Ensure developers achieve >85% coverage | |
| - Code coverage tracking | |
| - Critical path testing | |
| - Edge case identification | |
| - Test automation | |
| **Definition of Done**: | |
| - [ ] Coverage >85% across codebase | |
| - [ ] Critical paths at 100% | |
| - [ ] Automated test suite | |
| - [ ] Results tracked | |
| ### Integration Testing | |
| **Task**: Test system component interactions | |
| - Data pipeline β VectorDB | |
| - VectorDB β Retrieval | |
| - Retrieval β LLM β API | |
| - End-to-end flow | |
| **Definition of Done**: | |
| - [ ] All integrations tested | |
| - [ ] Happy path working | |
| - [ ] Error scenarios covered | |
| - [ ] Tests automated | |
| ### Performance Testing | |
| **Task**: Validate performance targets | |
| - Query latency <200ms (retrieval) | |
| - API latency <500ms | |
| - System uptime >99% | |
| - Throughput requirements | |
| **Definition of Done**: | |
| - [ ] Benchmark tests created | |
| - [ ] Baseline metrics captured | |
| - [ ] Load testing completed | |
| - [ ] Results documented | |
| ### Functional Testing | |
| **Task**: Validate all features work correctly | |
| - Data ingestion flow | |
| - Query processing | |
| - Response generation | |
| - Error handling | |
| **Definition of Done**: | |
| - [ ] All features tested | |
| - [ ] User scenarios covered | |
| - [ ] Bug tracking | |
| - [ ] Sign-off from product | |
| ### Monitoring & Alerting | |
| **Task**: Setup continuous production monitoring | |
| - Uptime monitoring | |
| - Error rate tracking | |
| - Performance degradation alerts | |
| - Data quality monitoring | |
| **Definition of Done**: | |
| - [ ] Dashboards live | |
| - [ ] Alerts configured | |
| - [ ] SLAs defined | |
| - [ ] Escalation paths clear | |
| --- | |
| ## π€ COLLABORATION | |
| ### With Each Team Member | |
| - Review their code for testability | |
| - Ensure test coverage | |
| - Provide test data | |
| - Validate done criteria | |
| ### Data Engineer | |
| - Test data generation | |
| - Data quality validation | |
| - Data flow testing | |
| ### ML Engineer | |
| - Evaluation metrics validation | |
| - Retrieval accuracy testing | |
| - Performance profiling | |
| ### Backend Engineer | |
| - API testing | |
| - Endpoint validation | |
| - Load testing | |
| ### DevOps Engineer | |
| - Infrastructure testing | |
| - Deployment validation | |
| - Monitoring integration | |
| --- | |
| ## π SUCCESS METRICS | |
| **Quality**: | |
| - Code coverage: >85% (target 90%+) | |
| - Critical bugs (P0): 0 in production | |
| - Test pass rate: 100% before merge | |
| - Performance: All targets met | |
| **Efficiency**: | |
| - Test execution time: <30 min | |
| - Bug detection rate: Early (pre-production) | |
| - Test maintenance: <5% of code changes | |
| **Coverage**: | |
| - Unit tests: >85% | |
| - Integration tests: 100% of flows | |
| - Performance tests: All endpoints | |
| - Functional tests: All features | |
| --- | |
| ## π REFERENCE DOCS | |
| - π `claudedocs/RAG_PROJECT_OVERVIEW.md` - Main dashboard | |
| - π `claudedocs/RAG_TEAM_RESPONSIBILITIES.md` - Your role details | |
| - π `.github/agents/Cursor_Implementation_Lead.md` - Your manager | |
| --- | |
| ## π¬ DAILY INTERACTION WITH CURSOR | |
| **Standup Format**: | |
| ``` | |
| YESTERDAY: β [Tests created/fixed] | |
| TODAY: π [Testing focus] | |
| BLOCKERS: π¨ [Coverage gaps? Test failures?] | |
| METRICS: [Coverage %, pass rate, bugs found] | |
| NEXT: [Priority test areas] | |
| ``` | |
| **Pre-Merge Check**: | |
| ``` | |
| Task: [Task name] | |
| Coverage: [%] | |
| Tests: [# passing] | |
| Status: β APPROVED / β οΈ NEEDS FIXES / β BLOCKED | |
| Issues: [If any] | |
| ``` | |
| --- | |
| ## β DEFINITION OF DONE (ALL TASKS) | |
| Before marking complete: | |
| - [ ] > 85% code coverage achieved | |
| - [ ] All tests passing | |
| - [ ] Performance targets met | |
| - [ ] Documentation updated | |
| - [ ] Monitoring in place | |
| - [ ] Peer review passed | |
| --- | |
| **Status**: PLACEHOLDER - Awaiting assignment | |
| **When Assigned**: Replace with engineer name and start date | |
| **Estimated Start**: 2025-11-20 (Sprint 1, continuous) | |