ui-regression-testing-2 / QUICKSTART.md
riazmo's picture
Upload 61 files
6f38c76 verified
# Quick Start Guide - POC Execution
## πŸš€ Get Started in 5 Minutes
### Step 1: Extract and Setup (2 minutes)
```bash
# Extract the project
unzip langgraph_ui_regression.zip
cd langgraph_ui_regression
# Run setup script
chmod +x setup.sh
./setup.sh
# Or manually install dependencies
pip install -r requirements.txt
playwright install chromium
```
### Step 2: Configure Credentials (1 minute)
```bash
# Copy and edit environment file
cp .env.example .env
nano .env
```
**Add your credentials:**
```env
FIGMA_FILE_KEY=your_figma_file_key
FIGMA_ACCESS_TOKEN=your_figma_access_token
WEBSITE_URL=https://v0-replicate-checkout-design.vercel.app/
```
### Step 3: Run the System (2 minutes)
```bash
# Execute the workflow
python main.py
# Or with custom output directory
python main.py --output-dir ./my_reports --debug
```
### Step 4: Check Results
Reports are generated in `./reports/` directory:
- `report_YYYYMMDD_HHMMSS.json` - Machine-readable report
- `report_YYYYMMDD_HHMMSS.md` - Human-readable report
---
## πŸ“‹ What Happens During Execution
```
1. Agent 0: Creates test plan
↓
2. Agent 1: Extracts design specs from Figma
↓
3. Agent 2: Extracts CSS specs from website
↓
4. Agent 3: Compares and generates findings
↓
5. Reports: JSON + Markdown output
```
---
## 🎯 Expected Output
### Console Output
```
======================================================================
πŸš€ UI REGRESSION TESTING SYSTEM
======================================================================
πŸ“‹ Configuration:
Figma File Key: abc123def456...
Website URL: https://v0-replicate-checkout-design.vercel.app/
Output Directory: ./reports
Debug Mode: False
======================================================================
πŸ€– Agent 0: Super Agent - Generating Test Plan...
βœ“ Configured 2 viewports
βœ“ Configured 7 test categories
βœ“ Test plan generated with 140 total tests
🎨 Agent 1: Design Inspector - Extracting Design Specs...
βœ“ Loaded Figma file: Checkout Design
βœ“ Extracted 156 design specifications
- colors: 12 specs
- typography: 24 specs
- spacing: 18 specs
- layout: 22 specs
- borders: 15 specs
- effects: 8 specs
🌐 Agent 2: Website Inspector - Extracting Website Specs...
πŸ“± Processing desktop viewport (1440px)...
βœ“ Loaded https://v0-replicate-checkout-design.vercel.app/
βœ“ Extracted specs for desktop
πŸ“± Processing mobile viewport (375px)...
βœ“ Extracted specs for mobile
πŸ” Agent 3: Difference Analyzer - Comparing Specs...
πŸ“Š Analyzing colors...
βœ“ Found 5 differences
πŸ“Š Analyzing typography...
βœ“ Found 3 differences
πŸ“Š Analyzing spacing...
βœ“ Found 2 differences
...
βœ“ Total findings: 15
- High: 3
- Medium: 7
- Low: 5
πŸ“Š Generating Reports...
======================================================================
βœ… EXECUTION COMPLETED SUCCESSFULLY
======================================================================
πŸ“„ Reports:
JSON: ./reports/report_20240103_120000.json
Markdown: ./reports/report_20240103_120000.md
πŸ“ˆ Summary:
Total Findings: 15
High Severity: 3
Medium Severity: 7
Low Severity: 5
Overall Score: 75/100
======================================================================
```
### Report Output
**Markdown Report Preview:**
```markdown
# UI Regression Testing Report
**Generated**: 2024-01-03 12:00:00
## Summary
- **Total Findings**: 15
- **πŸ”΄ High Severity**: 3
- **🟑 Medium Severity**: 7
- **🟒 Low Severity**: 5
- **Overall Score**: 75/100
## High Severity Issues
#### submit_button - backgroundColor
**Category**: colors
**Design Value**: `#FF0000`
**Website Value**: `#FF0001`
**Analysis**: Significant color mismatch affecting brand consistency...
```
---
## πŸ”§ Troubleshooting
### Problem: "ModuleNotFoundError: No module named 'langgraph'"
**Solution:**
```bash
pip install -r requirements.txt
```
### Problem: "Figma API authentication failed"
**Solution:**
1. Verify your Figma access token is correct
2. Ensure the token has access to the file
3. Check token hasn't expired
### Problem: "Website not loading"
**Solution:**
```bash
# Test the URL directly
curl https://v0-replicate-checkout-design.vercel.app/
# If it works, try running with debug mode
python main.py --debug
```
### Problem: "Playwright browser not found"
**Solution:**
```bash
playwright install chromium
```
---
## πŸ“Š Understanding the Reports
### JSON Report Structure
```json
{
"metadata": { ... },
"summary": {
"total_findings": 15,
"high_severity": 3,
"medium_severity": 7,
"low_severity": 5,
"overall_score": 75
},
"findings": [
{
"component": "button",
"category": "colors",
"property": "backgroundColor",
"severity": "high",
"design_value": "#FF0000",
"website_value": "#FF0001",
"hf_analysis": "..."
}
]
}
```
### Severity Levels
- πŸ”΄ **High**: Critical issues affecting functionality or brand consistency
- 🟑 **Medium**: Visual inconsistencies that should be addressed
- 🟒 **Low**: Minor differences that are visually acceptable
---
## 🎯 Next Steps After POC
1. **Review Findings**: Check the generated reports
2. **Fix Issues**: Address high and medium severity findings
3. **Re-run Tests**: Execute again to verify fixes
4. **Add Gradio Dashboard**: For visual comparison (Phase 7)
5. **CI/CD Integration**: Automate testing in your pipeline
---
## πŸ“ž Need Help?
1. Check the main **README.md** for detailed documentation
2. Enable debug mode: `python main.py --debug`
3. Review the generated reports for specific issues
4. Check the troubleshooting section above
---
**Ready to test? Run: `python main.py`** πŸš€