| # Design System Extractor v2 — Master Context File |
|
|
| > **Upload this file to refresh Claude's context when continuing work on this project.** |
|
|
| **Last Updated:** January 2026 |
|
|
| --- |
|
|
| ## 📁 Files Changed in Latest Session |
|
|
| | File | What Changed | |
| |------|--------------| |
| | `agents/extractor.py` | Enhanced 7-source extraction (DOM, CSS vars, SVG, inline, stylesheets, external CSS, page scan) | |
| | `agents/firecrawl_extractor.py` | **NEW** Agent 1B for deep CSS parsing | |
| | `agents/semantic_analyzer.py` | **NEW** Agent 1C for semantic color categorization (brand/text/bg/border) | |
| | `core/preview_generator.py` | AS-IS previews + Color Ramps sorted by brand priority | |
| | `app.py` | Stage 1 UI now has 6 preview tabs including Semantic Colors | |
| | `docs/CONTEXT.md` | Updated with semantic analyzer, full architecture diagrams | |
|
|
| --- |
|
|
| ## 🎯 Project Goal |
|
|
| Build a **semi-automated, human-in-the-loop agentic system** that: |
| 1. Reverse-engineers a design system from a live website |
| 2. Reconstructs and upgrades it into a modern, scalable design system |
| 3. Outputs production-ready JSON tokens (Figma Tokens Studio compatible) |
|
|
| **Philosophy:** This is a design-aware co-pilot, NOT a magic button. Humans decide, agents propose. |
|
|
| --- |
|
|
| ## 🤔 Why This Project? (Market Differentiation) |
|
|
| ### The Problem We Solve |
|
|
| | Pain Point | Who Has It | Current Solutions | Why They Fail | |
| |------------|------------|-------------------|---------------| |
| | Legacy websites with no design system | Enterprise teams | Manual audit (weeks) | Time-consuming, error-prone | |
| | Inconsistent design tokens scattered in CSS | Agencies inheriting projects | Figma plugins (style extractors) | Only extract from Figma, not live sites | |
| | Need to modernize without breaking existing | Product teams | Design system generators | Generate new, don't reverse-engineer existing | |
| | AA compliance gaps unknown | Accessibility teams | Contrast checkers | Check one color at a time, no system view | |
|
|
| ### Existing Tools & Their Gaps |
|
|
| | Tool | What It Does | Gap We Fill | |
| |------|--------------|-------------| |
| | **Figma Tokens Studio** | Manages tokens in Figma | Doesn't extract from websites | |
| | **Style Dictionary** | Transforms tokens to code | Needs tokens first (we create them) | |
| | **Polypane/VisBug** | Inspect live sites | No systematic extraction or upgrade | |
| | **AI Design Tools** (Galileo, Uizard) | Generate new designs | Don't reverse-engineer existing | |
| | **CSS Stats** | Analyze CSS files | Statistics only, no actionable tokens | |
| | **Chromatic/Percy** | Visual regression | Compare, don't extract or upgrade | |
|
|
| ### Our Unique Value Proposition |
|
|
| ``` |
| ┌─────────────────────────────────────────────────────────────────────────────┐ |
| │ WHAT MAKES US DIFFERENT │ |
| ├─────────────────────────────────────────────────────────────────────────────┤ |
| │ │ |
| │ 1. REVERSE-ENGINEERING (not generation) │ |
| │ • Extracts from LIVE websites, not design files │ |
| │ • Preserves what's working, upgrades what's broken │ |
| │ • Respects existing brand decisions │ |
| │ │ |
| │ 2. MULTI-AGENT REASONING (not single LLM) │ |
| │ • Two analysts with different perspectives │ |
| │ • HEAD compiler resolves conflicts │ |
| │ • Shows reasoning, not just results │ |
| │ │ |
| │ 3. HUMAN-IN-THE-LOOP (not magic button) │ |
| │ • Designer reviews every stage │ |
| │ • Accept/reject individual tokens │ |
| │ • Choose from upgrade OPTIONS, not forced decisions │ |
| │ │ |
| │ 4. VISUAL PREVIEWS (not just data tables) │ |
| │ • Typography rendered in actual detected font │ |
| │ • Color ramps with AA compliance per shade │ |
| │ • See before you export │ |
| │ │ |
| │ 5. COST-TRANSPARENT (not black box) │ |
| │ • Shows token usage and cost per analysis │ |
| │ • Uses HF free tier ($0.10/mo) or Pro ($2/mo) │ |
| │ • ~$0.05 per full analysis │ |
| │ │ |
| └─────────────────────────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| ### Target Users |
|
|
| | User | Use Case | Value | |
| |------|----------|-------| |
| | **UX Managers** (like you!) | Modernize legacy booking platforms | Weeks → Hours | |
| | **Design System Teams** | Audit and standardize existing properties | Systematic, not ad-hoc | |
| | **Agencies** | Onboard client projects with no documentation | Instant design inventory | |
| | **Accessibility Consultants** | AA compliance audit with fixes | Full palette view | |
| | **Developers** | Get production-ready tokens from designer's website | No manual translation | |
|
|
| ### Why Not Just Use [X]? |
|
|
| **"Why not just inspect the CSS manually?"** |
| → You could, but it takes weeks for a complex site. We do it in minutes with systematic coverage. |
|
|
| **"Why not use Figma's native styles?"** |
| → Many legacy sites were never in Figma. We extract from the source of truth: the live website. |
|
|
| **"Why do you need AI? Can't rules handle this?"** |
| → Rules extract tokens. AI understands *design intent* — why is this color used here? What scale was intended? Where does it deviate from best practices? |
|
|
| **"Isn't this just CSS Stats with AI?"** |
| → CSS Stats tells you what exists. We tell you what it *should* be and give you actionable upgrade paths. |
|
|
| --- |
|
|
| ## 🏗️ Architecture Overview |
|
|
| ``` |
| ┌─────────────────────────────────────────────────────────────────────────────┐ |
| │ TECH STACK │ |
| ├─────────────────────────────────────────────────────────────────────────────┤ |
| │ Frontend: Gradio (long-scroll, sectioned UI with live preview) │ |
| │ Orchestration: LangGraph (agent state management & workflow) │ |
| │ Models: HuggingFace Inference Providers (Novita, Groq, etc.) │ |
| │ Hosting: Hugging Face Spaces │ |
| │ Storage: HF Spaces persistent storage │ |
| │ Output: Platform-agnostic JSON tokens (Figma Tokens Studio) │ |
| └─────────────────────────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| --- |
|
|
| ## 🧠 Model Assignments |
|
|
| ### Stage 2: Multi-Agent Analysis |
|
|
| | Agent | Role | Model | Provider | Cost | |
| |-------|------|-------|----------|------| |
| | **LLM 1** | Design Analyst 1 | `Qwen/Qwen2.5-72B-Instruct` | Novita | $0.29/M in, $0.59/M out | |
| | **LLM 2** | Design Analyst 2 | `meta-llama/Llama-3.3-70B-Instruct` | Novita | $0.59/M in, $0.79/M out | |
| | **HEAD** | Compiler | `meta-llama/Llama-3.3-70B-Instruct` | Novita | $0.59/M in, $0.79/M out | |
| | **Rules** | Calculations | None (Rule-based) | — | FREE | |
|
|
| **Architecture:** |
| ``` |
| ┌─────────────────────────────────────────────────────────────────────────────┐ |
| │ PARALLEL ANALYSIS │ |
| │ │ |
| │ LLM 1 (Qwen) LLM 2 (Llama) Rule Engine │ |
| │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ |
| │ │ Global │ │ Western │ │ Math │ │ |
| │ │ Design │ │ Design │ │ Only │ │ |
| │ │ Patterns │ │ Patterns │ │ (FREE) │ │ |
| │ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ |
| │ │ │ │ │ |
| │ └─────────────────────┼─────────────────────┘ │ |
| │ │ │ |
| │ ▼ │ |
| │ ┌─────────────────┐ │ |
| │ │ HEAD COMPILER │ │ |
| │ │ │ │ |
| │ │ • Compare views │ │ |
| │ │ • Resolve diff │ │ |
| │ │ • Final recs │ │ |
| │ └─────────────────┘ │ |
| └─────────────────────────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| ### Other Agents |
|
|
| | Agent | Role | Model | Provider | Why | |
| |-------|------|-------|----------|-----| |
| | **Agent 1** | Crawler & Extractor | None (Rule-based) | — | Pure CSS extraction, no LLM needed | |
| | **Agent 2** | Normalizer | `microsoft/Phi-3.5-mini-instruct` | Novita | Fast, great structured output | |
| | **Agent 4** | Generator | `mistralai/Codestral-22B-v0.1` | Novita | Code specialist, JSON formatting | |
|
|
| ### Provider Configuration |
|
|
| Default provider: **Novita** (configurable in `config/agents.yaml`) |
|
|
| Available providers (via HuggingFace Inference Providers): |
| - **novita** - Default, good balance |
| - **groq** - Fastest |
| - **cerebras** - Ultra-fast |
| - **sambanova** - Good for Llama |
| - **together** - Wide model selection |
|
|
| ### Cost Tracking |
|
|
| Estimated cost per Stage 2 analysis: **~$0.05** |
| - Free tier: $0.10/month |
| - Pro tier: $2.00/month ($9/mo subscription) |
|
|
| --- |
|
|
| ## 👁️ Visual Previews |
|
|
| ### Stage 1: AS-IS Previews (No Enhancements) |
|
|
| Shows raw extracted values exactly as found on the website: |
|
|
| | Preview | What It Shows | |
| |---------|---------------| |
| | **Typography** | Actual font rendered with detected styles | |
| | **Colors** | Simple swatches with hex, frequency, context, AA status | |
| | **Spacing** | Visual bars representing each spacing value | |
| | **Radius** | Boxes with each border-radius applied | |
| | **Shadows** | Cards with each box-shadow applied | |
|
|
| ### Stage 2: Enhanced Previews (Upgraded) |
|
|
| Shows proposed upgrades and improvements: |
|
|
| | Preview | What It Shows | |
| |---------|---------------| |
| | **Typography** | Type scale comparison (1.2, 1.25, 1.333 ratios) | |
| | **Color Ramps** | 11 shades (50-950) with AA compliance per shade | |
|
|
| --- |
|
|
| ## 🔍 Enhanced Extraction (Agent 1) |
|
|
| Agent 1 now extracts from **5 sources** to capture ALL colors: |
|
|
| ``` |
| ┌─────────────────────────────────────────────────────────────────────────────┐ |
| │ ENHANCED EXTRACTION SOURCES │ |
| ├─────────────────────────────────────────────────────────────────────────────┤ |
| │ │ |
| │ 1. DOM Computed Styles │ |
| │ • window.getComputedStyle(element) │ |
| │ • Captures: color, background-color, border-color, etc. │ |
| │ │ |
| │ 2. CSS Variables │ |
| │ • :root { --primary-color: #3860be; } │ |
| │ • Parses all stylesheets for CSS custom properties │ |
| │ │ |
| │ 3. SVG Colors │ |
| │ • <svg fill="#00c4cc"> │ |
| │ • <path stroke="#3860be"> │ |
| │ │ |
| │ 4. Inline Styles │ |
| │ • <div style="background-color: #bcd432;"> │ |
| │ • Parses style attributes for color values │ |
| │ │ |
| │ 5. Stylesheet Rules │ |
| │ • Parses CSS rules that may not be applied to visible elements │ |
| │ • Catches hover states, pseudo-elements, etc. │ |
| │ │ |
| └─────────────────────────────────────────────────────────────────────────────┘ |
| ``` |
|
|
| --- |
|
|
| ## 📋 Enhanced Logging |
|
|
| ### Stage 1 Extraction Logs |
|
|
| Shows detailed extraction progress: |
| ``` |
| ============================================================ |
| 🖥️ DESKTOP EXTRACTION (1440px) |
| ============================================================ |
|
|
| 📡 Enhanced extraction from 5 sources: |
| 1. DOM computed styles (getComputedStyle) |
| 2. CSS variables (:root { --color: }) |
| 3. SVG colors (fill, stroke) |
| 4. Inline styles (style='color:') |
| 5. Stylesheet rules (CSS files) |
| 6. External CSS files (fetch & parse) |
| 7. Page content scan (brute-force) |
|
|
| 📊 EXTRACTION RESULTS: |
| Colors: 45 unique |
| Typography: 12 styles |
| Spacing: 28 values |
| Radius: 8 values |
| Shadows: 4 values |
|
|
| 🎨 CSS Variables found: 15 |
| --primary-color: #3860be |
| --accent-color: #00c4cc |
| --brand-lime: #bcd432 |
| ... and 12 more |
|
|
| 🔄 Normalizing (deduping, naming)... |
| ✅ Normalized: 32 colors, 10 typography, 18 spacing |
|
|
| ============================================================ |
| 🔥 FIRECRAWL CSS EXTRACTION |
| ============================================================ |
|
|
| 🌐 Scraping: https://example.com |
| ✅ Page scraped (125000 chars) |
| 📝 Parsing <style> blocks... |
| Found 5 style blocks |
| 🔗 Finding linked CSS files... |
| Found 8 CSS files |
| 📄 Fetching: main.css... |
| ✅ Parsed (234 colors) |
| 📄 Fetching: theme.css... |
| ✅ Parsed (45 colors) |
| |
| 📊 FIRECRAWL RESULTS: |
| CSS files parsed: 8 |
| Style blocks parsed: 5 |
| CSS variables found: 23 |
| Unique colors found: 156 |
|
|
| 🎨 Top colors found: |
| #06b2c4 (used 45x) |
| #c1df1f (used 38x) |
| #373737 (used 120x) |
| |
| 🔀 Merging Firecrawl colors with Playwright extraction... |
| ✅ Added 12 new colors from Firecrawl |
| 📊 Total colors now: 44 |
|
|
| ============================================================ |
| 🧠 SEMANTIC COLOR ANALYSIS |
| ============================================================ |
|
|
| 📊 Analyzing 143 colors... |
| Using rule-based analysis (no LLM) |
|
|
| 📊 SEMANTIC ANALYSIS RESULTS: |
|
|
| 🎨 BRAND COLORS: |
| primary: #06b2c4 (high) |
| └─ Most frequent saturated color on interactive elements (freq: 33) |
| secondary: #c1df1f (medium) |
| └─ Second most frequent brand color (freq: 15) |
| |
| 📝 TEXT COLORS: |
| primary: #373737 (high) |
| secondary: #666666 (medium) |
| |
| 🖼️ BACKGROUND COLORS: |
| primary: #ffffff (high) |
| secondary: #f5f5f5 (medium) |
| |
| 📈 SUMMARY: |
| Total colors analyzed: 143 |
| Brand colors found: 2 |
| Clear hierarchy: Yes |
| Analysis method: rule-based |
| ``` |
| |
| ### Stage 2 LLM Analysis Logs (With Semantic Context) |
|
|
| Shows detailed reasoning from each agent WITH semantic context: |
|
|
| ``` |
| ============================================================ |
| 🧠 STAGE 2: MULTI-AGENT ANALYSIS |
| ============================================================ |
|
|
| 🧠 SEMANTIC CONTEXT FROM STAGE 1: |
| Brand Primary: #06b2c4 |
| Text Primary: #373737 |
| Analysis Method: rule-based |
|
|
| ======================================================= |
| 🤖 LLM 1: meta-llama/Llama-3.1-70B-Instruct |
| ======================================================= |
| Provider: novita |
| 💰 Cost: $0.29/M in, $0.59/M out |
| 📝 Task: Typography, Colors, AA, Spacing analysis |
| 🧠 Semantic context: Yes ← NEW: LLM knows color roles! |
|
|
| 📊 LLM 1 FINDINGS: |
| |
| COLORS (with semantic context): |
| ├─ Brand Primary (#06b2c4): "Fails AA on white (3.2:1)" |
| ├─ Suggested fix: "#0891a8 (4.6:1)" |
| └─ Score: 6/10 |
|
|
| ======================================================= |
| 🎯 HEAD: Compiling final recommendations... |
| ======================================================= |
|
|
| 📥 INPUT: Analyzing outputs from LLM 1 + LLM 2 + Rules + Semantic... |
| |
| 📊 HEAD SYNTHESIS: |
| |
| COLOR RECOMMENDATIONS (per semantic role): |
| ├─ brand.primary: #06b2c4 → Keep for branding, use #0891a8 for text |
| ├─ text.primary: #373737 → Keep (passes AA) |
| └─ Generate ramps for: brand.primary, brand.secondary, neutral |
| ``` |
| |
| --- |
| |
| ## 🤖 Agent Personas |
| |
| ### Agent 1A: Website Crawler & Enhanced Extractor |
| - **Persona:** Meticulous Design Archaeologist |
| - **Tool:** Playwright |
| - **Job:** |
| - Auto-discover 10+ pages from base URL |
| - Crawl Desktop (1440px) + Mobile (375px) separately |
| - Scroll to bottom + wait for network idle |
| - **ENHANCED: Extract from 7 sources:** |
| 1. DOM computed styles (`getComputedStyle`) |
| 2. CSS variables (`:root { --primary: #xxx }`) |
| 3. SVG colors (`fill`, `stroke` attributes) |
| 4. Inline styles (`style="background-color: #xxx"`) |
| 5. Stylesheet rules (CSS files, hover states, pseudo-elements) |
| 6. External CSS files (fetch & parse to bypass CORS) |
| 7. Page content scan (brute-force regex on HTML) |
| - **Output:** Raw tokens with frequency, context, confidence, source type |
| |
| ### Agent 1B: Firecrawl CSS Deep Diver |
| - **Persona:** CSS Deep Diver |
| - **Tool:** Firecrawl / httpx fallback |
| - **Job:** |
| - Fetch and parse ALL linked CSS files |
| - Extract colors from CSS rules and variables |
| - Bypass CORS restrictions |
| - Find colors missed by DOM inspection |
| - **Output:** Additional colors merged into main extraction |
| |
| ### Agent 1C: Semantic Color Analyzer (NEW - LLM) |
| - **Persona:** Design System Semanticist |
| - **Tool:** Rule-based analysis (LLM optional) |
| - **Job:** |
| - Analyze colors based on actual CSS usage (not guessing) |
| - Categorize into semantic roles: |
| - **Brand Colors:** Used on buttons, CTAs, links (interactive elements) |
| - **Text Colors:** Used with `color` property on p, span, h1-h6 |
| - **Background Colors:** Used with `background-color` on containers |
| - **Border Colors:** Used with `border-color` properties |
| - **Feedback Colors:** Error (red), success (green), warning (yellow) |
| - Detect color hierarchy (primary → secondary → muted) |
| - **Input:** Colors WITH context data (css_properties, elements, frequency) |
| - **Output:** Semantic categorization with confidence levels |
| - **Why:** Stage 2 LLMs can now give SPECIFIC recommendations per role |
| |
| ### Agent 2: Token Normalizer & Structurer |
| - **Persona:** Design System Librarian |
| - **Job:** |
| - Clean noisy extraction, dedupe |
| - Infer naming patterns |
| - Tag tokens as: `detected` | `inferred` | `low-confidence` |
| - **Output:** Structured token sets with metadata |
| |
| ### Agent 3: Design System Best Practices Advisor |
| - **Persona:** Senior Staff Design Systems Architect |
| - **Job:** |
| - Research modern DS patterns (Material, Polaris, Carbon, etc.) |
| - Propose upgrade OPTIONS (not decisions) |
| - Suggest: type scales (3 options), spacing (8px), color ramps (AA compliant), naming conventions |
| - **Output:** Option sets with rationale |
| |
| ### Agent 4: Plugin & JSON Generator |
| - **Persona:** Automation Engineer |
| - **Job:** |
| - Convert finalized tokens to Figma-compatible JSON |
| - Generate: typography, color (with tints/shades), spacing variables |
| - Maintain Desktop + Mobile + version metadata |
| - **Output:** Production-ready JSON (flat structure for Figma Tokens Studio) |
| |
| --- |
| |
| ## 🖥️ UI Stages (3 Stages) |
| |
| ### Stage 1: Extraction Review (AS-IS) |
| - **Purpose:** Trust building — show exactly what was extracted |
| - **Shows:** |
| - Token tables (colors, typography, spacing) |
| - **6 Visual Preview Tabs (AS-IS, no enhancements):** |
| 1. 🔤 Typography — actual font rendered |
| 2. 🎨 Colors — simple swatches sorted by frequency (no ramps) |
| 3. 🧠 Semantic Colors — colors organized by usage (brand/text/bg/border) |
| 4. 📏 Spacing — visual bars |
| 5. 🔘 Radius — rounded boxes |
| 6. 🌑 Shadows — shadow cards |
| - **Human Actions:** Accept/reject tokens, flag anomalies, toggle Desktop↔Mobile |
| |
| ### Stage 2: Upgrade Playground (MOST IMPORTANT) |
| - **Purpose:** Decision-making through live visuals |
| - **Shows:** |
| - Side-by-side option selector + live preview |
| - **Color Ramps (50-950 shades with AA compliance)** |
| - Type scale options (1.2, 1.25, 1.333) |
| - **Semantic-aware recommendations:** "Your brand primary #06b2c4 fails AA, consider #0891a8" |
| - **Human Actions:** Select type scale A/B/C, spacing system, color ramps — preview updates instantly |
| |
| ### Stage 3: Final Review & Export |
| - **Purpose:** Confidence before export |
| - **Shows:** Token preview, JSON tree, diff view (original vs final) |
| - **Human Actions:** Download JSON, save version, label version |
| |
| --- |
| |
| ## 📁 Project Structure |
| |
| ``` |
| design-system-extractor/ |
| ├── app.py # Gradio main entry point |
| ├── requirements.txt |
| ├── README.md |
| │ |
| ├── config/ |
| │ ├── .env.example # Environment variables template |
| │ ├── agents.yaml # Agent personas & configurations |
| │ └── settings.py # Application settings |
| │ |
| ├── agents/ |
| │ ├── __init__.py |
| │ ├── state.py # LangGraph state definitions |
| │ ├── graph.py # LangGraph workflow orchestration |
| │ ├── crawler.py # Agent 1A: Website crawler |
| │ ├── extractor.py # Agent 1A: Token extraction (7 sources) |
| │ ├── firecrawl_extractor.py # Agent 1B: Deep CSS parsing |
| │ ├── semantic_analyzer.py # Agent 1C: Semantic color categorization |
| │ ├── normalizer.py # Agent 2: Token normalization |
| │ ├── advisor.py # Agent 3: Best practices |
| │ ├── stage2_graph.py # Stage 2 multi-agent LLM workflow |
| │ └── generator.py # Agent 4: JSON generator |
| │ |
| ├── core/ |
| │ ├── __init__.py |
| │ ├── color_utils.py # Color analysis, contrast, ramps |
| │ ├── preview_generator.py # HTML preview generation |
| │ ├── hf_inference.py # HuggingFace LLM inference |
| │ └── token_schema.py # Token data structures (Pydantic) |
| │ |
| ├── ui/ |
| │ └── __init__.py |
| │ |
| ├── templates/ |
| │ |
| ├── storage/ |
| │ └── __init__.py |
| │ |
| ├── tests/ |
| │ └── __init__.py |
| │ |
| └── docs/ |
| └── CONTEXT.md # THIS FILE - upload for context refresh |
| ``` |
| |
| --- |
| |
| ## 🔧 Key Technical Decisions |
| |
| | Decision | Choice | Rationale | |
| |----------|--------|-----------| |
| | Viewports | Fixed 1440px + 375px | Simplicity, covers main use cases | |
| | Scrolling | Bottom + network idle | Captures lazy-loaded content | |
| | Infinite scroll | Skip | Avoid complexity | |
| | Modals | Manual trigger | User decides what to capture | |
| | Color ramps | 5-10 shades, AA compliant | Industry standard | |
| | Type scales | 3 options (1.25, 1.333, 1.414) | User selects | |
| | Spacing | 8px base system | Modern standard | |
| | ML models | Minimal, rule-based preferred | Simplicity, reliability | |
| | Versioning | HF Spaces persistent storage | Built-in, free | |
| | Preview | Gradio + iframe (best for dynamic) | Smooth updates | |
| |
| --- |
| |
| ## 📊 Token Schema (Core Data Structures) |
| |
| ```python |
| class TokenSource(Enum): |
| DETECTED = "detected" # Directly found in CSS |
| INFERRED = "inferred" # Derived from patterns |
| UPGRADED = "upgraded" # User-selected improvement |
| |
| class Confidence(Enum): |
| HIGH = "high" # 10+ occurrences |
| MEDIUM = "medium" # 3-9 occurrences |
| LOW = "low" # 1-2 occurrences |
| |
| class Viewport(Enum): |
| DESKTOP = "desktop" # 1440px |
| MOBILE = "mobile" # 375px |
| ``` |
| |
| ### Token Types: |
| - **ColorToken:** value, frequency, contexts, elements, contrast ratios |
| - **TypographyToken:** family, size, weight, line-height, elements |
| - **SpacingToken:** value, frequency, contexts, fits_base_8 |
| - **RadiusToken:** value, frequency, elements |
| - **ShadowToken:** value, frequency, elements |
| |
| --- |
| |
| ## 🔄 LangGraph Workflow |
| |
| ``` |
| ┌─────────────┐ |
| │ START │ |
| └──────┬──────┘ |
| │ |
| ▼ |
| ┌─────────────┐ |
| │ URL Input │ |
| └──────┬──────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ Agent 1: Discover │ |
| │ (find pages) │ |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ HUMAN: Confirm pages │◄─── Checkpoint 1 |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ Agent 1: Extract │ |
| │ (crawl & extract) │ |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ Agent 2: Normalize │ |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ HUMAN: Review tokens │◄─── Checkpoint 2 (Stage 1 UI) |
| └───────────┬────────────┘ |
| │ |
| ┌───────────────┴───────────────┐ |
| │ │ |
| ▼ ▼ |
| ┌──────────────────┐ ┌──────────────────┐ |
| │ Agent 3: Advise │ │ (parallel) │ |
| │ (best practices) │ │ │ |
| └────────┬─────────┘ └──────────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ HUMAN: Select options │◄─── Checkpoint 3 (Stage 2 UI) |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ Agent 4: Generate │ |
| │ (final JSON) │ |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌────────────────────────┐ |
| │ HUMAN: Export │◄─── Checkpoint 4 (Stage 3 UI) |
| └───────────┬────────────┘ |
| │ |
| ▼ |
| ┌─────────┐ |
| │ END │ |
| └─────────┘ |
| ``` |
| |
| --- |
| |
| ## 🚦 Human-in-the-Loop Rules |
| |
| 1. **No irreversible automation** |
| 2. **Agents propose → Humans decide** |
| 3. **Every auto action must be:** |
| - Visible |
| - Reversible |
| - Previewed |
| |
| --- |
| |
| ## 📦 Output JSON Format |
| |
| ```json |
| { |
| "metadata": { |
| "source_url": "https://example.com", |
| "extracted_at": "2025-01-23T10:00:00Z", |
| "version": "v1-recovered", |
| "viewport": "desktop" |
| }, |
| "colors": { |
| "primary": { |
| "50": { "value": "#e6f2ff", "source": "upgraded" }, |
| "500": { "value": "#007bff", "source": "detected" }, |
| "900": { "value": "#001a33", "source": "upgraded" } |
| } |
| }, |
| "typography": { |
| "heading-xl": { |
| "fontFamily": "Inter", |
| "fontSize": "32px", |
| "fontWeight": 700, |
| "lineHeight": "1.2", |
| "source": "detected" |
| } |
| }, |
| "spacing": { |
| "xs": { "value": "4px", "source": "upgraded" }, |
| "sm": { "value": "8px", "source": "detected" }, |
| "md": { "value": "16px", "source": "detected" } |
| } |
| } |
| ``` |
| |
| --- |
|
|
| ## 🛠️ Implementation Phases & Current Status |
|
|
| ### Phase 1 ✅ COMPLETE |
| - [x] Project structure |
| - [x] Configuration files |
| - [x] Token schema (Pydantic models) |
| - [x] Agent 1: Crawler (page discovery) |
| - [x] Agent 1: Enhanced Extractor (5-source extraction) |
| - [x] Agent 2: Normalizer |
| - [x] Stage 1 UI with 5 AS-IS preview tabs |
| - [x] LangGraph basic workflow |
| - [x] JSON export (flat structure for Figma) |
|
|
| ### Phase 2 ✅ MOSTLY COMPLETE |
| - [x] Agent 3: Multi-LLM Advisor (Qwen + Llama + HEAD) |
| - [x] Stage 2 UI (Upgrade Playground) |
| - [x] Live preview system (typography, color ramps) |
| - [x] Enhanced LLM logging with reasoning |
| - [ ] Accept/Reject checkbox wiring to export |
|
|
| ### Phase 3 🔄 IN PROGRESS |
| - [ ] Agent 4: Generator (component patterns) |
| - [ ] Stage 3 UI (diff view) |
| - [ ] Arabic page filtering |
|
|
| ### Phase 4 ⏳ PENDING |
| - [ ] Full LangGraph orchestration |
| - [ ] HF Spaces deployment |
| - [ ] Persistent storage |
| - [ ] MCP Claude / Figma plugin integration (Part 2 of article) |
|
|
| --- |
|
|
| ## 🐛 Known Issues & Pending Fixes |
|
|
| | Issue | Status | Fix | |
| |-------|--------|-----| |
| | Arabic pages included | Pending | Filter `/ar/` URLs in crawler | |
| | Accept/Reject not wired | Pending | Export should respect checkbox state | |
| | Stage 1 vs Stage 2 preview confusion | ✅ Fixed | Stage 1 now shows AS-IS (no ramps) | |
| | Colors missed from CSS variables | ✅ Fixed | Enhanced 5-source extraction | |
| | JSON nested structure | ✅ Fixed | Flat structure for Figma compatibility | |
|
|
| --- |
|
|
| ## 🔑 Environment Variables |
|
|
| ```env |
| # Required |
| HF_TOKEN=your_huggingface_token |
| |
| # Model Configuration (defaults shown — diverse providers) |
| AGENT2_MODEL=microsoft/Phi-3.5-mini-instruct # Microsoft - Fast naming |
| AGENT3_MODEL=meta-llama/Llama-3.1-70B-Instruct # Meta - Strong reasoning |
| AGENT4_MODEL=mistralai/Codestral-22B-v0.1 # Mistral - Code/JSON |
| |
| # Optional |
| DEBUG=true |
| LOG_LEVEL=INFO |
| ``` |
|
|
| --- |
|
|
| ## 📝 Notes for Claude |
|
|
| When continuing this project: |
| 1. **Check current phase** in Implementation Phases section |
| 2. **Review agent personas** in agents.yaml for consistent behavior |
| 3. **Follow token schema** defined in core/token_schema.py |
| 4. **Maintain LangGraph state** consistency across agents |
| 5. **Use Gradio components** from ui/components.py for consistency |
| 6. **Test with** real websites before deployment |
| 7. **Enhanced extraction** captures from 5 sources — check logs to verify |
| 8. **Stage 1 = AS-IS** (no ramps), **Stage 2 = Enhanced** (with ramps) |
| |
| --- |
| |
| *Last updated: 2025-01-23* |
| |
