Genesis AI — Complete System Flowchart

End-to-end flow: Excel upload → Python pipeline → JSON outputs → React dashboard

Master Flow — Upload to Dashboard
flowchart TD A([" 👤 User\nOpens Dashboard "]) --> B subgraph FE [" REACT FRONTEND (localhost:5173) "] B["Header.jsx\nShows file badge\n+ Upload button"] B --> C["UploadModal.jsx\nUser picks .xlsx file"] C --> D["POST /api/upload\nfetch() with FormData"] D2["DataContext.jsx\nfetchKey() per tab"] D3["Dashboard tabs render\nResults / PPA / Market / Recs …"] D2 --> D3 end subgraph BE [" FASTAPI BACKEND (localhost:8000) "] E["upload_server.py\n@app.post('/api/upload')\nSaves .xlsx to _uploads/"] F["background thread\n_run_pipeline()"] E --> F G["GET /api/data/{key}\n@app.get('/api/data/{key}')\nReads output/{key}.json"] end subgraph PY [" PYTHON PIPELINE (pipeline/) "] P1["loader.py\nload_data()\nRead 1_Source_Data sheet\ncompute Price = Val/Vol"] P2["run.py\nread_brand_config()\nFocal brand + competitors\nfrom 2_Brand_Config sheet"] P3["modelling.py\nbuild_wide_pivot()\nOne table per CH×RG grain"] P4["diagnostics.py\nrun_diagnostics()\nCorrelation, CV, RPI checks"] P5["modelling.py\nrun_elasticity_models()\nOLS regression per grain\nBest Adj-R² spec selected"] P6["proxies.py\nassign_proxies()\nWrong-sign grains get\nborrowed elasticity"] P7["exporters/stats.py\nexporters/market.py\nexporters/ppa.py\nexporters/analytics.py\nexporters/recommendations.py"] P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> P7 end subgraph OUT [" OUTPUT FILES (output/) "] J1["models.json\nstats.json\nfreq_anchors.json"] J2["trend.json · ms.json\nvol_salience.json\nval_share.json\ncomp_ms.json · vtm.json"] J3["ppa_mt.json\nppa_tt.json"] J4["interaction.json\ngrowth_decomp.json\npgi.json"] J5["recs_full.json"] end D --> E F --> P1 P7 --> J1 P7 --> J2 P7 --> J3 P7 --> J4 P7 --> J5 J1 & J2 & J3 & J4 & J5 --> G G --> D2 style FE fill:#e3f2fd,stroke:#1565c0,color:#0d1b4b style BE fill:#fff3e0,stroke:#c84b00,color:#3e2000 style PY fill:#e8f5e9,stroke:#2e7d32,color:#1b2e1b style OUT fill:#f3e5f5,stroke:#7b1fa2,color:#2e0040
React Frontend
FastAPI Backend
Python Pipeline
JSON Outputs
API Sequence — Upload → Poll → Fetch Data
sequenceDiagram participant UI as React (browser) participant API as FastAPI :8000 participant PL as Pipeline (subprocess) participant FS as output/*.json UI->>API: POST /api/upload (FormData: file.xlsx) API-->>UI: 200 { ok:true, filename } Note over API: background thread starts loop every 2 s while status == "running" UI->>API: GET /api/upload/status API-->>UI: { status, message, elapsed_s } end Note over PL: loader → config → pivot → diag → OLS → proxies → export API->>PL: subprocess python -m pipeline.run --input file.xlsx PL-->>FS: writes 16 × *.json to output/ API-->>UI: status = "done" loop for each of 16 data keys UI->>API: GET /api/data/{key} API->>FS: reads output/{key}.json FS-->>API: JSON bytes API-->>UI: JSON response end Note over UI: Dashboard tabs render with fresh data
Pipeline Steps — Detailed Breakdown
flowchart LR classDef step fill:#e8f5e9,stroke:#2e7d32,color:#1b2e1b,rx:8 classDef json fill:#f3e5f5,stroke:#7b1fa2,color:#2e0040,rx:8 classDef file fill:#fff3e0,stroke:#c84b00,color:#3e2000,rx:4 S1["STEP 1\nloader.py\nload_data()"]:::step S2["STEP 2\nrun.py\nread_brand_config()"]:::step S3["STEP 3\nmodelling.py\nbuild_wide_pivot()"]:::step S4["STEP 4\ndiagnostics.py\nrun_diagnostics()"]:::step S5["STEP 5\nmodelling.py\nrun_elasticity_models()"]:::step S6["STEP 6\nproxies.py\nassign_proxies()"]:::step E1["stats.py\nexport_stats()"]:::step E2["market.py\nexport_market()"]:::step E3["ppa.py\nexport_ppa()"]:::step E4["analytics.py\nexport_analytics()"]:::step E5["recommendations.py\nexport_recs()"]:::step J1["models.json\nstats.json\nfreq_anchors.json"]:::json J2["trend.json\nms.json\nvol_salience.json\nval_share.json\ncomp_ms.json\nvtm.json"]:::json J3["ppa_mt.json\nppa_tt.json"]:::json J4["interaction.json\ngrowth_decomp.json\npgi.json"]:::json J5["recs_full.json"]:::json S1 --> S2 --> S3 --> S4 --> S5 --> S6 S6 --> E1 & E2 & E3 & E4 & E5 E1 --> J1 E2 --> J2 E3 --> J3 E4 --> J4 E5 --> J5
Step-by-Step Reference Table
Step Layer File Function / Endpoint What it does Output
1 FE Header.jsx Upload button click Opens UploadModal; shows active filename badge from localStorage —
2 FE UploadModal.jsx handleUpload() Sends POST /api/upload with FormData; polls /api/upload/status every 2 s Filename stored in localStorage
3 BE upload_server.py POST /api/upload Validates extension, saves .xlsx to _uploads/, spawns background thread { ok: true, filename }
4 BE upload_server.py _run_pipeline() Calls python -m pipeline.run --input file.xlsx --output output/ via subprocess.Popen (streams live) Terminal log
5 PY pipeline/loader.py load_data() Reads sheet 1_Source_Data; computes Price = Value / Volume; drops nulls DataFrame
6 PY pipeline/run.py read_brand_config() Reads sheet 2_Brand_Config; extracts focal brand + competitors brand config dict
7 PY pipeline/modelling.py build_wide_pivot() One wide table per Channel × Region grain with focal + competitor columns wide DataFrame
8 PY pipeline/diagnostics.py run_diagnostics() Pearson correlation, CV, RPI checks per grain; writes Excel diagnostics file *_Diagnostics.xlsx
9 PY pipeline/modelling.py run_elasticity_models() OLS regression on all spec combos; selects best Adj-R² with correct sign; clamps to [−6, 0] results DataFrame
10 PY pipeline/proxies.py assign_proxies() Wrong-sign grains borrow elasticity from adjacent/nearby grains final elasticities
11 PY pipeline/exporters/stats.py export_stats() Writes elasticity results + brand KPIs models.json, stats.json, freq_anchors.json
12 PY pipeline/exporters/market.py export_market() Writes time-series, market share, volume/value share trend.json, ms.json, vol_salience.json, val_share.json, comp_ms.json, vtm.json
13 PY pipeline/exporters/ppa.py export_ppa() Reads sheets 3_PPA_MT and 4_PPA_TT from original xlsx ppa_mt.json, ppa_tt.json
14 PY pipeline/exporters/analytics.py export_analytics() Cross-brand correlations, growth decomposition, price gradient index interaction.json, growth_decomp.json, pgi.json
15 PY pipeline/exporters/recommendations.py export_recs() Builds pricing recommendation cards per pack using elasticities recs_full.json
16 BE upload_server.py GET /api/data/{key} Reads output/{key}.json and returns as JSON response to React JSON to browser
17 FE DataContext.jsx fetchKey() Tries /api/data/{key} first; falls back to /data/{key}.json (static) React state updated
18 FE src/components/tabs/ All tab components Read from DataContext and render charts, tables, recommendation cards Dashboard UI
Data Serving — Two-Tier Fallback
flowchart TD A["React: fetchKey(key)"] --> B{"FastAPI server\nreachable?"} B -- Yes --> C["GET /api/data/{key}\nlocalhost:8000"] B -- No --> D["GET /data/{key}.json\nVite static (public/data/)"] C -- "200 OK" --> E["Use API response\n(always fresh)"] C -- "404 / error" --> D D -- "200 OK" --> F["Use static file\n(last pipeline run)"] D -- "404 / error" --> G["null — tab shows empty state"] style E fill:#e8f5e9,stroke:#2e7d32 style F fill:#fff3e0,stroke:#c84b00 style G fill:#fce4ec,stroke:#b71c1c
Output File Map — JSON Key → Dashboard Tab
JSON file API key Pipeline exporter Dashboard tab / component
models.jsonmodelsstats.pyResults — elasticity table
stats.jsonstatsstats.pyHeader KPIs, brand summary
freq_anchors.jsonfreq_anchorsstats.pyResults — frequency anchors
trend.jsontrendmarket.pyTrends tab — time-series chart
ms.jsonmsmarket.pyMarket Share tab
vol_salience.jsonvol_saliencemarket.pyGrowth tab — pack salience
val_share.jsonval_sharemarket.pyGrowth tab — value share
comp_ms.jsoncomp_msmarket.pyMarket Share — all brands
vtm.jsonvtmmarket.pyResults — volume-to-market
ppa_mt.jsonppa_mtppa.pyPPA tab — Modern Trade
ppa_tt.jsonppa_ttppa.pyPPA tab — Traditional Trade
interaction.jsoninteractionanalytics.pyInteraction tab — correlation matrix
growth_decomp.jsongrowth_decompanalytics.pyGrowth tab — decomposition chart
pgi.jsonpgianalytics.pyMethodology tab — price gradient
recs_full.jsonrecs / recs_fullrecommendations.pyRecommendations tab — cards