Aryan Mishra commited on
Commit
1e71331
Β·
1 Parent(s): 64a410d

Phase 6: Final Cleanup and Python-First Migration Complete

Browse files
Files changed (33) hide show
  1. .DS_Store +0 -0
  2. README.md +6 -9
  3. api/app/main.py +5 -21
  4. {dashboard β†’ dashboard_backup}/.env.example +0 -0
  5. {dashboard β†’ dashboard_backup}/dist/_redirects +0 -0
  6. {dashboard β†’ dashboard_backup}/dist/assets/index-DGl1EjcM.css +0 -0
  7. {dashboard β†’ dashboard_backup}/dist/assets/index-w96O4ZPi.js +0 -0
  8. {dashboard β†’ dashboard_backup}/dist/index.html +0 -0
  9. {dashboard β†’ dashboard_backup}/index.html +0 -0
  10. {dashboard β†’ dashboard_backup}/package-lock.json +0 -0
  11. {dashboard β†’ dashboard_backup}/package.json +0 -0
  12. {dashboard β†’ dashboard_backup}/postcss.config.js +0 -0
  13. {dashboard β†’ dashboard_backup}/public/_redirects +0 -0
  14. {dashboard β†’ dashboard_backup}/src/App.jsx +0 -0
  15. {dashboard β†’ dashboard_backup}/src/api/client.js +0 -0
  16. {dashboard β†’ dashboard_backup}/src/components/AspectHeatmap.jsx +0 -0
  17. {dashboard β†’ dashboard_backup}/src/components/LanguagePie.jsx +0 -0
  18. {dashboard β†’ dashboard_backup}/src/components/Layout.jsx +0 -0
  19. {dashboard β†’ dashboard_backup}/src/components/LivePredictor.jsx +0 -0
  20. {dashboard β†’ dashboard_backup}/src/components/SentimentChart.jsx +0 -0
  21. {dashboard β†’ dashboard_backup}/src/components/Sidebar.jsx +0 -0
  22. {dashboard β†’ dashboard_backup}/src/config.js +0 -0
  23. {dashboard β†’ dashboard_backup}/src/index.css +0 -0
  24. {dashboard β†’ dashboard_backup}/src/main.jsx +0 -0
  25. {dashboard β†’ dashboard_backup}/src/pages/Analytics.jsx +0 -0
  26. {dashboard β†’ dashboard_backup}/src/pages/Monitor.jsx +0 -0
  27. {dashboard β†’ dashboard_backup}/src/pages/Predict.jsx +0 -0
  28. {dashboard β†’ dashboard_backup}/tailwind.config.js +0 -0
  29. {dashboard β†’ dashboard_backup}/vercel.json +0 -0
  30. {dashboard β†’ dashboard_backup}/vite.config.js +0 -0
  31. docs/DEPLOYMENT.md +2 -8
  32. docs/SECURITY.md +1 -1
  33. docs/architecture.md +3 -3
.DS_Store CHANGED
Binary files a/.DS_Store and b/.DS_Store differ
 
README.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
4
  ![FastAPI](https://img.shields.io/badge/FastAPI-0.111.0-00a393.svg)
5
- ![React](https://img.shields.io/badge/React-18-61dafb.svg)
6
  ![DVC](https://img.shields.io/badge/DVC-3.51.1-945dd6.svg)
7
  ![MLflow](https://img.shields.io/badge/MLflow-2.13.0-0194E2.svg)
8
 
@@ -18,7 +18,7 @@ The system leverages state-of-the-art models like **XLM-RoBERTa** and **IndicBER
18
  - **Rule-Based Fallback**: An instantaneous, pure-Python fallback leveraging a curated multi-lingual lexicon to handle aspect extraction and sentiment scoring without any heavy downloads.
19
  - **Modern Tech Stack**:
20
  - **Backend**: Asynchronous, high-performance API built with FastAPI.
21
- - **Frontend**: A responsive dashboard built with React and TailwindCSS. Features real-time predictions, batch analytics, and system monitoring.
22
  - **MLOps Integrated**: Complete integration with DVC (Data Version Control) for pipeline reproducibility, MLflow for experiment tracking, and Evidently AI for data drift monitoring.
23
  - **Scalable Architecture**: Support for async tasks via Celery + Redis, robust data storage via PostgreSQL, and metric exporting using Prometheus.
24
 
@@ -26,7 +26,7 @@ The system leverages state-of-the-art models like **XLM-RoBERTa** and **IndicBER
26
 
27
  This repository is designed following a **Python-first paradigm**:
28
  - **Python (67.5%)**: Handling all business logic, data processing, configuration, API routing, ML inference, and utility functions using FastAPI and Python data science libraries.
29
- - **JavaScript/TypeScript (32.5%)**: Strictly limited to the frontend `dashboard/` directory, used *only* for the React UI, component rendering, browser events, and client-side state.
30
  - *Note: There is no backend or ML logic written in JavaScript.*
31
 
32
  ## Repository Structure
@@ -35,7 +35,7 @@ This repository is designed following a **Python-first paradigm**:
35
  Multilingual-Absa/
36
  β”œβ”€β”€ api/app/ # FastAPI backend and inference services
37
  β”œβ”€β”€ config/ # Docker and application configuration files
38
- β”œβ”€β”€ dashboard/ # React frontend for inference & monitoring
39
  β”œβ”€β”€ data/ # Dataset directory (DVC-tracked)
40
  β”œβ”€β”€ docs/ # Extended documentation (architecture, ml, api)
41
  β”œβ”€β”€ ml/ # ML training, notebooks, MLflow, and tracking
@@ -52,7 +52,7 @@ Multilingual-Absa/
52
 
53
  ### Prerequisites
54
  - Python 3.10+
55
- - Node.js (for Dashboard)
56
  - Docker & Docker Compose (Optional, but recommended)
57
 
58
  ### 1. Local Setup
@@ -100,11 +100,8 @@ PYTHONPATH=. uvicorn api.app.main:app --reload --host 0.0.0.0 --port 8000
100
 
101
  **Start the Dashboard:**
102
  ```bash
103
- cd dashboard
104
- npm install
105
- npm run dev
106
  ```
107
- *Access the dashboard at `http://localhost:5173`.*
108
 
109
  ## How it Works
110
 
 
2
 
3
  ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
4
  ![FastAPI](https://img.shields.io/badge/FastAPI-0.111.0-00a393.svg)
5
+ ![React](https://img.shields.io/badge/HTMX-1.9-3d72d4.svg)
6
  ![DVC](https://img.shields.io/badge/DVC-3.51.1-945dd6.svg)
7
  ![MLflow](https://img.shields.io/badge/MLflow-2.13.0-0194E2.svg)
8
 
 
18
  - **Rule-Based Fallback**: An instantaneous, pure-Python fallback leveraging a curated multi-lingual lexicon to handle aspect extraction and sentiment scoring without any heavy downloads.
19
  - **Modern Tech Stack**:
20
  - **Backend**: Asynchronous, high-performance API built with FastAPI.
21
+ - **Frontend**: A responsive dashboard built with FastAPI, Jinja2, HTMX, and TailwindCSS. Features real-time predictions, batch analytics, and system monitoring.
22
  - **MLOps Integrated**: Complete integration with DVC (Data Version Control) for pipeline reproducibility, MLflow for experiment tracking, and Evidently AI for data drift monitoring.
23
  - **Scalable Architecture**: Support for async tasks via Celery + Redis, robust data storage via PostgreSQL, and metric exporting using Prometheus.
24
 
 
26
 
27
  This repository is designed following a **Python-first paradigm**:
28
  - **Python (67.5%)**: Handling all business logic, data processing, configuration, API routing, ML inference, and utility functions using FastAPI and Python data science libraries.
29
+ - **JavaScript/TypeScript (32.5%)**: Strictly limited to the frontend `dashboard/` directory, used *only* for the Alpine.js for minimal client-side state.
30
  - *Note: There is no backend or ML logic written in JavaScript.*
31
 
32
  ## Repository Structure
 
35
  Multilingual-Absa/
36
  β”œβ”€β”€ api/app/ # FastAPI backend and inference services
37
  β”œβ”€β”€ config/ # Docker and application configuration files
38
+ β”œβ”€β”€ api/app/templates/ # Jinja2 HTMX frontend templates
39
  β”œβ”€β”€ data/ # Dataset directory (DVC-tracked)
40
  β”œβ”€β”€ docs/ # Extended documentation (architecture, ml, api)
41
  β”œβ”€β”€ ml/ # ML training, notebooks, MLflow, and tracking
 
52
 
53
  ### Prerequisites
54
  - Python 3.10+
55
+
56
  - Docker & Docker Compose (Optional, but recommended)
57
 
58
  ### 1. Local Setup
 
100
 
101
  **Start the Dashboard:**
102
  ```bash
 
 
 
103
  ```
104
+ *Access the dashboard at `http://localhost:8000/predict`.*
105
 
106
  ## How it Works
107
 
api/app/main.py CHANGED
@@ -1,7 +1,7 @@
1
  from fastapi import FastAPI
2
  from contextlib import asynccontextmanager
3
  from dotenv import load_dotenv
4
- from fastapi.middleware.cors import CORSMiddleware
5
  from fastapi.staticfiles import StaticFiles # Phase 2: serves api/app/static/
6
  from pathlib import Path # Phase 2: resolve static directory path
7
 
@@ -36,32 +36,16 @@ app = FastAPI(
36
  lifespan=lifespan,
37
  )
38
 
39
- # Allow the dashboard (and any origin in dev) to call the API
40
- app.add_middleware(
41
- CORSMiddleware,
42
- allow_origins=["http://localhost:3000", "http://127.0.0.1:3000", "*"],
43
- allow_credentials=True,
44
- allow_methods=["*"],
45
- allow_headers=["*"],
46
- )
47
 
48
  app.include_router(predict.router, tags=["Predict"])
49
  app.include_router(results.router, tags=["System"])
50
 
51
  instrumentator.instrument(app).expose(app, endpoint="/metrics")
52
 
53
- # ── Phase 2: Jinja2 / HTMX frontend ──────────────────────────────────────────
54
- #
55
- # WHY pages router is included AFTER instrumentator:
56
- # The instrumentator middleware wraps the entire ASGI app; order of
57
- # router inclusion does not affect which routes get instrumented. Including
58
- # the pages router last is simply a readability convention β€” API routes first.
59
- #
60
- # WHY StaticFiles is mounted AFTER instrumentator:
61
- # app.mount() creates a sub-application. Mounting after the instrumentator
62
- # call means the Prometheus middleware still wraps /static/* requests, but
63
- # since static files are not hot paths for an ML tool this is acceptable.
64
- # (Phase 7 cleanup will add /static to excluded_handlers in metrics.py.)
65
 
66
  app.include_router(pages.router) # include_in_schema=False is set on the router itself
67
 
 
1
  from fastapi import FastAPI
2
  from contextlib import asynccontextmanager
3
  from dotenv import load_dotenv
4
+
5
  from fastapi.staticfiles import StaticFiles # Phase 2: serves api/app/static/
6
  from pathlib import Path # Phase 2: resolve static directory path
7
 
 
36
  lifespan=lifespan,
37
  )
38
 
39
+
 
 
 
 
 
 
 
40
 
41
  app.include_router(predict.router, tags=["Predict"])
42
  app.include_router(results.router, tags=["System"])
43
 
44
  instrumentator.instrument(app).expose(app, endpoint="/metrics")
45
 
46
+ # Jinja2 / HTMX frontend routing and static files
47
+ # Note: StaticFiles is mounted after instrumentator so prometheus
48
+ # ignores it for /metrics, although this might still log /static requests.
 
 
 
 
 
 
 
 
 
49
 
50
  app.include_router(pages.router) # include_in_schema=False is set on the router itself
51
 
{dashboard β†’ dashboard_backup}/.env.example RENAMED
File without changes
{dashboard β†’ dashboard_backup}/dist/_redirects RENAMED
File without changes
{dashboard β†’ dashboard_backup}/dist/assets/index-DGl1EjcM.css RENAMED
File without changes
{dashboard β†’ dashboard_backup}/dist/assets/index-w96O4ZPi.js RENAMED
File without changes
{dashboard β†’ dashboard_backup}/dist/index.html RENAMED
File without changes
{dashboard β†’ dashboard_backup}/index.html RENAMED
File without changes
{dashboard β†’ dashboard_backup}/package-lock.json RENAMED
File without changes
{dashboard β†’ dashboard_backup}/package.json RENAMED
File without changes
{dashboard β†’ dashboard_backup}/postcss.config.js RENAMED
File without changes
{dashboard β†’ dashboard_backup}/public/_redirects RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/App.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/api/client.js RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/AspectHeatmap.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/LanguagePie.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/Layout.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/LivePredictor.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/SentimentChart.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/components/Sidebar.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/config.js RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/index.css RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/main.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/pages/Analytics.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/pages/Monitor.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/src/pages/Predict.jsx RENAMED
File without changes
{dashboard β†’ dashboard_backup}/tailwind.config.js RENAMED
File without changes
{dashboard β†’ dashboard_backup}/vercel.json RENAMED
File without changes
{dashboard β†’ dashboard_backup}/vite.config.js RENAMED
File without changes
docs/DEPLOYMENT.md CHANGED
@@ -3,10 +3,10 @@
3
  ## Prerequisites
4
 
5
  - Python 3.11+
6
- - Node.js 20+
7
  - Docker & Docker Compose (for containerized deployment)
8
  - Railway account (for API deployment)
9
- - Vercel account (for dashboard deployment)
10
 
11
  ## Environment Variables
12
 
@@ -36,11 +36,6 @@ uvicorn api.main:app --reload --host 0.0.0.0 --port 8000
36
  # MLflow UI at http://localhost:5000
37
 
38
  # Frontend
39
- cd dashboard
40
- cp .env.example .env # VITE_API_URL=http://localhost:8000
41
- npm install
42
- npm run dev
43
- # Dashboard at http://localhost:5173
44
  ```
45
 
46
  ## Docker Compose (Full Stack)
@@ -90,7 +85,6 @@ graph TB
90
 
91
  ### Vercel (Dashboard)
92
 
93
- 1. Import `dashboard/` as a Vercel project
94
  2. Framework preset: Vite
95
  3. Environment variable: `VITE_API_URL=https://your-railway-api-url.railway.app`
96
  4. `vercel.json` rewrites `/api/*` to Railway API
 
3
  ## Prerequisites
4
 
5
  - Python 3.11+
6
+ -
7
  - Docker & Docker Compose (for containerized deployment)
8
  - Railway account (for API deployment)
9
+ -
10
 
11
  ## Environment Variables
12
 
 
36
  # MLflow UI at http://localhost:5000
37
 
38
  # Frontend
 
 
 
 
 
39
  ```
40
 
41
  ## Docker Compose (Full Stack)
 
85
 
86
  ### Vercel (Dashboard)
87
 
 
88
  2. Framework preset: Vite
89
  3. Environment variable: `VITE_API_URL=https://your-railway-api-url.railway.app`
90
  4. `vercel.json` rewrites `/api/*` to Railway API
docs/SECURITY.md CHANGED
@@ -9,7 +9,7 @@
9
  | HTTPS | ❌ Not enforced | Expects reverse proxy to terminate TLS |
10
  | Input Validation | βœ… Partial | Pydantic validation present; no server-side max_length |
11
  | SQL Injection | βœ… Protected | SQLAlchemy ORM parameterized queries |
12
- | XSS | βœ… Protected | React JSX auto-escaping |
13
  | CSRF | ❌ Not implemented | No CSRF middleware; CORS `"*"` mitigates partially |
14
  | Secrets Management | ⚠️ Manual | `.env` gitignored; Docker Compose has hardcoded dev creds |
15
  | Rate Limiting | ❌ Not implemented | No throttling on any endpoint |
 
9
  | HTTPS | ❌ Not enforced | Expects reverse proxy to terminate TLS |
10
  | Input Validation | βœ… Partial | Pydantic validation present; no server-side max_length |
11
  | SQL Injection | βœ… Protected | SQLAlchemy ORM parameterized queries |
12
+ | XSS | βœ… Protected | Jinja2 auto-escaping |
13
  | CSRF | ❌ Not implemented | No CSRF middleware; CORS `"*"` mitigates partially |
14
  | Secrets Management | ⚠️ Manual | `.env` gitignored; Docker Compose has hardcoded dev creds |
15
  | Rate Limiting | ❌ Not implemented | No throttling on any endpoint |
docs/architecture.md CHANGED
@@ -43,10 +43,10 @@ graph TB
43
  ```mermaid
44
  graph LR
45
  subgraph "Presentation Layer"
46
- SPA[React SPA]
47
  T_TAIL[TailwindCSS Theme]
48
  RECH[Recharts Visualizations]
49
- RQ[React Query]
50
  end
51
  subgraph "API Layer"
52
  FAST[FastAPI]
@@ -111,7 +111,7 @@ graph TB
111
  RW_REDIS[Redis]
112
  end
113
  subgraph "Vercel (Production)"
114
- VC_DASH[React SPA]
115
  VC_RW_ROUTE[rewrite /api/* -> Railway]
116
  end
117
 
 
43
  ```mermaid
44
  graph LR
45
  subgraph "Presentation Layer"
46
+ SPA[HTMX Frontend]
47
  T_TAIL[TailwindCSS Theme]
48
  RECH[Recharts Visualizations]
49
+ RQ[HTMX Polling]
50
  end
51
  subgraph "API Layer"
52
  FAST[FastAPI]
 
111
  RW_REDIS[Redis]
112
  end
113
  subgraph "Vercel (Production)"
114
+ VC_DASH[HTMX Frontend]
115
  VC_RW_ROUTE[rewrite /api/* -> Railway]
116
  end
117