Spaces:
Runtime error
Runtime error
Aryan Mishra commited on
Commit Β·
1e71331
1
Parent(s): 64a410d
Phase 6: Final Cleanup and Python-First Migration Complete
Browse files- .DS_Store +0 -0
- README.md +6 -9
- api/app/main.py +5 -21
- {dashboard β dashboard_backup}/.env.example +0 -0
- {dashboard β dashboard_backup}/dist/_redirects +0 -0
- {dashboard β dashboard_backup}/dist/assets/index-DGl1EjcM.css +0 -0
- {dashboard β dashboard_backup}/dist/assets/index-w96O4ZPi.js +0 -0
- {dashboard β dashboard_backup}/dist/index.html +0 -0
- {dashboard β dashboard_backup}/index.html +0 -0
- {dashboard β dashboard_backup}/package-lock.json +0 -0
- {dashboard β dashboard_backup}/package.json +0 -0
- {dashboard β dashboard_backup}/postcss.config.js +0 -0
- {dashboard β dashboard_backup}/public/_redirects +0 -0
- {dashboard β dashboard_backup}/src/App.jsx +0 -0
- {dashboard β dashboard_backup}/src/api/client.js +0 -0
- {dashboard β dashboard_backup}/src/components/AspectHeatmap.jsx +0 -0
- {dashboard β dashboard_backup}/src/components/LanguagePie.jsx +0 -0
- {dashboard β dashboard_backup}/src/components/Layout.jsx +0 -0
- {dashboard β dashboard_backup}/src/components/LivePredictor.jsx +0 -0
- {dashboard β dashboard_backup}/src/components/SentimentChart.jsx +0 -0
- {dashboard β dashboard_backup}/src/components/Sidebar.jsx +0 -0
- {dashboard β dashboard_backup}/src/config.js +0 -0
- {dashboard β dashboard_backup}/src/index.css +0 -0
- {dashboard β dashboard_backup}/src/main.jsx +0 -0
- {dashboard β dashboard_backup}/src/pages/Analytics.jsx +0 -0
- {dashboard β dashboard_backup}/src/pages/Monitor.jsx +0 -0
- {dashboard β dashboard_backup}/src/pages/Predict.jsx +0 -0
- {dashboard β dashboard_backup}/tailwind.config.js +0 -0
- {dashboard β dashboard_backup}/vercel.json +0 -0
- {dashboard β dashboard_backup}/vite.config.js +0 -0
- docs/DEPLOYMENT.md +2 -8
- docs/SECURITY.md +1 -1
- docs/architecture.md +3 -3
.DS_Store
CHANGED
|
Binary files a/.DS_Store and b/.DS_Store differ
|
|
|
README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |

|
| 4 |

|
| 5 |
-

|
| 7 |

|
| 8 |
|
|
@@ -18,7 +18,7 @@ The system leverages state-of-the-art models like **XLM-RoBERTa** and **IndicBER
|
|
| 18 |
- **Rule-Based Fallback**: An instantaneous, pure-Python fallback leveraging a curated multi-lingual lexicon to handle aspect extraction and sentiment scoring without any heavy downloads.
|
| 19 |
- **Modern Tech Stack**:
|
| 20 |
- **Backend**: Asynchronous, high-performance API built with FastAPI.
|
| 21 |
-
- **Frontend**: A responsive dashboard built with
|
| 22 |
- **MLOps Integrated**: Complete integration with DVC (Data Version Control) for pipeline reproducibility, MLflow for experiment tracking, and Evidently AI for data drift monitoring.
|
| 23 |
- **Scalable Architecture**: Support for async tasks via Celery + Redis, robust data storage via PostgreSQL, and metric exporting using Prometheus.
|
| 24 |
|
|
@@ -26,7 +26,7 @@ The system leverages state-of-the-art models like **XLM-RoBERTa** and **IndicBER
|
|
| 26 |
|
| 27 |
This repository is designed following a **Python-first paradigm**:
|
| 28 |
- **Python (67.5%)**: Handling all business logic, data processing, configuration, API routing, ML inference, and utility functions using FastAPI and Python data science libraries.
|
| 29 |
-
- **JavaScript/TypeScript (32.5%)**: Strictly limited to the frontend `dashboard/` directory, used *only* for the
|
| 30 |
- *Note: There is no backend or ML logic written in JavaScript.*
|
| 31 |
|
| 32 |
## Repository Structure
|
|
@@ -35,7 +35,7 @@ This repository is designed following a **Python-first paradigm**:
|
|
| 35 |
Multilingual-Absa/
|
| 36 |
βββ api/app/ # FastAPI backend and inference services
|
| 37 |
βββ config/ # Docker and application configuration files
|
| 38 |
-
βββ
|
| 39 |
βββ data/ # Dataset directory (DVC-tracked)
|
| 40 |
βββ docs/ # Extended documentation (architecture, ml, api)
|
| 41 |
βββ ml/ # ML training, notebooks, MLflow, and tracking
|
|
@@ -52,7 +52,7 @@ Multilingual-Absa/
|
|
| 52 |
|
| 53 |
### Prerequisites
|
| 54 |
- Python 3.10+
|
| 55 |
-
|
| 56 |
- Docker & Docker Compose (Optional, but recommended)
|
| 57 |
|
| 58 |
### 1. Local Setup
|
|
@@ -100,11 +100,8 @@ PYTHONPATH=. uvicorn api.app.main:app --reload --host 0.0.0.0 --port 8000
|
|
| 100 |
|
| 101 |
**Start the Dashboard:**
|
| 102 |
```bash
|
| 103 |
-
cd dashboard
|
| 104 |
-
npm install
|
| 105 |
-
npm run dev
|
| 106 |
```
|
| 107 |
-
*Access the dashboard at `http://localhost:
|
| 108 |
|
| 109 |
## How it Works
|
| 110 |
|
|
|
|
| 2 |
|
| 3 |

|
| 4 |

|
| 5 |
+

|
| 6 |

|
| 7 |

|
| 8 |
|
|
|
|
| 18 |
- **Rule-Based Fallback**: An instantaneous, pure-Python fallback leveraging a curated multi-lingual lexicon to handle aspect extraction and sentiment scoring without any heavy downloads.
|
| 19 |
- **Modern Tech Stack**:
|
| 20 |
- **Backend**: Asynchronous, high-performance API built with FastAPI.
|
| 21 |
+
- **Frontend**: A responsive dashboard built with FastAPI, Jinja2, HTMX, and TailwindCSS. Features real-time predictions, batch analytics, and system monitoring.
|
| 22 |
- **MLOps Integrated**: Complete integration with DVC (Data Version Control) for pipeline reproducibility, MLflow for experiment tracking, and Evidently AI for data drift monitoring.
|
| 23 |
- **Scalable Architecture**: Support for async tasks via Celery + Redis, robust data storage via PostgreSQL, and metric exporting using Prometheus.
|
| 24 |
|
|
|
|
| 26 |
|
| 27 |
This repository is designed following a **Python-first paradigm**:
|
| 28 |
- **Python (67.5%)**: Handling all business logic, data processing, configuration, API routing, ML inference, and utility functions using FastAPI and Python data science libraries.
|
| 29 |
+
- **JavaScript/TypeScript (32.5%)**: Strictly limited to the frontend `dashboard/` directory, used *only* for the Alpine.js for minimal client-side state.
|
| 30 |
- *Note: There is no backend or ML logic written in JavaScript.*
|
| 31 |
|
| 32 |
## Repository Structure
|
|
|
|
| 35 |
Multilingual-Absa/
|
| 36 |
βββ api/app/ # FastAPI backend and inference services
|
| 37 |
βββ config/ # Docker and application configuration files
|
| 38 |
+
βββ api/app/templates/ # Jinja2 HTMX frontend templates
|
| 39 |
βββ data/ # Dataset directory (DVC-tracked)
|
| 40 |
βββ docs/ # Extended documentation (architecture, ml, api)
|
| 41 |
βββ ml/ # ML training, notebooks, MLflow, and tracking
|
|
|
|
| 52 |
|
| 53 |
### Prerequisites
|
| 54 |
- Python 3.10+
|
| 55 |
+
|
| 56 |
- Docker & Docker Compose (Optional, but recommended)
|
| 57 |
|
| 58 |
### 1. Local Setup
|
|
|
|
| 100 |
|
| 101 |
**Start the Dashboard:**
|
| 102 |
```bash
|
|
|
|
|
|
|
|
|
|
| 103 |
```
|
| 104 |
+
*Access the dashboard at `http://localhost:8000/predict`.*
|
| 105 |
|
| 106 |
## How it Works
|
| 107 |
|
api/app/main.py
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
from fastapi import FastAPI
|
| 2 |
from contextlib import asynccontextmanager
|
| 3 |
from dotenv import load_dotenv
|
| 4 |
-
|
| 5 |
from fastapi.staticfiles import StaticFiles # Phase 2: serves api/app/static/
|
| 6 |
from pathlib import Path # Phase 2: resolve static directory path
|
| 7 |
|
|
@@ -36,32 +36,16 @@ app = FastAPI(
|
|
| 36 |
lifespan=lifespan,
|
| 37 |
)
|
| 38 |
|
| 39 |
-
|
| 40 |
-
app.add_middleware(
|
| 41 |
-
CORSMiddleware,
|
| 42 |
-
allow_origins=["http://localhost:3000", "http://127.0.0.1:3000", "*"],
|
| 43 |
-
allow_credentials=True,
|
| 44 |
-
allow_methods=["*"],
|
| 45 |
-
allow_headers=["*"],
|
| 46 |
-
)
|
| 47 |
|
| 48 |
app.include_router(predict.router, tags=["Predict"])
|
| 49 |
app.include_router(results.router, tags=["System"])
|
| 50 |
|
| 51 |
instrumentator.instrument(app).expose(app, endpoint="/metrics")
|
| 52 |
|
| 53 |
-
#
|
| 54 |
-
#
|
| 55 |
-
#
|
| 56 |
-
# The instrumentator middleware wraps the entire ASGI app; order of
|
| 57 |
-
# router inclusion does not affect which routes get instrumented. Including
|
| 58 |
-
# the pages router last is simply a readability convention β API routes first.
|
| 59 |
-
#
|
| 60 |
-
# WHY StaticFiles is mounted AFTER instrumentator:
|
| 61 |
-
# app.mount() creates a sub-application. Mounting after the instrumentator
|
| 62 |
-
# call means the Prometheus middleware still wraps /static/* requests, but
|
| 63 |
-
# since static files are not hot paths for an ML tool this is acceptable.
|
| 64 |
-
# (Phase 7 cleanup will add /static to excluded_handlers in metrics.py.)
|
| 65 |
|
| 66 |
app.include_router(pages.router) # include_in_schema=False is set on the router itself
|
| 67 |
|
|
|
|
| 1 |
from fastapi import FastAPI
|
| 2 |
from contextlib import asynccontextmanager
|
| 3 |
from dotenv import load_dotenv
|
| 4 |
+
|
| 5 |
from fastapi.staticfiles import StaticFiles # Phase 2: serves api/app/static/
|
| 6 |
from pathlib import Path # Phase 2: resolve static directory path
|
| 7 |
|
|
|
|
| 36 |
lifespan=lifespan,
|
| 37 |
)
|
| 38 |
|
| 39 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
app.include_router(predict.router, tags=["Predict"])
|
| 42 |
app.include_router(results.router, tags=["System"])
|
| 43 |
|
| 44 |
instrumentator.instrument(app).expose(app, endpoint="/metrics")
|
| 45 |
|
| 46 |
+
# Jinja2 / HTMX frontend routing and static files
|
| 47 |
+
# Note: StaticFiles is mounted after instrumentator so prometheus
|
| 48 |
+
# ignores it for /metrics, although this might still log /static requests.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
app.include_router(pages.router) # include_in_schema=False is set on the router itself
|
| 51 |
|
{dashboard β dashboard_backup}/.env.example
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/dist/_redirects
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/dist/assets/index-DGl1EjcM.css
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/dist/assets/index-w96O4ZPi.js
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/dist/index.html
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/index.html
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/package-lock.json
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/package.json
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/postcss.config.js
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/public/_redirects
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/App.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/api/client.js
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/AspectHeatmap.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/LanguagePie.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/Layout.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/LivePredictor.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/SentimentChart.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/components/Sidebar.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/config.js
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/index.css
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/main.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/pages/Analytics.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/pages/Monitor.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/src/pages/Predict.jsx
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/tailwind.config.js
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/vercel.json
RENAMED
|
File without changes
|
{dashboard β dashboard_backup}/vite.config.js
RENAMED
|
File without changes
|
docs/DEPLOYMENT.md
CHANGED
|
@@ -3,10 +3,10 @@
|
|
| 3 |
## Prerequisites
|
| 4 |
|
| 5 |
- Python 3.11+
|
| 6 |
-
-
|
| 7 |
- Docker & Docker Compose (for containerized deployment)
|
| 8 |
- Railway account (for API deployment)
|
| 9 |
-
-
|
| 10 |
|
| 11 |
## Environment Variables
|
| 12 |
|
|
@@ -36,11 +36,6 @@ uvicorn api.main:app --reload --host 0.0.0.0 --port 8000
|
|
| 36 |
# MLflow UI at http://localhost:5000
|
| 37 |
|
| 38 |
# Frontend
|
| 39 |
-
cd dashboard
|
| 40 |
-
cp .env.example .env # VITE_API_URL=http://localhost:8000
|
| 41 |
-
npm install
|
| 42 |
-
npm run dev
|
| 43 |
-
# Dashboard at http://localhost:5173
|
| 44 |
```
|
| 45 |
|
| 46 |
## Docker Compose (Full Stack)
|
|
@@ -90,7 +85,6 @@ graph TB
|
|
| 90 |
|
| 91 |
### Vercel (Dashboard)
|
| 92 |
|
| 93 |
-
1. Import `dashboard/` as a Vercel project
|
| 94 |
2. Framework preset: Vite
|
| 95 |
3. Environment variable: `VITE_API_URL=https://your-railway-api-url.railway.app`
|
| 96 |
4. `vercel.json` rewrites `/api/*` to Railway API
|
|
|
|
| 3 |
## Prerequisites
|
| 4 |
|
| 5 |
- Python 3.11+
|
| 6 |
+
-
|
| 7 |
- Docker & Docker Compose (for containerized deployment)
|
| 8 |
- Railway account (for API deployment)
|
| 9 |
+
-
|
| 10 |
|
| 11 |
## Environment Variables
|
| 12 |
|
|
|
|
| 36 |
# MLflow UI at http://localhost:5000
|
| 37 |
|
| 38 |
# Frontend
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
```
|
| 40 |
|
| 41 |
## Docker Compose (Full Stack)
|
|
|
|
| 85 |
|
| 86 |
### Vercel (Dashboard)
|
| 87 |
|
|
|
|
| 88 |
2. Framework preset: Vite
|
| 89 |
3. Environment variable: `VITE_API_URL=https://your-railway-api-url.railway.app`
|
| 90 |
4. `vercel.json` rewrites `/api/*` to Railway API
|
docs/SECURITY.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
| 9 |
| HTTPS | β Not enforced | Expects reverse proxy to terminate TLS |
|
| 10 |
| Input Validation | β
Partial | Pydantic validation present; no server-side max_length |
|
| 11 |
| SQL Injection | β
Protected | SQLAlchemy ORM parameterized queries |
|
| 12 |
-
| XSS | β
Protected |
|
| 13 |
| CSRF | β Not implemented | No CSRF middleware; CORS `"*"` mitigates partially |
|
| 14 |
| Secrets Management | β οΈ Manual | `.env` gitignored; Docker Compose has hardcoded dev creds |
|
| 15 |
| Rate Limiting | β Not implemented | No throttling on any endpoint |
|
|
|
|
| 9 |
| HTTPS | β Not enforced | Expects reverse proxy to terminate TLS |
|
| 10 |
| Input Validation | β
Partial | Pydantic validation present; no server-side max_length |
|
| 11 |
| SQL Injection | β
Protected | SQLAlchemy ORM parameterized queries |
|
| 12 |
+
| XSS | β
Protected | Jinja2 auto-escaping |
|
| 13 |
| CSRF | β Not implemented | No CSRF middleware; CORS `"*"` mitigates partially |
|
| 14 |
| Secrets Management | β οΈ Manual | `.env` gitignored; Docker Compose has hardcoded dev creds |
|
| 15 |
| Rate Limiting | β Not implemented | No throttling on any endpoint |
|
docs/architecture.md
CHANGED
|
@@ -43,10 +43,10 @@ graph TB
|
|
| 43 |
```mermaid
|
| 44 |
graph LR
|
| 45 |
subgraph "Presentation Layer"
|
| 46 |
-
SPA[
|
| 47 |
T_TAIL[TailwindCSS Theme]
|
| 48 |
RECH[Recharts Visualizations]
|
| 49 |
-
RQ[
|
| 50 |
end
|
| 51 |
subgraph "API Layer"
|
| 52 |
FAST[FastAPI]
|
|
@@ -111,7 +111,7 @@ graph TB
|
|
| 111 |
RW_REDIS[Redis]
|
| 112 |
end
|
| 113 |
subgraph "Vercel (Production)"
|
| 114 |
-
VC_DASH[
|
| 115 |
VC_RW_ROUTE[rewrite /api/* -> Railway]
|
| 116 |
end
|
| 117 |
|
|
|
|
| 43 |
```mermaid
|
| 44 |
graph LR
|
| 45 |
subgraph "Presentation Layer"
|
| 46 |
+
SPA[HTMX Frontend]
|
| 47 |
T_TAIL[TailwindCSS Theme]
|
| 48 |
RECH[Recharts Visualizations]
|
| 49 |
+
RQ[HTMX Polling]
|
| 50 |
end
|
| 51 |
subgraph "API Layer"
|
| 52 |
FAST[FastAPI]
|
|
|
|
| 111 |
RW_REDIS[Redis]
|
| 112 |
end
|
| 113 |
subgraph "Vercel (Production)"
|
| 114 |
+
VC_DASH[HTMX Frontend]
|
| 115 |
VC_RW_ROUTE[rewrite /api/* -> Railway]
|
| 116 |
end
|
| 117 |
|