Aryan Mishra commited on
Commit
5f1c2eb
·
1 Parent(s): 1202a82

Refresh README project overview

Browse files

Update the README to better describe the FastAPI + HTMX app, clarify the inference and monitoring stack, and modernize the repository layout and setup instructions. Also align the run commands and notes with the current Docker, DVC, and app entrypoint structure.

Files changed (1) hide show
  1. README.md +56 -66
README.md CHANGED
@@ -1,111 +1,101 @@
1
  # Multilingual Aspect-Based Sentiment Analysis (ABSA)
2
 
3
- ![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)
4
  ![FastAPI](https://img.shields.io/badge/FastAPI-0.111.0-00a393.svg)
5
- ![React](https://img.shields.io/badge/HTMX-1.9-3d72d4.svg)
6
  ![DVC](https://img.shields.io/badge/DVC-3.51.1-945dd6.svg)
7
  ![MLflow](https://img.shields.io/badge/MLflow-2.13.0-0194E2.svg)
8
 
9
- A production-ready, highly optimized Aspect-Based Sentiment Analysis (ABSA) system designed to process multilingual product reviews in **English, Hindi, and Hinglish**. It accurately extracts aspects and classifies their underlying sentiments.
10
 
11
- The system leverages state-of-the-art models like **XLM-RoBERTa** and **IndicBERT**, optimized using ONNX runtime, and includes a highly robust, zero-download, pure-Python rule-based fallback engine for instant inference.
12
 
13
- ## Key Features
14
 
15
- - **Multilingual Support**: First-class support for English, Hindi, and code-mixed Hinglish.
16
- - **Dual Inference Engine**:
17
- - **Neural Path**: Uses custom fine-tuned, INT8-quantized ONNX models (XLM-RoBERTa based) for extremely fast and accurate token classification and sequence classification.
18
- - **Rule-Based Fallback**: An instantaneous, pure-Python fallback leveraging a curated multi-lingual lexicon to handle aspect extraction and sentiment scoring without any heavy downloads.
19
- - **Modern Tech Stack**:
20
- - **Backend**: Asynchronous, high-performance API built with FastAPI.
21
- - **Frontend**: A responsive dashboard built with FastAPI, Jinja2, HTMX, and TailwindCSS. Features real-time predictions, batch analytics, and system monitoring.
22
- - **MLOps Integrated**: Complete integration with DVC (Data Version Control) for pipeline reproducibility, MLflow for experiment tracking, and Evidently AI for data drift monitoring.
23
- - **Scalable Architecture**: Support for async tasks via Celery + Redis, robust data storage via PostgreSQL, and metric exporting using Prometheus.
24
 
25
- ## Python-First Architecture
26
-
27
- This repository is designed following a **Python-first paradigm**:
28
- - **Python (99%)**: Handling all business logic, data processing, configuration, API routing, ML inference, SSR (Server-Side Rendering) via Jinja2, and utility functions.
29
- - **JavaScript (1%)**: Strictly limited to Alpine.js for minimal client-side interactivity and Chart.js for visualization (loaded via CDN).
30
- - *Note: There is no decoupled SPA frontend (like React) or Node.js runtime required.*
31
-
32
- ## Repository Structure
33
 
34
  ```text
35
- Multilingual-Absa/
36
- ├── api/app/ # FastAPI backend and inference services
37
- ├── config/ # Docker and application configuration files
38
- ├── api/app/templates/ # Jinja2 HTMX frontend templates
39
- ├── data/ # Dataset directory (DVC-tracked)
40
- ├── docs/ # Extended documentation (architecture, ml, api)
41
- ├── ml/ # ML training, notebooks, MLflow, and tracking
42
- ├── models/ # Model artifacts (DVC-tracked)
43
- ├── monitoring/ # Monitoring configurations (Prometheus, Evidently)
44
- ├── scripts/ # Utility and automation scripts
45
- ├── src/ # Core Machine Learning pipeline source code
46
- ├── tests/ # Unit and integration test suite
47
- ├── dvc.yaml # DVC pipeline orchestration
48
- └── requirements.txt # Python dependencies
49
  ```
50
 
51
  ## Getting Started
52
 
53
  ### Prerequisites
54
- - Python 3.10+
55
 
56
- - Docker & Docker Compose (Optional, but recommended)
 
57
 
58
- ### 1. Local Setup
59
 
60
- Clone the repository and install the backend dependencies:
61
  ```bash
62
  git clone <repository-url>
63
  cd Multilingual-Absa
64
 
65
- # Create a virtual environment and install dependencies
66
  python -m venv .venv
67
- source .venv/bin/activate # On Windows use `.venv\Scripts\activate`
68
  pip install -r requirements.txt
69
  ```
70
 
71
- Set up your environment variables:
72
- ```bash
73
- cp .env.example .env
74
- # Edit .env with your specific configurations
75
- ```
76
 
77
- ### 2. Run with Docker (Recommended)
78
 
79
- The easiest way to get the entire stack (API, Dashboard, Redis, Postgres) running is via Docker Compose:
80
  ```bash
81
- docker-compose -f deployment/docker/docker-compose.yml up --build
82
  ```
83
 
84
- ### 3. Manual ML Pipeline Execution (DVC)
 
 
 
 
 
 
 
85
 
86
- To reproduce the ML pipeline or sync artifacts:
87
  ```bash
88
- dvc pull # Pull data/models from remote storage
89
- dvc repro # Run the full end-to-end ML training pipeline
90
- dvc push # Push newly generated artifacts to remote
91
  ```
92
 
93
- ### 4. Running the Application Locally
 
 
94
 
95
- **Start the Application Server (API & Dashboard):**
96
  ```bash
97
- PYTHONPATH=. uvicorn api.app.main:app --reload --host 0.0.0.0 --port 8000
 
 
98
  ```
99
- - Access the dashboard at: `http://localhost:8000/predict`
100
- - API Documentation available at: `http://localhost:8000/docs`
101
 
102
- ## How it Works
 
 
 
 
 
 
 
 
103
 
104
- 1. **Prediction API**: When a review is submitted, the language is auto-detected.
105
- 2. **Inference**: The `ABSAPipeline` attempts to load INT8 quantized ONNX models for extraction and sentiment scoring.
106
- 3. **Fallback Mechanism**: If the custom models are not downloaded, the engine automatically falls back to a dictionary/rule-based engine tailored for product reviews, guaranteeing zero downtime and instant availability.
107
- 4. **Monitoring**: All predictions are logged. Performance metrics and data drift are tracked via Evidently and Prometheus.
108
 
109
  ## License
110
 
111
- This project is licensed under the MIT License - see the LICENSE file for details.
 
1
  # Multilingual Aspect-Based Sentiment Analysis (ABSA)
2
 
3
+ ![Python](https://img.shields.io/badge/Python-3.10%2B-blue.svg)
4
  ![FastAPI](https://img.shields.io/badge/FastAPI-0.111.0-00a393.svg)
5
+ ![HTMX](https://img.shields.io/badge/HTMX-1.9-3d72d4.svg)
6
  ![DVC](https://img.shields.io/badge/DVC-3.51.1-945dd6.svg)
7
  ![MLflow](https://img.shields.io/badge/MLflow-2.13.0-0194E2.svg)
8
 
9
+ This repository contains a multilingual Aspect-Based Sentiment Analysis system for English, Hindi, and Hinglish product reviews. It provides both a FastAPI JSON API and a server-rendered web UI backed by Jinja2 templates and HTMX interactions.
10
 
11
+ The inference stack is centered on ONNX Runtime models, with a fallback pipeline for environments where the custom artifacts are not available. The project also includes DVC pipelines, MLflow tracking, Redis/Celery workers, PostgreSQL, and Prometheus/Grafana monitoring.
12
 
13
+ ## What’s Included
14
 
15
+ - Multilingual review processing for English, Hindi, and Hinglish.
16
+ - FastAPI application with prediction, results, and monitoring routes.
17
+ - Server-rendered UI available at `/predict`, `/batch`, and `/monitor`.
18
+ - Batch processing, async task execution, and database-backed persistence.
19
+ - DVC, MLflow, and monitoring assets for experiment and system tracking.
 
 
 
 
20
 
21
+ ## Repository Layout
 
 
 
 
 
 
 
22
 
23
  ```text
24
+ api/ FastAPI app, routes, middleware, schemas, services, templates
25
+ config/ Docker and deployment configuration
26
+ data/ Raw and processed datasets
27
+ docs/ Architecture, API, deployment, and design notes
28
+ ml/ Training notebooks and experimentation assets
29
+ models/ Model artifacts, including ONNX assets
30
+ monitoring/ Prometheus and Grafana configuration
31
+ scripts/ Utility scripts for data, models, and monitoring
32
+ src/ Core data, training, and evaluation code
33
+ tests/ Test suite
34
+ dashboard_backup/ Legacy React dashboard preserved as a backup
 
 
 
35
  ```
36
 
37
  ## Getting Started
38
 
39
  ### Prerequisites
 
40
 
41
+ - Python 3.10+
42
+ - Docker and Docker Compose if you want the full stack
43
 
44
+ ### Local Setup
45
 
 
46
  ```bash
47
  git clone <repository-url>
48
  cd Multilingual-Absa
49
 
 
50
  python -m venv .venv
51
+ source .venv/bin/activate
52
  pip install -r requirements.txt
53
  ```
54
 
55
+ If you use environment variables, create a local `.env` file before starting the app.
 
 
 
 
56
 
57
+ ### Run the App Locally
58
 
 
59
  ```bash
60
+ PYTHONPATH=. uvicorn api.app.main:app --reload --host 0.0.0.0 --port 8000
61
  ```
62
 
63
+ Open:
64
+
65
+ - `http://localhost:8000/predict`
66
+ - `http://localhost:8000/batch`
67
+ - `http://localhost:8000/monitor`
68
+ - `http://localhost:8000/docs`
69
+
70
+ ### Run with Docker
71
 
 
72
  ```bash
73
+ docker compose -f config/docker/docker-compose.yml up --build
 
 
74
  ```
75
 
76
+ That compose file starts the API, worker, PostgreSQL, Redis, Prometheus, Grafana, and the dashboard service.
77
+
78
+ ### Reproduce the ML Pipeline
79
 
 
80
  ```bash
81
+ dvc pull
82
+ dvc repro
83
+ dvc push
84
  ```
 
 
85
 
86
+ ## How It Works
87
+
88
+ 1. A review is submitted through the API or web UI.
89
+ 2. The app detects or accepts the language and routes the request through the ABSA pipeline.
90
+ 3. The pipeline uses the available ONNX-backed models when present.
91
+ 4. If the model artifacts are missing, the system falls back to the rule-based path so inference can still continue.
92
+ 5. Predictions and runtime signals can be observed through the app, metrics endpoint, and monitoring stack.
93
+
94
+ ## Notes
95
 
96
+ - The active web UI is served by the FastAPI app. `dashboard_backup/` is kept only as a legacy reference.
97
+ - The main application entrypoint is `api.app.main:app`.
 
 
98
 
99
  ## License
100
 
101
+ This project is licensed under the MIT License. See the LICENSE file for details.