Spaces:
Runtime error
Runtime error
File size: 4,176 Bytes
90e5963 dbd4e8b 90e5963 dbd4e8b 90e5963 a00fee9 90e5963 a00fee9 90e5963 a00fee9 90e5963 a00fee9 dbd4e8b 90e5963 dbd4e8b 90e5963 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | # Technology Stack — Multilingual ABSA
## Core Technologies
| Technology | Version | Purpose | Where Used |
|------------|---------|---------|------------|
| **Python** | 3.11+ | Runtime | All backend/ML code |
| **FastAPI** | 0.111.0 | REST API framework | `api/` routes and middleware |
| **Uvicorn** | 0.29.0 | ASGI server | API entry point |
| **Streamlit** | 1.37.0 | Dashboard frontend (pure Python) | `frontend/` |
| **PostgreSQL** | 16 (alpine) | Production database | Docker Compose |
| **SQLite** | (built-in) | Development database | `absa.db` |
| **Redis** | 7 (alpine) | Celery broker + cache | `api/tasks/` |
| **Docker** | 27.x | Containerization | `docker/` |
| **Docker Compose** | 3.8+ | Orchestration | `docker/docker-compose.yml` |
## ML / AI Stack
| Technology | Version | Purpose | Where Used |
|------------|---------|---------|------------|
| **PyTorch** | 2.3.0 | Deep learning framework | `src/absa/models/` training |
| **Transformers** | 4.39.3 | Model zoo, training, tokenization | All ML scripts |
| **XLM-RoBERTa** | base | Multilingual encoder | `FacebookAI/xlm-roberta-base` |
| **ONNX Runtime** | 1.18.0 | Production inference | `api/services/absa_pipeline.py` |
| **Optimum** | 1.19.0 | ONNX export bridge | `src/absa/models/export_onnx.py` |
| **optimum-onnx** | (bundled) | ONNX runtime models | `ORTModelForTokenClassification`, `ORTModelForSequenceClassification` |
| **PEFT** | 0.10.0 | Parameter-efficient fine-tuning | `src/absa/models/train_qlora.py` (LoRA) |
| **scikit-learn** | 1.4.2 | Metrics + baseline | `src/absa/models/baseline.py`, `train_sentiment.py` |
| **Datasets** | 2.19.0 | Data loading | `src/absa/data/hf_dataset.py` |
| **seqeval** | 1.2.2 | BIO tagging evaluation | `src/absa/models/train_aspect_extraction.py` |
| **fasttext-predict** | 0.9.2.4 | Language identification | `src/absa/data/lang_detect.py`, `api/services/lang_service.py` |
| **indic-nlp-library** | (git) | Devanagari transliteration | `src/absa/data/transliterate.py` |
| **nlpaug** | 1.1.11 | Text augmentation | `src/absa/data/augmentation.py` |
## MLOps Stack
| Technology | Version | Purpose | Where Used |
|------------|---------|---------|------------|
| **MLflow** | 2.13.0 | Experiment tracking | `src/absa/training/mlflow_utils.py`, all `src/absa/models/` |
| **DVC** | 3.51.1 | Data version control | `dvc.yaml`, `.dvc/` |
| **Evidently AI** | 0.4.30 | Data drift monitoring | `scripts/drift_monitor.py` |
| **Prometheus** | latest | Metrics collection | `monitoring/prometheus.yml` |
| **Grafana** | latest | Dashboard visualization | `monitoring/grafana/dashboards/` |
| **prometheus-fastapi-instrumentator** | 7.0.0 | Metrics middleware | `api/middleware/metrics.py` |
## API / Backend Libraries
| Technology | Version | Purpose |
|------------|---------|---------|
| **Pydantic** | 2.7.1 | Request/response schema validation |
| **SQLAlchemy** | (via psycopg2) | ORM |
| **psycopg2-binary** | 2.9.9 | PostgreSQL driver |
| **Celery** | 5.4.0 | Async task queue |
| **python-dotenv** | 1.0.1 | Environment variable loading |
| **python-multipart** | 0.0.9 | File upload parsing |
| **NumPy** | 1.26.4 | Numerical computing |
| **Pandas** | 2.2.2 | Data manipulation |
## Frontend Libraries
| Technology | Version | Purpose |
|------------|---------|---------|
| **Streamlit** | 1.37.0 | Native dashboard widgets — no HTML templates or JS build step |
| **httpx** | 0.28.0 | HTTP client talking to the FastAPI backend |
## Infrastructure / Deployment
| Technology | Purpose |
|------------|---------|
| **Docker** (multi-stage) | Build optimization |
| **Railway** | API + worker cloud hosting |
| **HuggingFace Hub** | Model storage/pull |
## Version Compatibility Matrix
| Package | Python | PyTorch | ONNX Runtime |
|---------|--------|---------|--------------|
| transformers 4.39.3 | 3.8+ | 1.11+ | — |
| optimum 1.19.0 | 3.8+ | 1.13+ | 1.15+ |
| onnxruntime 1.18.0 | 3.8+ | — | — |
| peft 0.10.0 | 3.8+ | 2.0+ | — |
| mlflow 2.13.0 | 3.8+ | — | — |
| dvc 3.51.1 | 3.8+ | — | — |
| evidently 0.4.30 | 3.8+ | — | — |
| fastapi 0.111.0 | 3.8+ | — | — |
| celery 5.4.0 | 3.8+ | — | — |
|