multilingual-absa / docs /TECH_STACK.md
Aryan Mishra
Add CI, typed ORM models, and packaging cleanup
a00fee9
|
Raw
History Blame Contribute Delete
4.18 kB

A newer version of the Gradio SDK is available: 6.26.0

Upgrade

Technology Stack β€” Multilingual ABSA

Core Technologies

Technology Version Purpose Where Used
Python 3.11+ Runtime All backend/ML code
FastAPI 0.111.0 REST API framework api/ routes and middleware
Uvicorn 0.29.0 ASGI server API entry point
Streamlit 1.37.0 Dashboard frontend (pure Python) frontend/
PostgreSQL 16 (alpine) Production database Docker Compose
SQLite (built-in) Development database absa.db
Redis 7 (alpine) Celery broker + cache api/tasks/
Docker 27.x Containerization docker/
Docker Compose 3.8+ Orchestration docker/docker-compose.yml

ML / AI Stack

Technology Version Purpose Where Used
PyTorch 2.3.0 Deep learning framework src/absa/models/ training
Transformers 4.39.3 Model zoo, training, tokenization All ML scripts
XLM-RoBERTa base Multilingual encoder FacebookAI/xlm-roberta-base
ONNX Runtime 1.18.0 Production inference api/services/absa_pipeline.py
Optimum 1.19.0 ONNX export bridge src/absa/models/export_onnx.py
optimum-onnx (bundled) ONNX runtime models ORTModelForTokenClassification, ORTModelForSequenceClassification
PEFT 0.10.0 Parameter-efficient fine-tuning src/absa/models/train_qlora.py (LoRA)
scikit-learn 1.4.2 Metrics + baseline src/absa/models/baseline.py, train_sentiment.py
Datasets 2.19.0 Data loading src/absa/data/hf_dataset.py
seqeval 1.2.2 BIO tagging evaluation src/absa/models/train_aspect_extraction.py
fasttext-predict 0.9.2.4 Language identification src/absa/data/lang_detect.py, api/services/lang_service.py
indic-nlp-library (git) Devanagari transliteration src/absa/data/transliterate.py
nlpaug 1.1.11 Text augmentation src/absa/data/augmentation.py

MLOps Stack

Technology Version Purpose Where Used
MLflow 2.13.0 Experiment tracking src/absa/training/mlflow_utils.py, all src/absa/models/
DVC 3.51.1 Data version control dvc.yaml, .dvc/
Evidently AI 0.4.30 Data drift monitoring scripts/drift_monitor.py
Prometheus latest Metrics collection monitoring/prometheus.yml
Grafana latest Dashboard visualization monitoring/grafana/dashboards/
prometheus-fastapi-instrumentator 7.0.0 Metrics middleware api/middleware/metrics.py

API / Backend Libraries

Technology Version Purpose
Pydantic 2.7.1 Request/response schema validation
SQLAlchemy (via psycopg2) ORM
psycopg2-binary 2.9.9 PostgreSQL driver
Celery 5.4.0 Async task queue
python-dotenv 1.0.1 Environment variable loading
python-multipart 0.0.9 File upload parsing
NumPy 1.26.4 Numerical computing
Pandas 2.2.2 Data manipulation

Frontend Libraries

Technology Version Purpose
Streamlit 1.37.0 Native dashboard widgets β€” no HTML templates or JS build step
httpx 0.28.0 HTTP client talking to the FastAPI backend

Infrastructure / Deployment

Technology Purpose
Docker (multi-stage) Build optimization
Railway API + worker cloud hosting
HuggingFace Hub Model storage/pull

Version Compatibility Matrix

Package Python PyTorch ONNX Runtime
transformers 4.39.3 3.8+ 1.11+ β€”
optimum 1.19.0 3.8+ 1.13+ 1.15+
onnxruntime 1.18.0 3.8+ β€” β€”
peft 0.10.0 3.8+ 2.0+ β€”
mlflow 2.13.0 3.8+ β€” β€”
dvc 3.51.1 3.8+ β€” β€”
evidently 0.4.30 3.8+ β€” β€”
fastapi 0.111.0 3.8+ β€” β€”
celery 5.4.0 3.8+ β€” β€”