README.md

metadata

title: Model Speed Comparator
emoji: 🚀
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 8000
pinned: false

Model Speed Comparator

Compare PyTorch baseline vs ONNX vs INT8 Quantized inference — same model, same prediction, dramatically different performance.

Built to demonstrate real-world AI inference optimization techniques used in production ML systems and AI accelerator pipelines.

What It Does

Takes any text input and runs it through 3 versions of the same NLP model (DistilBERT sentiment classifier):

Variant	Format	What changes
Baseline	PyTorch .bin	Standard HuggingFace model, no optimization
ONNX	.onnx	Exported + graph-optimized by ONNX Runtime
Quantized	INT8 .onnx	Weights compressed from FP32 to INT8

Key Results (CPU)

Variant	Latency	Size	vs Baseline
PyTorch Baseline	5594ms	268MB	1x
ONNX	547ms	255MB	10x faster
INT8 Quantized	26ms	64MB	213x faster, 4x smaller

Setup

git clone https://github.com/Mridul0603/Model-Speed-Comparator
cd Model-Speed-Comparator
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

Open http://localhost:8000

Tech Stack

• FastAPI
• HuggingFace Transformers
• ONNX Runtime
• Optimum
• Docker

API

POST /compare - runs all 3 variants and returns latency comparison POST /benchmark - runs 20x stress test with p95 stats GET /history - last 10 comparisons GET /stats - session aggregate stats GET /health - health check