README.md

metadata

title: Vapi Sathi Embeddings
emoji: 🔎
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

vapi-sathi embeddings

FastAPI /embed service hosting libryo-ai/BAAI-bge-m3-int8 — the int8 ONNX, dense-only build of bge-m3 (1024-dim, multilingual incl. Hindi/Gujarati). ~3× faster + ~half the RAM of fp32, negligible accuracy loss. Served via raw onnxruntime (no torch/optimum) with CLS pooling + L2-normalize. Offloads embedding off the NestJS API box (see ../INFRA-DEPLOYMENT-PLAN.md). Deployed on a free CPU Basic Hugging Face Space (2 vCPU / 16 GB).

API

GET / → health: { "status": "ok", "model": "...", "ready": true }

POST /embed

{ "texts": ["latest mandi prices", "ભાવ સમાચાર"] }

→

{ "embeddings": [[...1024 floats...], [...]], "dim": 1024, "model": "libryo-ai/BAAI-bge-m3-int8" }

Vectors are L2-normalized → use cosine / dot directly. bge-m3 needs no query/passage prefix.

Auth

If EMBED_TOKEN is set (Space secret), every /embed call must send Authorization: Bearer <EMBED_TOKEN>. Leave unset only for private testing. The NestJS client reads EMBEDDING_SERVICE_URL + EMBEDDING_SERVICE_TOKEN.

Keep-warm

Free Spaces sleep after ~48 h idle. Worker traffic keeps it warm; otherwise ping GET / on a cron (e.g. a NestJS @Cron).

Local test

docker build -t vapi-embed . && docker run -p 7860:7860 vapi-embed
curl -s localhost:7860/embed -H 'content-type: application/json' \
  -d '{"texts":["hello","नमस्ते"]}' | head -c 200