Spaces:
Sleeping
Sleeping
Commit History
-
feat: add nfcorpus quality eval for fp32 vs INT8
Aditya KulkarniMeasure cosine drift, rank agreement, and nDCG/recall so INT8 is judged on embedding quality, not speed alone.c745c50 -
chore: add pyright CI and use get_running_loop
Aditya KulkarniReplace deprecated get_event_loop, drop the tutorial Settings docstring, and fail CI if pyright reports type errors.14b0898 -
feat: landing server-status panel and embed ttft/ttfr
Aditya KulkarniRender GET /health on the landing as server status so the App tab does not jump to raw JSON. POST /embed includes ttft_ms (enqueue to encode start) and ttfr_ms (enqueue to embeddings ready). Keep GET /metrics as an unused always-200 JSON alias.5e96f99 -
feat: add HTML landing page for the Spaces App tab
Aditya KulkarniServe GET / as a static page that lists the public endpoints and links to Swagger. Restore the live-demo README framing now that the Space responds.c4fe3e6 -
feat: add /metrics snapshot and demote dead live-demo badge
Aditya KulkarniReuse the /health body on GET /metrics and always return 200. Drop the "metrics not exposed" limitation. The HF Space timed out on /health and /embed, so the README no longer frames it as a live demo.84098e5
-
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya KulkarniPass max_batch_size through model.encode so 128/256/512 batches are one forward pass, not silently chunked at 32. Add file-sourced bench harness, prepare_dataset.py, and the 24-run latency/throughput matrix including dynamic INT8. Document the two-regime result and the INT8 slowdown. Keep nfcorpus text pools gitignored; they regenerate with a fixed seed.dfdc75f
-
feat: add readiness /health and structured request logging
Aditya Kulkarni- /health is now a readiness probe (200/503) reporting model_loaded, worker_alive, queue_depth, and inflight; previously a static "ok" that inspected nothing - DynamicBatcher exposes is_running()/queue_depth()/inflight() with an in-flight counter tracked in submit() - JSON logging (app/logging_config.py) with a per-request X-Request-ID correlation id via middleware; structured access logs (method, path, status, duration_ms); inference errors logged server-side with a sanitized 500 returned to the client - drop the unused request param from /embed; add LOG_LEVEL setting - tests for readiness-503 and request-id generate/echo; README /health + logging docs3e47cef -
feat: add input validation, backpressure, and request timeouts
Aditya KulkarniHarden the /embed serving contract: - Validate request body (non-empty texts, max texts/request, max chars/text) -> 422 - Bound the request queue; reject with 503 when full instead of growing unbounded - Enforce a per-request deadline; return 504 on timeout - Map batcher errors to clean HTTP statuses; sanitize unexpected errors to 500 - Guard future resolution so a timed-out request can't crash the worker Limits are env-configurable (MAX_TEXTS_PER_REQUEST, MAX_CHARS_PER_TEXT, MAX_QUEUE_SIZE, REQUEST_TIMEOUT_S). Add tests for queue-full, timeout, and the 422/503/504 endpoint paths; update README failure-handling and API docs.3c0d20e
-
test: add pytest suite and CI
Aditya Kulkarni- DynamicBatcher tests: size/time flush triggers, request->response mapping across mixed batch sizes, exception propagation, and shutdown queue-drain, using a deterministic fake model (no MiniLM download, CPU-only) - endpoint tests for /health and /embed via FastAPI TestClient with load_model stubbed - pytest + pytest-asyncio + ruff as a dev dependency group; pytest/ruff config - GitHub Actions workflow: uv sync, ruff check, pytest - Dockerfile: uv sync --no-dev so dev tools and the plot extra stay out of the image - add CI badge to README299567b
-
feat: add POST /predict endpoint with MiniLM embedding generation
Aditya Kulkarni0b7903a