Replace deprecated get_event_loop, drop the tutorial Settings docstring, and
fail CI if pyright reports type errors.
feat: landing server-status panel and embed ttft/ttfr
Aditya Kulkarni
5e96f99
Render GET /health on the landing as server status so the App tab does
not jump to raw JSON. POST /embed includes ttft_ms (enqueue to encode
start) and ttfr_ms (enqueue to embeddings ready). Keep GET /metrics as
an unused always-200 JSON alias.
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya Kulkarni
dfdc75f
Pass max_batch_size through model.encode so 128/256/512 batches are one
forward pass, not silently chunked at 32. Add file-sourced bench harness,
prepare_dataset.py, and the 24-run latency/throughput matrix including
dynamic INT8.
Document the two-regime result and the INT8 slowdown. Keep nfcorpus text
pools gitignored; they regenerate with a fixed seed.
feat: add input validation, backpressure, and request timeouts
Aditya Kulkarni
3c0d20e
Harden the /embed serving contract:
- Validate request body (non-empty texts, max texts/request, max chars/text) -> 422
- Bound the request queue; reject with 503 when full instead of growing unbounded
- Enforce a per-request deadline; return 504 on timeout
- Map batcher errors to clean HTTP statuses; sanitize unexpected errors to 500
- Guard future resolution so a timed-out request can't crash the worker
Limits are env-configurable (MAX_TEXTS_PER_REQUEST, MAX_CHARS_PER_TEXT,
MAX_QUEUE_SIZE, REQUEST_TIMEOUT_S). Add tests for queue-full, timeout, and the
422/503/504 endpoint paths; update README failure-handling and API docs.
test: add pytest suite and CI
Aditya Kulkarni
299567b
- DynamicBatcher tests: size/time flush triggers, request->response mapping across
mixed batch sizes, exception propagation, and shutdown queue-drain, using a
deterministic fake model (no MiniLM download, CPU-only)
- endpoint tests for /health and /embed via FastAPI TestClient with load_model stubbed
- pytest + pytest-asyncio + ruff as a dev dependency group; pytest/ruff config
- GitHub Actions workflow: uv sync, ruff check, pytest
- Dockerfile: uv sync --no-dev so dev tools and the plot extra stay out of the image
- add CI badge to README
