feat: add nfcorpus quality eval for fp32 vs INT8Running
Aditya Kulkarni
c745c50
Measure cosine drift, rank agreement, and nDCG/recall so INT8 is judged on embedding quality, not speed alone.
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya Kulkarni
dfdc75f
Pass max_batch_size through model.encode so 128/256/512 batches are one
forward pass, not silently chunked at 32. Add file-sourced bench harness,
prepare_dataset.py, and the 24-run latency/throughput matrix including
dynamic INT8.
Document the two-regime result and the INT8 slowdown. Keep nfcorpus text
pools gitignored; they regenerate with a fixed seed.
test: add benchmarking suite and publish results
Aditya Kulkarni
72c62ed
- run_matrix.py drives the full 16-config matrix (one server per config)
- matrix spans cpu/mps, batch sizes 1-32, and pytorch vs onnx backends
- naive_bench + utils add model-only and serving-overhead baselines
- bench.py records the device in its output metadata
- plot_results.py renders the batching sweep chart
- results in benchmarks/README.md, methodology in benchmarks/ANALYSIS.md
- surface the results in the project README
