TinyFaceNet quantization report
Generated 2026-10-10 · CPU i7-1360P · torch 2.12 / onnxruntime 1.26
How to read: Family acc = centroid argmax on the date-held-out test faces (small set, so ±1 face ≈ ±2%). Open-set acc = same but with unknown threshold 0.45 and 'other' included. Agreement = share of 1500 faces from the full dataset where the quantized model picks the same person as FP32 — statistically the more reliable number. Cos = mean cosine similarity of quantized vs FP32 embeddings. INT4 rows are weight-only simulations (CPU/Pi have no int4 conv kernels; the gain is file size only). Latency here is noisy (measured while a training job was running) and is NOT Pi latency.
v1 — tinyfacenet_v1.pt
test faces: 59 (49 family) · agreement subset: 1500 · THR=0.45
| Variant | Family acc | Δ pts | Open-set acc | Δ pts | Agreement vs FP32 | Cos vs FP32 | Size | Latency/face (this PC) | Note |
|---|
| FP32 (PyTorch) | 98.0% | +0.0 | 89.8% | +0.0 | 100.0% | 1.0000 | 3.89 MB | 21.8 ms | PyTorch fp32 (deploy params only) |
| FP32 (ONNX Runtime) | 98.0% | +0.0 | 89.8% | +0.0 | 100.0% | 1.0000 | 3.88 MB | 59.3 ms | ONNX Runtime fp32 |
| INT8 static (ONNX Runtime) | 95.9% | -2.0 | 88.1% | -1.7 | 99.1% | 0.9957 | 1.09 MB | 82.3 ms | ONNX Runtime static int8, per-channel, 200 calib imgs (deployable) |
| INT8 weight-only (simulated) | 98.0% | +0.0 | 89.8% | +0.0 | 99.9% | 1.0000 | 0.97 MB | — | simulated: int8 weights per-channel, fp32 activations |
| INT4 weight-only (simulated) | 93.9% | -4.1 | 88.1% | -1.7 | 98.9% | 0.9755 | 0.49 MB | — | simulated: int4 weights per-channel (coarse) |
Per-person accuracy (test set)
| Variant | PERSON_A | PERSON_C | PERSON_D | PERSON_E | other | PERSON_G | PERSON_I |
|---|
| FP32 (PyTorch) | 100.0% | 100.0% | 90.9% | 100.0% | 40.0% | 100.0% | 100.0% |
| FP32 (ONNX Runtime) | 100.0% | 100.0% | 90.9% | 100.0% | 40.0% | 100.0% | 100.0% |
| INT8 static (ONNX Runtime) | 100.0% | 100.0% | 90.9% | 100.0% | 40.0% | 100.0% | 92.3% |
| INT8 weight-only (simulated) | 100.0% | 100.0% | 90.9% | 100.0% | 50.0% | 100.0% | 100.0% |
| INT4 weight-only (simulated) | 100.0% | 100.0% | 90.9% | 100.0% | 40.0% | 100.0% | 84.6% |
v2 — tinyfacenet_v2.pt
test faces: 59 (49 family) · agreement subset: 1500 · THR=0.45
| Variant | Family acc | Δ pts | Open-set acc | Δ pts | Agreement vs FP32 | Cos vs FP32 | Size | Latency/face (this PC) | Note |
|---|
| FP32 (PyTorch) | 95.9% | +0.0 | 89.8% | +0.0 | 100.0% | 1.0000 | 3.89 MB | 11.9 ms | PyTorch fp32 (deploy params only) |
| FP32 (ONNX Runtime) | 95.9% | +0.0 | 89.8% | +0.0 | 100.0% | 1.0000 | 3.88 MB | 63.2 ms | ONNX Runtime fp32 |
| INT8 static (ONNX Runtime) | 95.9% | +0.0 | 91.5% | +1.7 | 99.7% | 0.9980 | 1.09 MB | 91.3 ms | ONNX Runtime static int8, per-channel, 200 calib imgs (deployable) |
| INT8 weight-only (simulated) | 95.9% | +0.0 | 89.8% | +0.0 | 99.9% | 0.9999 | 0.97 MB | — | simulated: int8 weights per-channel, fp32 activations |
| INT4 weight-only (simulated) | 93.9% | -2.0 | 86.4% | -3.4 | 99.2% | 0.9783 | 0.49 MB | — | simulated: int4 weights per-channel (coarse) |
Per-person accuracy (test set)
| Variant | PERSON_A | PERSON_C | PERSON_D | PERSON_E | other | PERSON_G | PERSON_I |
|---|
| FP32 (PyTorch) | 100.0% | 85.7% | 90.9% | 100.0% | 60.0% | 100.0% | 100.0% |
| FP32 (ONNX Runtime) | 100.0% | 85.7% | 90.9% | 100.0% | 60.0% | 100.0% | 100.0% |
| INT8 static (ONNX Runtime) | 100.0% | 85.7% | 90.9% | 100.0% | 50.0% | 100.0% | 100.0% |
| INT8 weight-only (simulated) | 100.0% | 85.7% | 90.9% | 100.0% | 60.0% | 100.0% | 100.0% |
| INT4 weight-only (simulated) | 100.0% | 85.7% | 90.9% | 100.0% | 50.0% | 100.0% | 92.3% |
Recommendation
- Ship INT8 static (ONNX Runtime): ~1 MB, ~1 pt drop, runs natively on Pi 5 / Jetson with real int8 kernels.
- INT4 buys only ~0.5 MB more and costs ~2-4 pts with no speed benefit on these targets. Not worth it for a 1M-param net.
- If INT8 drop matters: quantization-aware fine-tune for 5 epochs, or calibrate with more images (200 used).