| <!doctype html><html><head><meta charset='utf-8'><title>TinyFaceNet quantization report</title><style>body{font-family:Segoe UI,system-ui,sans-serif;max-width:1100px;margin:2rem auto;padding:0 1rem;color:#222;line-height:1.45}
|
| h1{font-size:1.6rem}h2{font-size:1.2rem;margin-top:2rem;border-bottom:1px solid #ddd;padding-bottom:.3rem}
|
| table{border-collapse:collapse;width:100%;margin:1rem 0;font-size:.92rem}th,td{border:1px solid #ddd;padding:.45rem .6rem;text-align:right}
|
| th:first-child,td:first-child{text-align:left}th{background:#f4f4f4}tr:nth-child(even){background:#fafafa}
|
| .good{color:#1a7f37}.warn{color:#9a6700}.bad{color:#c0392b;font-weight:600}.muted{color:#666;font-size:.88rem}
|
| .bar{display:inline-block;height:10px;background:#4a7bd0;vertical-align:middle;margin-right:.4rem}code{background:#f2f2f2;padding:0 .25rem}
|
| .note{background:#fff8e1;border-left:4px solid #e0a800;padding:.6rem 1rem;margin:1rem 0}</style></head><body>
|
| <h1>TinyFaceNet quantization report</h1><p class='muted'>Generated 2026-10-10 · CPU i7-1360P · torch 2.12 / onnxruntime 1.26</p>
|
| <div class='note'><b>How to read:</b> <i>Family acc</i> = centroid argmax on the date-held-out test faces (small set, so ±1 face ≈ ±2%). <i>Open-set acc</i> = same but with unknown threshold 0.45 and 'other' included. <i>Agreement</i> = share of 1500 faces from the full dataset where the quantized model picks the same person as FP32 — statistically the more reliable number. <i>Cos</i> = mean cosine similarity of quantized vs FP32 embeddings. INT4 rows are weight-only simulations (CPU/Pi have no int4 conv kernels; the gain is file size only). Latency here is noisy (measured while a training job was running) and is NOT Pi latency.</div>
|
| <h2>v1 — <code>tinyfacenet_v1.pt</code></h2>
|
| <p class='muted'>test faces: 59 (49 family) · agreement subset: 1500 · THR=0.45</p>
|
| <table><tr><th>Variant</th><th>Family acc</th><th>Δ pts</th><th>Open-set acc</th><th>Δ pts</th><th>Agreement vs FP32</th><th>Cos vs FP32</th><th>Size</th><th>Latency/face (this PC)</th><th>Note</th></tr>
|
| <tr><td>FP32 (PyTorch)</td><td>98.0%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>100.0%</td><td>1.0000</td><td><span class='bar' style='width:120px'></span>3.89 MB</td><td>21.8 ms</td><td class='muted'>PyTorch fp32 (deploy params only)</td></tr>
|
| <tr><td>FP32 (ONNX Runtime)</td><td>98.0%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>100.0%</td><td>1.0000</td><td><span class='bar' style='width:119px'></span>3.88 MB</td><td>59.3 ms</td><td class='muted'>ONNX Runtime fp32</td></tr>
|
| <tr><td>INT8 static (ONNX Runtime)</td><td>95.9%</td><td class='warn'>-2.0</td><td>88.1%</td><td class='warn'>-1.7</td><td class='warn'>99.1%</td><td>0.9957</td><td><span class='bar' style='width:33px'></span>1.09 MB</td><td>82.3 ms</td><td class='muted'>ONNX Runtime static int8, per-channel, 200 calib imgs (deployable)</td></tr>
|
| <tr><td>INT8 weight-only (simulated)</td><td>98.0%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>99.9%</td><td>1.0000</td><td><span class='bar' style='width:30px'></span>0.97 MB</td><td>—</td><td class='muted'>simulated: int8 weights per-channel, fp32 activations</td></tr>
|
| <tr><td>INT4 weight-only (simulated)</td><td>93.9%</td><td class='bad'>-4.1</td><td>88.1%</td><td class='warn'>-1.7</td><td class='warn'>98.9%</td><td>0.9755</td><td><span class='bar' style='width:15px'></span>0.49 MB</td><td>—</td><td class='muted'>simulated: int4 weights per-channel (coarse)</td></tr>
|
| </table>
|
| <details><summary>Per-person accuracy (test set)</summary><table><tr><th>Variant</th><th>PERSON_A</th><th>PERSON_C</th><th>PERSON_D</th><th>PERSON_E</th><th>other</th><th>PERSON_G</th><th>PERSON_I</th></tr>
|
| <tr><td>FP32 (PyTorch)</td><td>100.0%</td><td>100.0%</td><td>90.9%</td><td>100.0%</td><td>40.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>FP32 (ONNX Runtime)</td><td>100.0%</td><td>100.0%</td><td>90.9%</td><td>100.0%</td><td>40.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>INT8 static (ONNX Runtime)</td><td>100.0%</td><td>100.0%</td><td>90.9%</td><td>100.0%</td><td>40.0%</td><td>100.0%</td><td>92.3%</td></tr>
|
| <tr><td>INT8 weight-only (simulated)</td><td>100.0%</td><td>100.0%</td><td>90.9%</td><td>100.0%</td><td>50.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>INT4 weight-only (simulated)</td><td>100.0%</td><td>100.0%</td><td>90.9%</td><td>100.0%</td><td>40.0%</td><td>100.0%</td><td>84.6%</td></tr>
|
| </table></details>
|
| <h2>v2 — <code>tinyfacenet_v2.pt</code></h2>
|
| <p class='muted'>test faces: 59 (49 family) · agreement subset: 1500 · THR=0.45</p>
|
| <table><tr><th>Variant</th><th>Family acc</th><th>Δ pts</th><th>Open-set acc</th><th>Δ pts</th><th>Agreement vs FP32</th><th>Cos vs FP32</th><th>Size</th><th>Latency/face (this PC)</th><th>Note</th></tr>
|
| <tr><td>FP32 (PyTorch)</td><td>95.9%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>100.0%</td><td>1.0000</td><td><span class='bar' style='width:120px'></span>3.89 MB</td><td>11.9 ms</td><td class='muted'>PyTorch fp32 (deploy params only)</td></tr>
|
| <tr><td>FP32 (ONNX Runtime)</td><td>95.9%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>100.0%</td><td>1.0000</td><td><span class='bar' style='width:119px'></span>3.88 MB</td><td>63.2 ms</td><td class='muted'>ONNX Runtime fp32</td></tr>
|
| <tr><td>INT8 static (ONNX Runtime)</td><td>95.9%</td><td class='good'>+0.0</td><td>91.5%</td><td class='good'>+1.7</td><td class='good'>99.7%</td><td>0.9980</td><td><span class='bar' style='width:33px'></span>1.09 MB</td><td>91.3 ms</td><td class='muted'>ONNX Runtime static int8, per-channel, 200 calib imgs (deployable)</td></tr>
|
| <tr><td>INT8 weight-only (simulated)</td><td>95.9%</td><td class='good'>+0.0</td><td>89.8%</td><td class='good'>+0.0</td><td class='good'>99.9%</td><td>0.9999</td><td><span class='bar' style='width:30px'></span>0.97 MB</td><td>—</td><td class='muted'>simulated: int8 weights per-channel, fp32 activations</td></tr>
|
| <tr><td>INT4 weight-only (simulated)</td><td>93.9%</td><td class='warn'>-2.0</td><td>86.4%</td><td class='bad'>-3.4</td><td class='warn'>99.2%</td><td>0.9783</td><td><span class='bar' style='width:15px'></span>0.49 MB</td><td>—</td><td class='muted'>simulated: int4 weights per-channel (coarse)</td></tr>
|
| </table>
|
| <details><summary>Per-person accuracy (test set)</summary><table><tr><th>Variant</th><th>PERSON_A</th><th>PERSON_C</th><th>PERSON_D</th><th>PERSON_E</th><th>other</th><th>PERSON_G</th><th>PERSON_I</th></tr>
|
| <tr><td>FP32 (PyTorch)</td><td>100.0%</td><td>85.7%</td><td>90.9%</td><td>100.0%</td><td>60.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>FP32 (ONNX Runtime)</td><td>100.0%</td><td>85.7%</td><td>90.9%</td><td>100.0%</td><td>60.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>INT8 static (ONNX Runtime)</td><td>100.0%</td><td>85.7%</td><td>90.9%</td><td>100.0%</td><td>50.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>INT8 weight-only (simulated)</td><td>100.0%</td><td>85.7%</td><td>90.9%</td><td>100.0%</td><td>60.0%</td><td>100.0%</td><td>100.0%</td></tr>
|
| <tr><td>INT4 weight-only (simulated)</td><td>100.0%</td><td>85.7%</td><td>90.9%</td><td>100.0%</td><td>50.0%</td><td>100.0%</td><td>92.3%</td></tr>
|
| </table></details>
|
| <h2>Recommendation</h2><ul><li><b>Ship INT8 static (ONNX Runtime)</b>: ~1 MB, ~1 pt drop, runs natively on Pi 5 / Jetson with real int8 kernels.</li><li><b>INT4</b> buys only ~0.5 MB more and costs ~2-4 pts with no speed benefit on these targets. Not worth it for a 1M-param net.</li><li>If INT8 drop matters: quantization-aware fine-tune for 5 epochs, or calibrate with more images (200 used).</li></ul>
|
| </body></html> |
