File size: 3,785 Bytes
5e970ac 65e6cb4 5e970ac 65e6cb4 5e970ac | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | ---
license: other
license_name: insightface-non-commercial
license_link: https://github.com/deepinsight/insightface#license
tags:
- face-detection
- face-recognition
- scrfd
- arcface
- onnx
- batch-inference
- tensorrt
library_name: onnx
pipeline_tag: image-classification
---
# InsightFace Batch-Optimized Models (Max Batch 32)
Re-exported InsightFace models with **proper dynamic batch support** and **no cross-frame contamination**.
## Version Comparison
| Repository | Max Batch | Recommendation |
|------------|-----------|----------------|
| **This repo** | **1-32** | ✅ **Recommended** - Optimal performance |
| [alonsorobots/scrfd_320_batched_64](https://huggingface.co/alonsorobots/scrfd_320_batched_64) | 1-64 | For experimentation |
**Batch=32 is optimal.** Testing on RTX 5090 shows batch=64 provides no additional throughput benefit.
## Why These Models?
The original InsightFace ONNX models have issues with batch inference:
- `buffalo_l` detection model: hardcoded batch=1
- `buffalo_l_batch` detection model: **broken** - has cross-frame contamination due to reshape operations that flatten the batch dimension
These re-exports fix the `dynamic_axes` in the ONNX graph for **true batch inference**.
## Models
| Model | Task | Input Shape | Output | Batch | Speedup |
|-------|------|-------------|--------|-------|---------|
| `scrfd_10g_320_batch.onnx` | Face Detection | `[N, 3, 320, 320]` | boxes, landmarks | 1-32 | **6×** |
| `arcface_w600k_r50_batch.onnx` | Face Embedding | `[N, 3, 112, 112]` | 512-dim vectors | 1-32 | **10×** |
## Performance (TensorRT FP16, RTX 5090)
### SCRFD Face Detection
| Batch Size | FPS | ms/frame |
|------------|-----|----------|
| 1 | 867 | 1.15 |
| 16 | **5,498** | 0.18 |
### ArcFace Embeddings
| Batch Size | FPS | ms/embedding |
|------------|-----|--------------|
| 1 | 292 | 3.4 |
| 16 | **3,029** | 0.33 |
## Usage
```python
import numpy as np
import onnxruntime as ort
# Load model
sess = ort.InferenceSession("scrfd_10g_320_batch.onnx",
providers=["TensorrtExecutionProvider", "CUDAExecutionProvider"])
# Batch inference
batch = np.random.randn(16, 3, 320, 320).astype(np.float32)
outputs = sess.run(None, {"input.1": batch})
# outputs[0-2]: scores per FPN level (stride 8, 16, 32)
# outputs[3-5]: bboxes per FPN level
# outputs[6-8]: keypoints per FPN level
```
## Verified: No Batch Contamination
```python
# Same frame processed alone vs in batch = identical results
single_output = sess.run(None, {"input.1": frame[np.newaxis, ...]})
batch[7] = frame
batch_output = sess.run(None, {"input.1": batch})
max_diff = np.max(np.abs(single_output[0] - batch_output[0][7]))
# max_diff < 1e-5 ✓
```
## Re-export Process
These models were re-exported from InsightFace's PyTorch source using MMDetection with proper `dynamic_axes`:
```python
dynamic_axes = {
"input.1": {0: "batch"},
"score_8": {0: "batch"},
"score_16": {0: "batch"},
# ... all outputs
}
```
See [SCRFD_320_EXPORT_INSTRUCTIONS.md](https://github.com/deepinsight/insightface/issues/1573) for details.
## License
**Non-commercial research purposes only** - per [InsightFace license](https://github.com/deepinsight/insightface#license).
For commercial licensing, contact: `recognition-oss-pack@insightface.ai`
## Credits
- Original models: [InsightFace](https://github.com/deepinsight/insightface) by Jia Guo et al.
- SCRFD paper: [Sample and Computation Redistribution for Efficient Face Detection](https://arxiv.org/abs/2105.04714)
- ArcFace paper: [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://arxiv.org/abs/1801.07698)
|