| --- |
| license: other |
| library_name: pytorch |
| pipeline_tag: image-feature-extraction |
| tags: |
| - fingerprint |
| - biometric-matching |
| - fastvit |
| - onnx |
| - qualcomm-ai-hub |
| --- |
| |
| # MDGT Matching - FastVIT Cross-Sensor Checkpoint |
|
|
| This repository currently points to the FastVIT cross-sensor branch, not the older MDGT/NIST300A checkpoint. |
|
|
| Primary checkpoint: |
|
|
| ```text |
| checkpoints_fastvit_cross_sensor_sa12_round5_consistency_light/best_rank1.pt |
| ``` |
|
|
| ## Model |
|
|
| - Architecture: `FastVITGraph` |
| - Backbone: `fastvit_sa12.apple_in1k` |
| - Feature index: `1` |
| - Token selection: TRAM |
| - Token count: `64` |
| - GNN: 2 layers, dim 256, 4 heads, k=8 |
| - Embedding dim: `256` |
| - Input: preprocessed grayscale tensor `(B, 1, 224, 224)` |
| - Preprocess used for export/eval: squeeze resize, CLAHE on, Gabor off |
|
|
| ## Metrics |
|
|
| These metrics are from the checkpoint metadata for the full NIST302 cross-sensor 1:N protocol: |
|
|
| - Probe: NIST302A challengers, 13,630 probes |
| - Reference: NIST302B baseline, 8,000 gallery images |
| - Identities: 2,000 |
|
|
| | Metric | Value | |
| |---|---:| |
| | Rank-1 | 0.8133528829 | |
| | Rank-5 | 0.9174614549 | |
| | Rank-10 | 0.9479089975 | |
| | mAP | 0.7901349983 | |
| | AUC | 0.9983431867 | |
| | EER | 0.0178200894 | |
|
|
| The earlier low `Rank-1: 0.390625` result belonged to an MDGT/NIST300A QAI smoke benchmark on a 64-identity spread subset. It is not the metric for this FastVIT checkpoint. |
|
|
| ## Files |
|
|
| ```text |
| pytorch/fastvit_cross_sensor_sa12_round5_consistency_light_best_rank1.pt |
| onnx/fastvit_cross_sensor_embedding_fp32.onnx |
| onnx/fastvit_cross_sensor_embedding_fp16.onnx |
| onnx/fastvit_cross_sensor_embedding_int8_linear_qdq.onnx |
| onnx/export_summary.json |
| qai/fastvit_cross_sensor_qai_target_model.dlc |
| qai/qai_qnn_context_binary_summary.json |
| qai/qai_qnn_dlc_summary.json |
| qai/qai_qnn_lib_attempt_summary.json |
| results/checkpoint_metrics.json |
| training/history.jsonl |
| artifact_metadata.json |
| ``` |
|
|
| ## ONNX Export |
|
|
| The ONNX export uses an export-safe FastVITGraph wrapper with vectorized deterministic token deduplication. |
|
|
| Observed local drift on 16 samples: |
|
|
| | Comparison | Cos mean | Cos min | |
| |---|---:|---:| |
| | Original PyTorch vs exportable PyTorch | 1.0000000000 | 0.9999999404 | |
| | Exportable PyTorch vs ONNX FP32 | 0.9999986887 | 0.9999958277 | |
| | ONNX FP16 vs ONNX FP32 | 0.9990564585 | 0.9972373843 | |
| | ONNX INT8 linear QDQ vs ONNX FP32 | 0.9941318631 | 0.9882222414 | |
|
|
| ## Qualcomm AI Hub |
|
|
| QAI test was rerun on the FastVIT ONNX, not on MDGT. |
|
|
| - `qnn_context_binary`: compile failed with exit code 14. |
| - `qnn_lib_aarch64_android`: unsupported by the installed QAI Hub client. |
| - `qnn_dlc`: compile succeeded, target DLC was produced. |
| - NPU profile/inference for the DLC failed with `MODEL_GRAPH_ERROR` from `QnnModel_composeGraphsFromDlc`. |
|
|
| Jobs: |
|
|
| ```text |
| qnn_context_binary compile: https://workbench.aihub.qualcomm.com/jobs/jgjwl2v75/ |
| qnn_dlc compile: https://workbench.aihub.qualcomm.com/jobs/jgdzvqxk5/ |
| qnn_dlc profile: https://workbench.aihub.qualcomm.com/jobs/jp847m2z5/ |
| qnn_dlc inference: https://workbench.aihub.qualcomm.com/jobs/jgz47j2zp/ |
| ``` |
|
|
| This means the current full FastVITGraph ONNX can be exported and compiled to DLC, but the graph is not yet deployable as a working NPU profile/inference artifact on QAI Hub. The likely next step is to simplify or split the dynamic token/GNN section for NPU compatibility. |
|
|
