Video-MAE: Optimized for Qualcomm Devices

Video MAE (Masked Auto Encoder) is a network for doing video classification that uses the ViT (Vision Transformer) backbone.

This is based on the implementation of Video-MAE found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.50, ONNX Runtime 1.30.0	Download
ONNX	w8a16	Universal	QAIRT 2.50, ONNX Runtime 1.30.0	Download
QNN_DLC	float	Universal	QAIRT 2.50	Download
QNN_DLC	w8a16	Universal	QAIRT 2.50	Download
TFLITE	float	Universal	QAIRT 2.50	Download

For more device-specific assets and performance metrics, visit Video-MAE on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

• Custom weights (e.g., fine-tuned checkpoints)
• Custom input shapes
• Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Video-MAE on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.video_classification

Model Stats:

• Input resolution: 224x224
• Model checkpoint: Kinectics-400
• Model size (float): 335 MB
• Number of parameters: 87.7M

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
Video-MAE	ONNX	float	Snapdragon® 8 Elite Gen 5 For Galaxy Mobile	291.368 ms	1 - 924 MB	NPU
Video-MAE	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	311.801 ms	1 - 692 MB	NPU
Video-MAE	ONNX	float	Snapdragon® X2 Elite	291.862 ms	9 - 9 MB	NPU
Video-MAE	ONNX	float	Snapdragon® X Elite	564.484 ms	188 - 188 MB	NPU
Video-MAE	ONNX	float	Snapdragon® 8 Gen 3 Mobile	402.734 ms	0 - 984 MB	NPU
Video-MAE	ONNX	float	Snapdragon® 8 Gen 1 Mobile	580.81 ms	10 - 886 MB	NPU
Video-MAE	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	545.358 ms	9 - 22 MB	NPU
Video-MAE	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	549.145 ms	0 - 217 MB	NPU
Video-MAE	ONNX	float	Qualcomm® QCS8450	580.81 ms	10 - 886 MB	NPU
Video-MAE	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	568.856 ms	9 - 21 MB	NPU
Video-MAE	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	564.484 ms	188 - 188 MB	NPU
Video-MAE	ONNX	float	Qualcomm® Dragonwing™ Q-8750	311.801 ms	1 - 692 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® 8 Elite Gen 5 For Galaxy Mobile	280.264 ms	1 - 1040 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® 8 Elite For Galaxy Mobile	278.387 ms	0 - 1035 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® X2 Elite	288.213 ms	5 - 5 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® X Elite	491.612 ms	102 - 102 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® 8 Gen 3 Mobile	373.939 ms	5 - 1252 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® 8 Gen 1 Mobile	524.984 ms	5 - 1290 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ QCS6490	1208.081 ms	0 - 8 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ IQ-8275	488.846 ms	1 - 10 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ QCS8550 (Proxy)	481.74 ms	0 - 110 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® QCS8450	524.984 ms	5 - 1290 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ IQ-9075	476.126 ms	0 - 8 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ IQ-X7181	491.612 ms	102 - 102 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ Q-7790	604.154 ms	5 - 1218 MB	NPU
Video-MAE	ONNX	w8a16	Qualcomm® Dragonwing™ Q-8750	278.387 ms	0 - 1035 MB	NPU
Video-MAE	ONNX	w8a16	Snapdragon® 7 Gen 4 Mobile	604.154 ms	5 - 1218 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 For Galaxy Mobile	258.292 ms	9 - 687 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® 8 Elite For Galaxy Mobile	268.85 ms	9 - 588 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® X2 Elite	273.807 ms	9 - 9 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® X Elite	506.932 ms	9 - 9 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	382.871 ms	10 - 850 MB	NPU
Video-MAE	QNN_DLC	float	Snapdragon® 8 Gen 1 Mobile	712.746 ms	9 - 827 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-8275	492.684 ms	9 - 21 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	486.779 ms	10 - 12 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® SA8775P	521.248 ms	1 - 604 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® SA8650P	521.248 ms	1 - 604 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® SA8255P	521.248 ms	1 - 604 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® QCS8450	712.746 ms	9 - 827 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-9075	516.283 ms	11 - 22 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-X7181	506.932 ms	9 - 9 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® Dragonwing™ Q-8750	268.85 ms	9 - 588 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® SA7255P	1067.466 ms	2 - 628 MB	NPU
Video-MAE	QNN_DLC	float	Qualcomm® SA8295P	702.906 ms	0 - 544 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® 8 Elite Gen 5 For Galaxy Mobile	531.954 ms	4 - 726 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® 8 Elite For Galaxy Mobile	294.629 ms	5 - 709 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® X2 Elite	493.158 ms	5 - 5 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® X Elite	492.717 ms	5 - 5 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® 8 Gen 3 Mobile	502.403 ms	5 - 966 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ IQ-8275	602.081 ms	5 - 12 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ QCS8550 (Proxy)	487.733 ms	5 - 8 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® SA8775P	618.976 ms	1 - 751 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® SA8650P	618.976 ms	1 - 751 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® SA8255P	618.976 ms	1 - 751 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ IQ-9075	612.155 ms	5 - 11 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ IQ-X7181	492.717 ms	5 - 5 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ Q-7790	743.358 ms	5 - 887 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® Dragonwing™ Q-8750	294.629 ms	5 - 709 MB	NPU
Video-MAE	QNN_DLC	w8a16	Qualcomm® SA7255P	1151.425 ms	1 - 750 MB	NPU
Video-MAE	QNN_DLC	w8a16	Snapdragon® 7 Gen 4 Mobile	743.358 ms	5 - 887 MB	NPU
Video-MAE	TFLITE	float	Snapdragon® 8 Elite Gen 5 For Galaxy Mobile	59.891 ms	0 - 655 MB	NPU
Video-MAE	TFLITE	float	Snapdragon® 8 Elite For Galaxy Mobile	82.613 ms	0 - 588 MB	NPU
Video-MAE	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	100.826 ms	0 - 1012 MB	NPU
Video-MAE	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	209.305 ms	0 - 839 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	162.477 ms	0 - 207 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	139.059 ms	0 - 3 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® SA8775P	161.436 ms	0 - 572 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® SA8650P	161.436 ms	0 - 572 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® SA8255P	161.436 ms	0 - 572 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® QCS8450	209.305 ms	0 - 839 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	159.598 ms	0 - 206 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	82.613 ms	0 - 588 MB	NPU
Video-MAE	TFLITE	float	Qualcomm® SA8295P	209.723 ms	0 - 671 MB	NPU

License

• The license for the original implementation of Video-MAE can be found here.

References

• Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
• Source Model Implementation

Community

• Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
• For questions or feedback please reach out to us.