Duplicate from qualcomm/First-Order-Motion-Model

Co-authored-by: QAIHM Bot <qaihm-bot@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+DEPLOYMENT_MODEL_LICENSE.pdf filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1 @@
+The license of the original trained model can be found at https://github.com/AliaksandrSiarohin/first-order-model/blob/master/LICENSE.md.

README.md

@@ -0,0 +1,82 @@
+---
+library_name: pytorch
+license: other
+tags:
+- android
+pipeline_tag: image-to-video
+
+---
+
+![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/fomm/web-assets/model_demo.png)
+
+# First-Order-Motion-Model: Optimized for Qualcomm Devices
+
+FOMM is a machine learning model that animates a still image to mirror the movements from a target video.
+
+This is based on the implementation of First-Order-Motion-Model found [here](https://github.com/AliaksandrSiarohin/first-order-model/tree/master).
+This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/fomm) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
+
+Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
+
+## Getting Started
+There are two ways to deploy this model on your device:
+
+### Option 1: Download Pre-Exported Models
+
+Below are pre-exported model assets ready for deployment.
+
+| Runtime | Precision | Chipset | SDK Versions | Download |
+|---|---|---|---|---|
+| ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/fomm/releases/v0.49.1/fomm-onnx-float.zip)
+
+For more device-specific assets and performance metrics, visit **[First-Order-Motion-Model on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/fomm)**.
+
+
+### Option 2: Export with Custom Configurations
+
+Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/fomm) Python library to compile and export the model with your own:
+- Custom weights (e.g., fine-tuned checkpoints)
+- Custom input shapes
+- Target device and runtime configurations
+
+This option is ideal if you need to customize the model beyond the default configuration provided here.
+
+See our repository for [First-Order-Motion-Model on GitHub](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/fomm) for usage instructions.
+
+## Model Details
+
+**Model Type:** Model_use_case.video_generation
+
+**Model Stats:**
+- Model checkpoint: vox-256
+- Input resolution: 256x256
+- Model size (FOMMDetector) (float): 54.2 MB
+- Model size (FOMMGenerator) (float): 174 MB
+
+## Performance Summary
+| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
+|---|---|---|---|---|---|---
+| FOMMDetector | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.75 ms | 0 - 24 MB | NPU
+| FOMMDetector | ONNX | float | Snapdragon® X2 Elite | 2.777 ms | 28 - 28 MB | NPU
+| FOMMDetector | ONNX | float | Snapdragon® X Elite | 4.579 ms | 27 - 27 MB | NPU
+| FOMMDetector | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 3.275 ms | 0 - 32 MB | NPU
+| FOMMDetector | ONNX | float | Qualcomm® QCS9075 | 5.8 ms | 1 - 4 MB | NPU
+| FOMMDetector | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.925 ms | 0 - 21 MB | NPU
+| FOMMGenerator | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 10.76 ms | 0 - 196 MB | NPU
+| FOMMGenerator | ONNX | float | Snapdragon® X2 Elite | 12.262 ms | 91 - 91 MB | NPU
+| FOMMGenerator | ONNX | float | Snapdragon® X Elite | 29.148 ms | 89 - 89 MB | NPU
+| FOMMGenerator | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 17.31 ms | 3 - 222 MB | NPU
+| FOMMGenerator | ONNX | float | Qualcomm® QCS9075 | 34.708 ms | 18 - 22 MB | NPU
+| FOMMGenerator | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 13.265 ms | 8 - 196 MB | NPU
+
+## License
+* The license for the original implementation of First-Order-Motion-Model can be found
+  [here](https://github.com/AliaksandrSiarohin/first-order-model/blob/master/LICENSE.md).
+
+## References
+* [First Order Motion Model for Image Animation](https://arxiv.org/abs/2003.00196)
+* [Source Model Implementation](https://github.com/AliaksandrSiarohin/first-order-model/tree/master)
+
+## Community
+* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
+* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).