Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,95 @@
+# SAM-Audio ONNX (Large)
+
+ONNX-converted models for [SAM-Audio](https://github.com/facebookresearch/sam-audio) (facebook/sam-audio-large) - Meta's Semantic Audio Modeling for audio source separation.
+
+## Model Files
+
+| File | Description | Size |
+|------|-------------|------|
+| `dacvae_encoder.onnx` | Audio encoder (48kHz → latent) | ~110 MB |
+| `dacvae_decoder.onnx` | Audio decoder (latent → 48kHz) | ~320 MB |
+| `t5_encoder.onnx` | Text encoder (T5-base) | ~440 MB |
+| `dit_single_step.onnx` | DiT denoiser (single ODE step) | ~2 GB |
+| `vision_encoder.onnx` | Vision encoder (CLIP-based) | ~1.2 GB |
+| `tokenizer/` | SentencePiece tokenizer files | - |
+
+## Installation
+
+```bash
+pip install onnxruntime sentencepiece torchaudio torchvision torchcodec soundfile
+# For CUDA support:
+pip install onnxruntime-gpu
+```
+
+## Quick Start
+
+```python
+import numpy as np
+import onnxruntime as ort
+from huggingface_hub import hf_hub_download
+
+# Download models
+model_dir = "sam-audio-large-onnx"
+for f in ["dacvae_encoder.onnx", "dacvae_decoder.onnx", "t5_encoder.onnx", 
+          "dit_single_step.onnx", "vision_encoder.onnx"]:
+    hf_hub_download("matbee/sam-audio-large-onnx", f, local_dir=model_dir)
+    if f != "vision_encoder.onnx":  # vision encoder embeds weights
+        hf_hub_download("matbee/sam-audio-large-onnx", f + ".data", local_dir=model_dir)
+```
+
+## Usage Examples
+
+### Audio-Only Separation
+```bash
+python onnx_inference.py \
+    --audio input.wav \
+    --text "a person speaking" \
+    --output separated.wav
+```
+
+### Video-Guided Separation
+```bash
+python onnx_inference.py \
+    --video input.mp4 \
+    --text "the sound of typing" \
+    --output separated.wav
+```
+
+### Visual Prompting with SAM3 Mask
+```bash
+# First generate a mask with SAM3 (see generate_sam3_mask.py)
+python onnx_inference.py \
+    --video input.mp4 \
+    --mask object_mask.mp4 \
+    --text "" \
+    --output isolated.wav \
+    --output-video visualization.mp4
+```
+
+## Model Details
+
+- **Audio Sample Rate**: 48kHz
+- **Audio Hop Length**: 1536 samples
+- **Vision Input Size**: 336×336 pixels
+- **Text Encoder**: T5-base (768-dim)
+- **Vision Encoder**: PE-Core-L14-336 (1024-dim)
+- **ODE Solver**: Midpoint method (configurable steps)
+
+## License
+
+SAM-Audio is released under the [CC-BY-NC 4.0 license](https://creativecommons.org/licenses/by-nc/4.0/).
+
+## Citation
+
+```bibtex
+@article{samaudio2024,
+  title={SAM-Audio: Semantic Audio Modeling},
+  author={Meta AI},
+  year={2024}
+}
+```
+
+## Acknowledgments
+
+Original model by [Meta AI Research](https://github.com/facebookresearch/sam-audio).
+ONNX conversion by [@matbee](https://huggingface.co/matbee).