README.md

metadata

title: VidEmbed
emoji: 💻
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.49.0
app_file: app.py
pinned: false
license: cc
short_description: generate embeddings of youtube video text and image data

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

Hugging Face Space for multimodal embeddings

Files:

• app.py
• requirements.txt
• finetuned_multimodal.pt <-- upload your 3GB checkpoint here

How to deploy:

1. Create a new Space on Hugging Face (https://huggingface.co/spaces).
   • SDK: Gradio
   • Hardware: If you need GPU inference, switch to a GPU runtime (note: GPU access may require a paid plan).
2. Upload these files to the Space repository. For a large checkpoint (~3GB) use Git LFS or the web UI file upload (the web UI supports large uploads).
3. Wait for the Space to build. The app serves:
   • Web UI at / (Gradio)
   • API endpoint at /api/get_embedding

API example: POST JSON to /api/get_embedding: { "title": "My video", "description": "Some description", "tags": "tag1,tag2", "thumbnail_url": "https://..." }

Response: { "embedding": [0.123, -0.456, ...] # fused vector used in training }

Notes:

• The app replicates the same fused embedding pipeline used in your notebook: text -> text encoder -> text_proj thumbnail -> image encoder -> img_proj fused = MultiheadAttention(query=text_proj, key=img_proj, value=img_proj) fused is returned as the embedding vector.
• If your checkpoint uses different key names, the loader prints a warning; update loader accordingly.
• If your model requires different shapes (text_dim/img_dim/proj_dim) adjust MultimodalRegressor init params.