|
|
--- |
|
|
title: VidEmbed |
|
|
emoji: 💻 |
|
|
colorFrom: purple |
|
|
colorTo: blue |
|
|
sdk: gradio |
|
|
sdk_version: 5.49.0 |
|
|
app_file: app.py |
|
|
pinned: false |
|
|
license: cc |
|
|
short_description: generate embeddings of youtube video text and image data |
|
|
--- |
|
|
|
|
|
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference |
|
|
|
|
|
# Hugging Face Space for multimodal embeddings |
|
|
|
|
|
Files: |
|
|
- app.py |
|
|
- requirements.txt |
|
|
- finetuned_multimodal.pt <-- upload your 3GB checkpoint here |
|
|
|
|
|
How to deploy: |
|
|
1. Create a new Space on Hugging Face (https://huggingface.co/spaces). |
|
|
- SDK: Gradio |
|
|
- Hardware: If you need GPU inference, switch to a GPU runtime (note: GPU access may require a paid plan). |
|
|
2. Upload these files to the Space repository. For a large checkpoint (~3GB) use Git LFS or the web UI file upload (the web UI supports large uploads). |
|
|
3. Wait for the Space to build. The app serves: |
|
|
- Web UI at `/` (Gradio) |
|
|
- API endpoint at `/api/get_embedding` |
|
|
|
|
|
API example: |
|
|
POST JSON to `/api/get_embedding`: |
|
|
{ |
|
|
"title": "My video", |
|
|
"description": "Some description", |
|
|
"tags": "tag1,tag2", |
|
|
"thumbnail_url": "https://..." |
|
|
} |
|
|
|
|
|
Response: |
|
|
{ |
|
|
"embedding": [0.123, -0.456, ...] # fused vector used in training |
|
|
} |
|
|
|
|
|
Notes: |
|
|
- The app replicates the same fused embedding pipeline used in your notebook: |
|
|
text -> text encoder -> text_proj |
|
|
thumbnail -> image encoder -> img_proj |
|
|
fused = MultiheadAttention(query=text_proj, key=img_proj, value=img_proj) |
|
|
fused is returned as the embedding vector. |
|
|
- If your checkpoint uses different key names, the loader prints a warning; update loader accordingly. |
|
|
- If your model requires different shapes (text_dim/img_dim/proj_dim) adjust MultimodalRegressor init params. |
|
|
|
