File size: 1,855 Bytes
8bdd018 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | # Deploy Audio Flamingo 3 Caption Endpoint (Dedicated Endpoint)
Note: this guide is for the HF-converted `audio-flamingo-3-hf` runtime path.
For NVIDIA Space stack parity (`llava` + `stage35` think adapter), use:
`docs/deploy/AF3_NVIDIA_ENDPOINT.md`.
## 1) Create endpoint runtime repo
```bash
python scripts/hf_clone.py af3-endpoint --repo-id YOUR_USERNAME/YOUR_AF3_ENDPOINT_REPO
```
This pushes:
- `handler.py`
- `requirements.txt`
- `README.md`
from `templates/hf-af3-caption-endpoint/`.
## 2) Create endpoint from that model repo
In Hugging Face Endpoints:
1. Create endpoint from `YOUR_USERNAME/YOUR_AF3_ENDPOINT_REPO`.
2. Choose a GPU instance.
3. Set task to `custom`.
4. Set env vars:
- `AF3_MODEL_ID=nvidia/audio-flamingo-3-hf`
- `AF3_BOOTSTRAP_RUNTIME=1`
- `AF3_TRANSFORMERS_SPEC=transformers==5.1.0`
## 3) Validate startup
If logs contain:
- `No custom pipeline found at /repository/handler.py`
then `handler.py` is not in repo root. Re-upload the runtime template files.
If logs contain:
- `Failed to load AF3 processor classes after runtime bootstrap`
keep endpoint task as `custom`, then check that startup could install runtime deps (network + disk). The first cold start can take several minutes.
## 4) Connect from local pipeline
Set:
- `HF_AF3_ENDPOINT_URL`
- `HF_TOKEN`
- `OPENAI_API_KEY`
Recommended local `.env`:
```env
HF_AF3_ENDPOINT_URL=https://bc3r76slij67lskb.us-east-1.aws.endpoints.huggingface.cloud
HF_TOKEN=hf_xxx
OPENAI_API_KEY=sk-...
```
`.env` is git-ignored in this repo. Do not commit real credentials.
Then run:
```bash
python scripts/pipeline/run_af3_chatgpt_pipeline.py \
--audio ./train-dataset/track.mp3 \
--backend hf_endpoint \
--endpoint-url "$HF_AF3_ENDPOINT_URL" \
--openai-api-key "$OPENAI_API_KEY"
```
Or launch full GUI stack:
```bash
python af3_gui_app.py
```
|