| # VoiceGate HF Space TODO |
|
|
| This is the execution checklist for bringing the VoiceGate Hugging Face Space |
| from scaffold to a working Gradio app. |
|
|
| ## Current Status |
|
|
| - [x] Create the `VoiceGate-hf` Space wrapper repository. |
| - [x] Configure and push to `build-small-hackathon/VoiceGate`. |
| - [x] Replace the default HF template with a VoiceGate scaffold. |
| - [x] Keep the upstream `VoiceGate/` checkout local-only and ignored. |
| - [x] Preserve Hugging Face LFS rules in `.gitattributes`. |
| - [x] Confirm `VoiceGate/workflows/VoiceGate-Workflow_api.json` is valid JSON. |
| - [x] Copy the validated API workflow into `workflows/voicegate_api.json`. |
| - [x] Confirm SSH access to the running Space container and document the |
| runbook. |
| - [x] Confirm `DEEPSEEK_API_KEY` is visible in the Space without printing it. |
| - [x] Download VoxCPM2 and MelBand RoFormer to persistent Space storage. |
| - [x] Confirm ZeroGPU CUDA can be invoked from the normal Gradio runtime through |
| `@spaces.GPU`. |
| - [x] Confirm the full short-audio workflow can run on the duplicated personal |
| T4 Small Space. |
|
|
| ## Phase 1: Repository Hygiene |
|
|
| - [x] Copy `VoiceGate/workflows/VoiceGate-Workflow_api.json` to |
| `workflows/voicegate_api.json`. |
| - [x] Copy `VoiceGate/workflows/VoiceGate-Workflow.json` to |
| `workflows/voicegate_ui.json` for reference. |
| - [x] Update docs so `VoiceGate/` is clearly documented as a local upstream |
| checkout, not Space runtime content. |
| - [x] Commit and push the workflow files after confirming they contain no |
| secrets or large media. |
|
|
| ## Phase 2: Dependency Inventory |
|
|
| - [x] Identify exact repositories and pinned commits for every required custom |
| node package: |
| - [x] `comfyui_voicebridge` |
| - [x] RunningHub VoxCPM nodes |
| - [x] MelBandRoFormer nodes |
| - [x] rgthree nodes |
| - [x] comfyui-easy-use |
| - [x] Comfyroll text nodes or equivalent provider for `CR Text` |
| - [x] `RH_LLMAPI_NODE` provider |
| - [x] `ReplaceText` provider |
| - [x] `MergeAudioMW` provider |
| - [x] Identify Python dependency constraints for ComfyUI and all custom nodes. |
| - [x] Identify system packages beyond `ffmpeg` and `git`, if any. |
| - [x] Decide whether custom nodes are vendored in `custom_nodes/` or installed |
| by pinned git URL during bootstrap. |
| - [x] Decide where model files are downloaded and cached in the Space. |
|
|
| ## Phase 3: Runtime Bootstrap |
|
|
| - [x] Add `scripts/bootstrap_comfy.py`. |
| - [x] Add `scripts/run_comfy.py`. |
| - [x] Add `scripts/workflow_client.py`. |
| - [ ] Install or prepare ComfyUI on Space startup. |
| - [x] Add bootstrap support for installing custom node dependencies. |
| - [x] Add opt-in model directory preparation and download commands. |
| - [x] Verify ComfyUI can start locally in the Space in CPU mode. |
| - [x] Verify ComfyUI can start from a Gradio `@spaces.GPU` function and report |
| CUDA through `/system_stats`. |
| - [x] Verify `ComfyUI_AudioTools` imports successfully in the Space after |
| adding its missing system and Python dependencies. |
| - [ ] Verify ComfyUI API endpoints are reachable: |
| - [x] `/system_stats` |
| - [x] `/upload/image` or the audio upload equivalent used by `LoadAudio` |
| - [x] `/prompt` |
| - [x] `/history/{prompt_id}` |
|
|
| ## Phase 4: Workflow Parameterization |
|
|
| - [x] Add Python-side patching for node `16` `LoadAudio.inputs.audio`. |
| - [x] Add Python-side patching for node `105` `api_key` from |
| `DEEPSEEK_API_KEY`. |
| - [x] Add Python-side patching for node `105` `api_baseurl`. |
| - [x] Add Python-side patching for node `105` `model`. |
| - [x] Add Python-side patching for node `110` target language. |
| - [x] Add unique job-specific output prefixes for node `180` and node `214`. |
| - [x] Decide which user controls are exposed first: |
| - [ ] source language |
| - [x] target language |
| - [ ] LLM model |
| - [ ] max input duration |
| - [ ] keep or drop background audio |
| - [ ] Remove or ignore display-only `easy showAnything` nodes if they are not |
| needed for API execution. |
|
|
| ## Phase 5: Gradio Interface |
|
|
| - [x] Replace the placeholder `app.py` with the first VoiceGate interface. |
| - [x] Add short audio upload input. |
| - [x] Add target language input. |
| - [x] Add status/log output. |
| - [x] Add generated audio output. |
| - [x] Add generated SRT file outputs. |
| - [ ] Wrap GPU-heavy execution with `@spaces.GPU(duration=...)`. |
| - [x] Add and run a minimal GPU smoke test button. |
| - [x] Keep a diagnostics tab for internal workflow tests. |
| - [x] Add automatic model-path preparation for the user-facing run path. |
| - [x] Add user-facing TTS segment trim control for node `268` |
| `TrimAudioDuration.start_index`. |
| - [ ] Add guardrails for file type and duration. |
|
|
| ## Phase 6: Minimal Runtime Tests |
|
|
| - [x] Start Space and confirm Gradio loads. |
| - [x] Start ComfyUI from the Space process. |
| - [x] Submit a minimal prompt to ComfyUI and receive a response. |
| - [x] Submit a minimal prompt to ComfyUI from the Gradio GPU runtime and receive |
| a response. |
| - [x] Run a DeepSeek LLM node smoke test. |
| - [x] Run a MelBand RoFormer smoke test. |
| - [x] Run a MelBand RoFormer smoke test inside ZeroGPU. |
| - [x] Run a tiny TTS-only workflow inside ZeroGPU. |
| - [x] Run a short ASR-only workflow inside ZeroGPU. |
| - [x] Verify the Space API workflow connection graph against |
| `VoiceGate/workflows/VoiceGate-Workflow.json`. |
| - [ ] Run SRT split -> VoxCPM -> SRT merge. |
| - [x] Run the full short-audio VoiceGate workflow on the personal T4 Small |
| Space. |
| - [ ] Run the full short-audio VoiceGate workflow on the organization ZeroGPU |
| Space after quota recovers. |
| - [x] Confirm output audio is downloadable/playable from Gradio. |
| - [ ] Confirm SRT files are downloadable from Gradio after redeploy. |
|
|
| ## Phase 7: Full VoiceGate Path |
|
|
| - [ ] Add video input support after the audio path is stable. |
| - [ ] Extract audio from video with `ffmpeg`. |
| - [ ] Merge generated audio back into the original video if required. |
| - [ ] Add subtitle download and optional subtitle burn-in. |
| - [ ] Add examples with small sample files, if allowed by Space storage limits. |
|
|
| ## Open Questions |
|
|
| - [x] Which exact node repository provides `RH_LLMAPI_NODE`? |
| - [x] Which exact node repository provides `RunningHub_VoxCPM_*`? |
| - [ ] Is `flash_attention_2` available and reliable in the ZeroGPU environment? |
| - [x] Can ASR run without `flash_attention_2`? Yes. The ASR-only smoke test |
| used `attention=sdpa`. |
| - [ ] Does VoxCPM2 fit comfortably in ZeroGPU memory with ASR and |
| MelBandRoFormer in the same run? A full run reached the heavy nodes but timed |
| out at the `800s` app wait window; the next shortened retry was blocked by |
| ZeroGPU quota before execution. |
| - [x] Can a full short-audio workflow run on T4 Small? Yes, on the duplicated |
| personal Space with a warm ComfyUI process, the test completed in `24.4s`. |
| - [x] How do we avoid user-facing failure after hardware changes? The user path |
| now checks required model paths and runs `bootstrap_comfy.py --with-models` |
| before submitting to ComfyUI. |
| - [x] Where should large model files live? `/data/voicegate_models`, with |
| symlinks into ComfyUI's expected model directories. |
| - [ ] Should the first public demo disable background separation to reduce |
| runtime and memory pressure? |
| - [ ] What maximum uploaded audio/video duration should the first version allow? |
| - [x] Can SSH access the ZeroGPU CUDA device directly? No. SSH enters the |
| normal running Space container without CUDA; GPU work must run inside a |
| `@spaces.GPU` function. |
| - [x] Has the Space successfully invoked ZeroGPU CUDA from Gradio? Yes. The |
| `gpu_smoke_test` button returned `cuda_available=True` on one CUDA device. |
|
|
