docs/ssh-runbook.md

Hugging Face Space SSH Runbook

This runbook records how to connect to the running Hugging Face Space container for debugging.

Space

https://huggingface.co/spaces/build-small-hackathon/VoiceGate

SSH target:

build-small-hackathon-voicegate@ssh.hf.space

Local SSH Key

The local private key generated for this Space is:

C:\Users\yantianlong\.ssh\codex_space_voicegate

The public key is:

C:\Users\yantianlong\.ssh\codex_space_voicegate.pub

Only the .pub file content should be added to Hugging Face. Never upload or share the private key file.

Connect

From PowerShell:

ssh -i "$env:USERPROFILE\.ssh\codex_space_voicegate" build-small-hackathon-voicegate@ssh.hf.space

Non-interactive smoke test:

ssh -i "$env:USERPROFILE\.ssh\codex_space_voicegate" `
  -o BatchMode=yes `
  -o StrictHostKeyChecking=accept-new `
  -o ConnectTimeout=20 `
  build-small-hackathon-voicegate@ssh.hf.space `
  "pwd && git rev-parse HEAD && ls -la"

Expected working directory:

/home/user/app

Important Runtime Caveat

SSH connects to the currently running Space container. That container may be stale after a git push.

On 2026-06-05, SSH connected successfully, but the running container was still at the original template commit:

a94117f35a42cb17f654ae70cbe619c15345d057

The Space git remote had already advanced to:

7acf781e730d4b84d40cf3c688a1575b11bd90aa

Before debugging runtime behavior through SSH, always verify the container commit:

git rev-parse HEAD

If the container commit does not match the latest Space commit, use the Hugging Face Space page to restart/rebuild/factory reboot the Space, then SSH again and re-check.

What SSH Is Good For

• Inspect the running app files.
• Check build/runtime environment differences.
• Run small diagnostic commands.
• Inspect installed packages and model cache paths.
• Debug ComfyUI startup logs once Phase 3 is implemented.
• Run ComfyUI in CPU mode to verify the local API can start:

cd /home/user/app
python scripts/bootstrap_comfy.py
python scripts/run_comfy.py --cpu

In another SSH command, check:

curl http://127.0.0.1:8188/system_stats

What SSH Is Not For

Do not rely on SSH edits for persistent code changes. Files changed inside the running container can be lost after rebuild or restart.

Persistent changes should be made locally, committed, and pushed to the Space git repository.

SSH also does not expose the ZeroGPU CUDA device in the normal shell. On 2026-06-05, starting ComfyUI without --cpu through SSH failed with:

RuntimeError: No CUDA GPUs are available

GPU-backed inference should be triggered from the Gradio app through a @spaces.GPU function. SSH remains useful for CPU startup and API diagnostics.

Runtime Findings

On 2026-06-05, python scripts/bootstrap_comfy.py completed successfully in the Space container. CPU-mode ComfyUI reached /system_stats, but ComfyUI_AudioTools required additional dependencies:

• system package sox
• system package libportaudio2
• system package portaudio19-dev
• Python package sounddevice
• Python package easydict
• Python package pytorch-lightning

These are now recorded in packages.txt and requirements.txt; a Space rebuild is required before the system packages are available in the container.

After adding these dependencies, CPU-mode ComfyUI reached /system_stats and the custom node import log showed:

0.4 seconds: /home/user/app/ComfyUI/custom_nodes/ComfyUI_AudioTools

with no IMPORT FAILED entry for ComfyUI_AudioTools.

On 2026-06-05, the ComfyUI API smoke test passed in Dev Mode using a local MP3 uploaded to /tmp/voicegate_test_audio.mp3:

• /upload/image accepted the MP3 and returned voicegate_test_audio.mp3.
• A minimal LoadAudio -> SaveAudioMP3 workflow submitted through /prompt returned no node_errors.
• /history/{prompt_id} returned status_str: success.
• The output was reported as audio/api_smoke_voicegate_00001.mp3.

When uploading local binary files through SSH, avoid plain PowerShell byte pipelines because they can corrupt binary data. Use a binary-safe method such as base64 from Python and verify sha256sum on the remote file.