Update README.md
Browse files
README.md
CHANGED
|
@@ -21,13 +21,15 @@ flowchart TD
|
|
| 21 |
Dashboard["Gradio Operator Dashboard"] --> Jobs
|
| 22 |
Jobs --> Engine["Render Engine"]
|
| 23 |
Engine --> Assets["Asset Probe + Metadata Cache"]
|
|
|
|
| 24 |
Engine --> Normalize["Normalization"]
|
| 25 |
Engine --> Scenes["Scene Timeline"]
|
| 26 |
Engine --> Subtitles["SRT / ASS Subtitle Engine"]
|
| 27 |
Engine --> Transitions["Transition Filter Builder"]
|
| 28 |
Engine --> Audio["Voiceover + Ducking Mixer"]
|
| 29 |
Engine --> Exports["Export Manager"]
|
| 30 |
-
|
|
|
|
| 31 |
Scenes --> FFmpeg
|
| 32 |
Subtitles --> FFmpeg
|
| 33 |
Transitions --> FFmpeg
|
|
@@ -38,7 +40,7 @@ Package layout:
|
|
| 38 |
|
| 39 |
```text
|
| 40 |
renderer/
|
| 41 |
-
core/ settings, models, orchestration
|
| 42 |
ffmpeg/ command builder, runner, probing, normalization
|
| 43 |
subtitles/ SRT and ASS generation
|
| 44 |
transitions/ xfade graph generation
|
|
@@ -51,7 +53,7 @@ renderer/
|
|
| 51 |
|
| 52 |
## REST API
|
| 53 |
|
| 54 |
-
All API endpoints are served at the Space root.
|
| 55 |
|
| 56 |
### `POST /render`
|
| 57 |
|
|
@@ -61,15 +63,23 @@ Submit one render job.
|
|
| 61 |
{
|
| 62 |
"template": "tiktok_classic",
|
| 63 |
"output_name": "campaign_clip.mp4",
|
| 64 |
-
"voiceover": "/
|
| 65 |
-
"background_music": "/
|
| 66 |
"subtitle_format": "ass",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
"normalize": true,
|
| 68 |
"scenes": [
|
| 69 |
{
|
| 70 |
"start": 0,
|
| 71 |
"duration": 5,
|
| 72 |
-
"media": "/
|
| 73 |
"caption": "Launch faster with automated rendering",
|
| 74 |
"transition": "fade"
|
| 75 |
}
|
|
@@ -83,10 +93,14 @@ Response:
|
|
| 83 |
{
|
| 84 |
"job_id": "job_abc123",
|
| 85 |
"status_url": "/status/job_abc123",
|
| 86 |
-
"download_url": "/download/job_abc123"
|
| 87 |
}
|
| 88 |
```
|
| 89 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
### `POST /render/ai-reels`
|
| 91 |
|
| 92 |
Submit a single-call AI Reels job using an existing voiceover and asset list. TTS is intentionally provider-pluggable in v1; the production path requires a supplied voiceover.
|
|
@@ -94,8 +108,8 @@ Submit a single-call AI Reels job using an existing voiceover and asset list. TT
|
|
| 94 |
```json
|
| 95 |
{
|
| 96 |
"script": "Launch faster with automated rendering.",
|
| 97 |
-
"voiceover": "/
|
| 98 |
-
"assets": ["/
|
| 99 |
"template": "youtube_shorts",
|
| 100 |
"output_name": "ai_reel.mp4"
|
| 101 |
}
|
|
@@ -111,8 +125,116 @@ Submit multiple render jobs. The worker pool defaults to one active render to av
|
|
| 111 |
{
|
| 112 |
"template": "modern_minimal",
|
| 113 |
"output_name": "clip_a.mp4",
|
| 114 |
-
"scenes": [{"start": 0, "duration": 3, "media": "/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 115 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
]
|
| 117 |
}
|
| 118 |
```
|
|
@@ -125,11 +247,46 @@ States: `PENDING`, `RUNNING`, `FAILED`, `COMPLETED`.
|
|
| 125 |
|
| 126 |
### `GET /download/{job_id}`
|
| 127 |
|
| 128 |
-
Returns the completed MP4 deliverable. If the job is still running, the endpoint returns `409`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
|
| 130 |
### `POST /inspect`
|
| 131 |
|
| 132 |
-
Probe one asset path and return MIME type, duration, codecs, bitrate, resolution, FPS, stream list, and cache metadata.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
|
| 134 |
## Operator Dashboard
|
| 135 |
|
|
@@ -156,10 +313,19 @@ Templates are selected per request and do not require code changes.
|
|
| 156 |
- Use `OUTPUT_CRF=23-28` to balance quality and file size.
|
| 157 |
- Keep source assets near the target duration to reduce normalization work.
|
| 158 |
- Prefer pre-trimmed voiceovers and assets for batch workloads.
|
|
|
|
|
|
|
|
|
|
| 159 |
|
| 160 |
## Reliability
|
| 161 |
|
| 162 |
- FFmpeg subprocesses are killed after `FFMPEG_TIMEOUT_SECONDS`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
- Jobs retry up to `MAX_RETRIES`.
|
| 164 |
- Intermediate files are created under `TEMP_DIR` and removed after export.
|
| 165 |
- Job records retain command history, logs, render time, output size, and failure reasons.
|
|
|
|
| 21 |
Dashboard["Gradio Operator Dashboard"] --> Jobs
|
| 22 |
Jobs --> Engine["Render Engine"]
|
| 23 |
Engine --> Assets["Asset Probe + Metadata Cache"]
|
| 24 |
+
Engine --> Ingest["Remote URL + Upload Ingestion"]
|
| 25 |
Engine --> Normalize["Normalization"]
|
| 26 |
Engine --> Scenes["Scene Timeline"]
|
| 27 |
Engine --> Subtitles["SRT / ASS Subtitle Engine"]
|
| 28 |
Engine --> Transitions["Transition Filter Builder"]
|
| 29 |
Engine --> Audio["Voiceover + Ducking Mixer"]
|
| 30 |
Engine --> Exports["Export Manager"]
|
| 31 |
+
Ingest --> FFmpeg["FFmpeg / FFprobe"]
|
| 32 |
+
Normalize --> FFmpeg
|
| 33 |
Scenes --> FFmpeg
|
| 34 |
Subtitles --> FFmpeg
|
| 35 |
Transitions --> FFmpeg
|
|
|
|
| 40 |
|
| 41 |
```text
|
| 42 |
renderer/
|
| 43 |
+
core/ settings, models, ingestion, orchestration
|
| 44 |
ffmpeg/ command builder, runner, probing, normalization
|
| 45 |
subtitles/ SRT and ASS generation
|
| 46 |
transitions/ xfade graph generation
|
|
|
|
| 53 |
|
| 54 |
## REST API
|
| 55 |
|
| 56 |
+
All API endpoints are served at the Space root. Asset fields may be absolute container paths or public `http`/`https` URLs. Remote assets are downloaded into a job-local temp directory with timeout and size limits before rendering.
|
| 57 |
|
| 58 |
### `POST /render`
|
| 59 |
|
|
|
|
| 63 |
{
|
| 64 |
"template": "tiktok_classic",
|
| 65 |
"output_name": "campaign_clip.mp4",
|
| 66 |
+
"voiceover": "https://cdn.example.com/voiceover.wav",
|
| 67 |
+
"background_music": "https://cdn.example.com/music.mp3",
|
| 68 |
"subtitle_format": "ass",
|
| 69 |
+
"auto_subtitles": true,
|
| 70 |
+
"subtitle_language": "en",
|
| 71 |
+
"whisper_model_size": "tiny",
|
| 72 |
+
"preset": "tiktok_9_16_fast",
|
| 73 |
+
"callback_url": "https://n8n.example.com/webhook/render-complete",
|
| 74 |
+
"export_target": "local",
|
| 75 |
+
"audio_normalize": true,
|
| 76 |
+
"preview": false,
|
| 77 |
"normalize": true,
|
| 78 |
"scenes": [
|
| 79 |
{
|
| 80 |
"start": 0,
|
| 81 |
"duration": 5,
|
| 82 |
+
"media": "https://cdn.example.com/scene1.mp4",
|
| 83 |
"caption": "Launch faster with automated rendering",
|
| 84 |
"transition": "fade"
|
| 85 |
}
|
|
|
|
| 93 |
{
|
| 94 |
"job_id": "job_abc123",
|
| 95 |
"status_url": "/status/job_abc123",
|
| 96 |
+
"download_url": "/download/job_abc123?token=..."
|
| 97 |
}
|
| 98 |
```
|
| 99 |
|
| 100 |
+
### `GET /presets`
|
| 101 |
+
|
| 102 |
+
Returns available render presets and caption templates. Presets let n8n submit compact requests such as `tiktok_9_16_fast`, `youtube_shorts_hd`, `podcast_square`, `reels_with_subtitles`, and `draft_preview`.
|
| 103 |
+
|
| 104 |
### `POST /render/ai-reels`
|
| 105 |
|
| 106 |
Submit a single-call AI Reels job using an existing voiceover and asset list. TTS is intentionally provider-pluggable in v1; the production path requires a supplied voiceover.
|
|
|
|
| 108 |
```json
|
| 109 |
{
|
| 110 |
"script": "Launch faster with automated rendering.",
|
| 111 |
+
"voiceover": "https://cdn.example.com/voiceover.wav",
|
| 112 |
+
"assets": ["https://cdn.example.com/scene1.jpg", "https://cdn.example.com/scene2.mp4"],
|
| 113 |
"template": "youtube_shorts",
|
| 114 |
"output_name": "ai_reel.mp4"
|
| 115 |
}
|
|
|
|
| 125 |
{
|
| 126 |
"template": "modern_minimal",
|
| 127 |
"output_name": "clip_a.mp4",
|
| 128 |
+
"scenes": [{"start": 0, "duration": 3, "media": "https://cdn.example.com/a.mp4"}]
|
| 129 |
+
}
|
| 130 |
+
]
|
| 131 |
+
}
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
### `POST /scene-builder`
|
| 135 |
+
|
| 136 |
+
Build a timeline from a script and asset list.
|
| 137 |
+
|
| 138 |
+
```json
|
| 139 |
+
{
|
| 140 |
+
"script": "One script can be split across several short scenes.",
|
| 141 |
+
"assets": ["https://cdn.example.com/a.mp4", "https://cdn.example.com/b.jpg"],
|
| 142 |
+
"duration": 8,
|
| 143 |
+
"transition": "fade"
|
| 144 |
+
}
|
| 145 |
+
```
|
| 146 |
+
|
| 147 |
+
### `POST /assets/upload`
|
| 148 |
+
|
| 149 |
+
Upload files from automation tools such as n8n and receive staged paths that can be used in `/render`.
|
| 150 |
+
|
| 151 |
+
Multipart form field:
|
| 152 |
+
|
| 153 |
+
- `files`: one or more images, videos, GIFs, or audio files.
|
| 154 |
+
|
| 155 |
+
Response:
|
| 156 |
+
|
| 157 |
+
```json
|
| 158 |
+
{
|
| 159 |
+
"assets": [
|
| 160 |
+
{
|
| 161 |
+
"filename": "clip.mp4",
|
| 162 |
+
"path": "/app/temp/uploads/abc/clip.mp4",
|
| 163 |
+
"reference": "upload://clip.mp4"
|
| 164 |
+
}
|
| 165 |
+
]
|
| 166 |
+
}
|
| 167 |
+
```
|
| 168 |
+
|
| 169 |
+
### `POST /render/upload`
|
| 170 |
+
|
| 171 |
+
Submit a render job and files in one multipart request.
|
| 172 |
+
|
| 173 |
+
Multipart fields:
|
| 174 |
+
|
| 175 |
+
- `request_json`: render payload JSON.
|
| 176 |
+
- `files`: uploaded files referenced by `upload://filename`.
|
| 177 |
+
|
| 178 |
+
Example `request_json`:
|
| 179 |
+
|
| 180 |
+
```json
|
| 181 |
+
{
|
| 182 |
+
"template": "tiktok_classic",
|
| 183 |
+
"output_name": "n8n_clip.mp4",
|
| 184 |
+
"scenes": [
|
| 185 |
+
{
|
| 186 |
+
"start": 0,
|
| 187 |
+
"duration": 5,
|
| 188 |
+
"media": "upload://clip.mp4",
|
| 189 |
+
"caption": "Rendered from n8n"
|
| 190 |
}
|
| 191 |
+
],
|
| 192 |
+
"voiceover": "upload://voice.wav"
|
| 193 |
+
}
|
| 194 |
+
```
|
| 195 |
+
|
| 196 |
+
### `POST /render/ai-reels/upload`
|
| 197 |
+
|
| 198 |
+
Multipart AI Reels endpoint. Use `upload://filename` inside `voiceover`, `background_music`, and `assets`.
|
| 199 |
+
|
| 200 |
+
### `POST /transcribe`
|
| 201 |
+
|
| 202 |
+
Transcribe local or remote audio/video with faster-whisper.
|
| 203 |
+
|
| 204 |
+
```json
|
| 205 |
+
{
|
| 206 |
+
"audio": "https://cdn.example.com/voiceover.wav",
|
| 207 |
+
"model_size": "tiny",
|
| 208 |
+
"language": "en",
|
| 209 |
+
"task": "transcribe",
|
| 210 |
+
"word_timestamps": true,
|
| 211 |
+
"vad_filter": true
|
| 212 |
+
}
|
| 213 |
+
```
|
| 214 |
+
|
| 215 |
+
Response includes `text`, detected `language`, segment timings, and optional word timings. Use this endpoint from n8n when you want subtitles before rendering.
|
| 216 |
+
|
| 217 |
+
### `POST /transcribe/upload`
|
| 218 |
+
|
| 219 |
+
Multipart faster-whisper endpoint for direct n8n file uploads.
|
| 220 |
+
|
| 221 |
+
Multipart fields:
|
| 222 |
+
|
| 223 |
+
- `file`: audio or video file.
|
| 224 |
+
- `model_size`: optional Whisper model, default `WHISPER_MODEL_SIZE`.
|
| 225 |
+
- `language`: optional ISO language code.
|
| 226 |
+
- `word_timestamps`: optional boolean, default `true`.
|
| 227 |
+
|
| 228 |
+
### `POST /subtitles`
|
| 229 |
+
|
| 230 |
+
Generate SRT or ASS files from timed events.
|
| 231 |
+
|
| 232 |
+
```json
|
| 233 |
+
{
|
| 234 |
+
"format": "ass",
|
| 235 |
+
"template": "tiktok_zoom",
|
| 236 |
+
"events": [
|
| 237 |
+
{"start": 0, "end": 1.2, "text": "Hello world"}
|
| 238 |
]
|
| 239 |
}
|
| 240 |
```
|
|
|
|
| 247 |
|
| 248 |
### `GET /download/{job_id}`
|
| 249 |
|
| 250 |
+
Returns the completed MP4 deliverable. Downloads require the signed token returned by render submission. If the job is still running, the endpoint returns `409`.
|
| 251 |
+
|
| 252 |
+
### `POST /cancel/{job_id}`
|
| 253 |
+
|
| 254 |
+
Requests job cancellation. Pending jobs are marked `CANCELLED`; running jobs are marked `CANCEL_REQUESTED` and stop before the next guarded execution point.
|
| 255 |
+
|
| 256 |
+
### `POST /admin/cleanup`
|
| 257 |
+
|
| 258 |
+
Deletes expired job records, old exports, and staged upload folders. Default retention is controlled by `JOB_RETENTION_SECONDS`.
|
| 259 |
|
| 260 |
### `POST /inspect`
|
| 261 |
|
| 262 |
+
Probe one local asset path and return MIME type, duration, codecs, bitrate, resolution, FPS, stream list, and cache metadata.
|
| 263 |
+
|
| 264 |
+
## n8n Workflow Integration
|
| 265 |
+
|
| 266 |
+
Recommended URL-based flow:
|
| 267 |
+
|
| 268 |
+
1. Use an HTTP Request node with `POST https://YOUR-SPACE.hf.space/render`.
|
| 269 |
+
2. Pass public file URLs from S3, Supabase Storage, Cloudinary, Google Drive direct-download links, or another CDN in `scenes[].media`, `voiceover`, and `background_music`.
|
| 270 |
+
3. Set `auto_subtitles` to `true` when you want faster-whisper captions generated from `voiceover`.
|
| 271 |
+
4. Add `callback_url` to receive a POST when the job completes or fails.
|
| 272 |
+
5. Poll `GET https://YOUR-SPACE.hf.space/status/{{$json.job_id}}` only if you do not use callbacks.
|
| 273 |
+
6. Download the final MP4 from the signed `download_url`.
|
| 274 |
+
|
| 275 |
+
Multipart flow:
|
| 276 |
+
|
| 277 |
+
1. Use an HTTP Request node set to multipart form-data.
|
| 278 |
+
2. Send `request_json` as a text field.
|
| 279 |
+
3. Attach files under the `files` field.
|
| 280 |
+
4. Reference those files in JSON as `upload://exact-filename.ext`.
|
| 281 |
+
|
| 282 |
+
Security defaults:
|
| 283 |
+
|
| 284 |
+
- Only `http` and `https` remote assets are accepted.
|
| 285 |
+
- Private, localhost, link-local, and multicast hosts are blocked by default.
|
| 286 |
+
- Set `ALLOW_PRIVATE_ASSET_URLS=true` only for trusted self-hosted deployments.
|
| 287 |
+
- `MAX_DOWNLOAD_BYTES` limits remote downloads and multipart upload totals.
|
| 288 |
+
- `WHISPER_MODEL_SIZE=tiny` and `WHISPER_COMPUTE_TYPE=int8` are the recommended CPU defaults.
|
| 289 |
+
- `BASYX_SIGNING_SECRET` should be set as a Space secret before public deployment.
|
| 290 |
|
| 291 |
## Operator Dashboard
|
| 292 |
|
|
|
|
| 313 |
- Use `OUTPUT_CRF=23-28` to balance quality and file size.
|
| 314 |
- Keep source assets near the target duration to reduce normalization work.
|
| 315 |
- Prefer pre-trimmed voiceovers and assets for batch workloads.
|
| 316 |
+
- Start with `WHISPER_MODEL_SIZE=tiny` or `base` on free/CPU Spaces.
|
| 317 |
+
- Use `preview=true` for low-resolution draft renders.
|
| 318 |
+
- Use `audio_normalize=true` for voiceover or podcast-style content.
|
| 319 |
|
| 320 |
## Reliability
|
| 321 |
|
| 322 |
- FFmpeg subprocesses are killed after `FFMPEG_TIMEOUT_SECONDS`.
|
| 323 |
+
- Remote asset downloads are killed after `DOWNLOAD_TIMEOUT_SECONDS`.
|
| 324 |
+
- Remote and multipart input size is capped by `MAX_DOWNLOAD_BYTES`.
|
| 325 |
+
- faster-whisper models are loaded lazily and cached under `WHISPER_MODEL_DIR`.
|
| 326 |
+
- Webhook callbacks are best-effort and callback failures are written to job logs.
|
| 327 |
+
- `export_target=local` copies completed renders to `STORAGE_DIR`.
|
| 328 |
+
- `export_target=https://...` uploads the completed MP4 with HTTP `PUT`, which works with presigned URLs from S3, Supabase Storage, and similar providers.
|
| 329 |
- Jobs retry up to `MAX_RETRIES`.
|
| 330 |
- Intermediate files are created under `TEMP_DIR` and removed after export.
|
| 331 |
- Job records retain command history, logs, render time, output size, and failure reasons.
|