Ava2lon commited on
Commit
b19c583
·
verified ·
1 Parent(s): c36b60c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +178 -12
README.md CHANGED
@@ -21,13 +21,15 @@ flowchart TD
21
  Dashboard["Gradio Operator Dashboard"] --> Jobs
22
  Jobs --> Engine["Render Engine"]
23
  Engine --> Assets["Asset Probe + Metadata Cache"]
 
24
  Engine --> Normalize["Normalization"]
25
  Engine --> Scenes["Scene Timeline"]
26
  Engine --> Subtitles["SRT / ASS Subtitle Engine"]
27
  Engine --> Transitions["Transition Filter Builder"]
28
  Engine --> Audio["Voiceover + Ducking Mixer"]
29
  Engine --> Exports["Export Manager"]
30
- Normalize --> FFmpeg["FFmpeg / FFprobe"]
 
31
  Scenes --> FFmpeg
32
  Subtitles --> FFmpeg
33
  Transitions --> FFmpeg
@@ -38,7 +40,7 @@ Package layout:
38
 
39
  ```text
40
  renderer/
41
- core/ settings, models, orchestration
42
  ffmpeg/ command builder, runner, probing, normalization
43
  subtitles/ SRT and ASS generation
44
  transitions/ xfade graph generation
@@ -51,7 +53,7 @@ renderer/
51
 
52
  ## REST API
53
 
54
- All API endpoints are served at the Space root.
55
 
56
  ### `POST /render`
57
 
@@ -61,15 +63,23 @@ Submit one render job.
61
  {
62
  "template": "tiktok_classic",
63
  "output_name": "campaign_clip.mp4",
64
- "voiceover": "/app/uploads/voiceover.wav",
65
- "background_music": "/app/uploads/music.mp3",
66
  "subtitle_format": "ass",
 
 
 
 
 
 
 
 
67
  "normalize": true,
68
  "scenes": [
69
  {
70
  "start": 0,
71
  "duration": 5,
72
- "media": "/app/uploads/scene1.mp4",
73
  "caption": "Launch faster with automated rendering",
74
  "transition": "fade"
75
  }
@@ -83,10 +93,14 @@ Response:
83
  {
84
  "job_id": "job_abc123",
85
  "status_url": "/status/job_abc123",
86
- "download_url": "/download/job_abc123"
87
  }
88
  ```
89
 
 
 
 
 
90
  ### `POST /render/ai-reels`
91
 
92
  Submit a single-call AI Reels job using an existing voiceover and asset list. TTS is intentionally provider-pluggable in v1; the production path requires a supplied voiceover.
@@ -94,8 +108,8 @@ Submit a single-call AI Reels job using an existing voiceover and asset list. TT
94
  ```json
95
  {
96
  "script": "Launch faster with automated rendering.",
97
- "voiceover": "/app/uploads/voiceover.wav",
98
- "assets": ["/app/uploads/scene1.jpg", "/app/uploads/scene2.mp4"],
99
  "template": "youtube_shorts",
100
  "output_name": "ai_reel.mp4"
101
  }
@@ -111,8 +125,116 @@ Submit multiple render jobs. The worker pool defaults to one active render to av
111
  {
112
  "template": "modern_minimal",
113
  "output_name": "clip_a.mp4",
114
- "scenes": [{"start": 0, "duration": 3, "media": "/app/uploads/a.mp4"}]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
  ]
117
  }
118
  ```
@@ -125,11 +247,46 @@ States: `PENDING`, `RUNNING`, `FAILED`, `COMPLETED`.
125
 
126
  ### `GET /download/{job_id}`
127
 
128
- Returns the completed MP4 deliverable. If the job is still running, the endpoint returns `409`.
 
 
 
 
 
 
 
 
129
 
130
  ### `POST /inspect`
131
 
132
- Probe one asset path and return MIME type, duration, codecs, bitrate, resolution, FPS, stream list, and cache metadata.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
133
 
134
  ## Operator Dashboard
135
 
@@ -156,10 +313,19 @@ Templates are selected per request and do not require code changes.
156
  - Use `OUTPUT_CRF=23-28` to balance quality and file size.
157
  - Keep source assets near the target duration to reduce normalization work.
158
  - Prefer pre-trimmed voiceovers and assets for batch workloads.
 
 
 
159
 
160
  ## Reliability
161
 
162
  - FFmpeg subprocesses are killed after `FFMPEG_TIMEOUT_SECONDS`.
 
 
 
 
 
 
163
  - Jobs retry up to `MAX_RETRIES`.
164
  - Intermediate files are created under `TEMP_DIR` and removed after export.
165
  - Job records retain command history, logs, render time, output size, and failure reasons.
 
21
  Dashboard["Gradio Operator Dashboard"] --> Jobs
22
  Jobs --> Engine["Render Engine"]
23
  Engine --> Assets["Asset Probe + Metadata Cache"]
24
+ Engine --> Ingest["Remote URL + Upload Ingestion"]
25
  Engine --> Normalize["Normalization"]
26
  Engine --> Scenes["Scene Timeline"]
27
  Engine --> Subtitles["SRT / ASS Subtitle Engine"]
28
  Engine --> Transitions["Transition Filter Builder"]
29
  Engine --> Audio["Voiceover + Ducking Mixer"]
30
  Engine --> Exports["Export Manager"]
31
+ Ingest --> FFmpeg["FFmpeg / FFprobe"]
32
+ Normalize --> FFmpeg
33
  Scenes --> FFmpeg
34
  Subtitles --> FFmpeg
35
  Transitions --> FFmpeg
 
40
 
41
  ```text
42
  renderer/
43
+ core/ settings, models, ingestion, orchestration
44
  ffmpeg/ command builder, runner, probing, normalization
45
  subtitles/ SRT and ASS generation
46
  transitions/ xfade graph generation
 
53
 
54
  ## REST API
55
 
56
+ All API endpoints are served at the Space root. Asset fields may be absolute container paths or public `http`/`https` URLs. Remote assets are downloaded into a job-local temp directory with timeout and size limits before rendering.
57
 
58
  ### `POST /render`
59
 
 
63
  {
64
  "template": "tiktok_classic",
65
  "output_name": "campaign_clip.mp4",
66
+ "voiceover": "https://cdn.example.com/voiceover.wav",
67
+ "background_music": "https://cdn.example.com/music.mp3",
68
  "subtitle_format": "ass",
69
+ "auto_subtitles": true,
70
+ "subtitle_language": "en",
71
+ "whisper_model_size": "tiny",
72
+ "preset": "tiktok_9_16_fast",
73
+ "callback_url": "https://n8n.example.com/webhook/render-complete",
74
+ "export_target": "local",
75
+ "audio_normalize": true,
76
+ "preview": false,
77
  "normalize": true,
78
  "scenes": [
79
  {
80
  "start": 0,
81
  "duration": 5,
82
+ "media": "https://cdn.example.com/scene1.mp4",
83
  "caption": "Launch faster with automated rendering",
84
  "transition": "fade"
85
  }
 
93
  {
94
  "job_id": "job_abc123",
95
  "status_url": "/status/job_abc123",
96
+ "download_url": "/download/job_abc123?token=..."
97
  }
98
  ```
99
 
100
+ ### `GET /presets`
101
+
102
+ Returns available render presets and caption templates. Presets let n8n submit compact requests such as `tiktok_9_16_fast`, `youtube_shorts_hd`, `podcast_square`, `reels_with_subtitles`, and `draft_preview`.
103
+
104
  ### `POST /render/ai-reels`
105
 
106
  Submit a single-call AI Reels job using an existing voiceover and asset list. TTS is intentionally provider-pluggable in v1; the production path requires a supplied voiceover.
 
108
  ```json
109
  {
110
  "script": "Launch faster with automated rendering.",
111
+ "voiceover": "https://cdn.example.com/voiceover.wav",
112
+ "assets": ["https://cdn.example.com/scene1.jpg", "https://cdn.example.com/scene2.mp4"],
113
  "template": "youtube_shorts",
114
  "output_name": "ai_reel.mp4"
115
  }
 
125
  {
126
  "template": "modern_minimal",
127
  "output_name": "clip_a.mp4",
128
+ "scenes": [{"start": 0, "duration": 3, "media": "https://cdn.example.com/a.mp4"}]
129
+ }
130
+ ]
131
+ }
132
+ ```
133
+
134
+ ### `POST /scene-builder`
135
+
136
+ Build a timeline from a script and asset list.
137
+
138
+ ```json
139
+ {
140
+ "script": "One script can be split across several short scenes.",
141
+ "assets": ["https://cdn.example.com/a.mp4", "https://cdn.example.com/b.jpg"],
142
+ "duration": 8,
143
+ "transition": "fade"
144
+ }
145
+ ```
146
+
147
+ ### `POST /assets/upload`
148
+
149
+ Upload files from automation tools such as n8n and receive staged paths that can be used in `/render`.
150
+
151
+ Multipart form field:
152
+
153
+ - `files`: one or more images, videos, GIFs, or audio files.
154
+
155
+ Response:
156
+
157
+ ```json
158
+ {
159
+ "assets": [
160
+ {
161
+ "filename": "clip.mp4",
162
+ "path": "/app/temp/uploads/abc/clip.mp4",
163
+ "reference": "upload://clip.mp4"
164
+ }
165
+ ]
166
+ }
167
+ ```
168
+
169
+ ### `POST /render/upload`
170
+
171
+ Submit a render job and files in one multipart request.
172
+
173
+ Multipart fields:
174
+
175
+ - `request_json`: render payload JSON.
176
+ - `files`: uploaded files referenced by `upload://filename`.
177
+
178
+ Example `request_json`:
179
+
180
+ ```json
181
+ {
182
+ "template": "tiktok_classic",
183
+ "output_name": "n8n_clip.mp4",
184
+ "scenes": [
185
+ {
186
+ "start": 0,
187
+ "duration": 5,
188
+ "media": "upload://clip.mp4",
189
+ "caption": "Rendered from n8n"
190
  }
191
+ ],
192
+ "voiceover": "upload://voice.wav"
193
+ }
194
+ ```
195
+
196
+ ### `POST /render/ai-reels/upload`
197
+
198
+ Multipart AI Reels endpoint. Use `upload://filename` inside `voiceover`, `background_music`, and `assets`.
199
+
200
+ ### `POST /transcribe`
201
+
202
+ Transcribe local or remote audio/video with faster-whisper.
203
+
204
+ ```json
205
+ {
206
+ "audio": "https://cdn.example.com/voiceover.wav",
207
+ "model_size": "tiny",
208
+ "language": "en",
209
+ "task": "transcribe",
210
+ "word_timestamps": true,
211
+ "vad_filter": true
212
+ }
213
+ ```
214
+
215
+ Response includes `text`, detected `language`, segment timings, and optional word timings. Use this endpoint from n8n when you want subtitles before rendering.
216
+
217
+ ### `POST /transcribe/upload`
218
+
219
+ Multipart faster-whisper endpoint for direct n8n file uploads.
220
+
221
+ Multipart fields:
222
+
223
+ - `file`: audio or video file.
224
+ - `model_size`: optional Whisper model, default `WHISPER_MODEL_SIZE`.
225
+ - `language`: optional ISO language code.
226
+ - `word_timestamps`: optional boolean, default `true`.
227
+
228
+ ### `POST /subtitles`
229
+
230
+ Generate SRT or ASS files from timed events.
231
+
232
+ ```json
233
+ {
234
+ "format": "ass",
235
+ "template": "tiktok_zoom",
236
+ "events": [
237
+ {"start": 0, "end": 1.2, "text": "Hello world"}
238
  ]
239
  }
240
  ```
 
247
 
248
  ### `GET /download/{job_id}`
249
 
250
+ Returns the completed MP4 deliverable. Downloads require the signed token returned by render submission. If the job is still running, the endpoint returns `409`.
251
+
252
+ ### `POST /cancel/{job_id}`
253
+
254
+ Requests job cancellation. Pending jobs are marked `CANCELLED`; running jobs are marked `CANCEL_REQUESTED` and stop before the next guarded execution point.
255
+
256
+ ### `POST /admin/cleanup`
257
+
258
+ Deletes expired job records, old exports, and staged upload folders. Default retention is controlled by `JOB_RETENTION_SECONDS`.
259
 
260
  ### `POST /inspect`
261
 
262
+ Probe one local asset path and return MIME type, duration, codecs, bitrate, resolution, FPS, stream list, and cache metadata.
263
+
264
+ ## n8n Workflow Integration
265
+
266
+ Recommended URL-based flow:
267
+
268
+ 1. Use an HTTP Request node with `POST https://YOUR-SPACE.hf.space/render`.
269
+ 2. Pass public file URLs from S3, Supabase Storage, Cloudinary, Google Drive direct-download links, or another CDN in `scenes[].media`, `voiceover`, and `background_music`.
270
+ 3. Set `auto_subtitles` to `true` when you want faster-whisper captions generated from `voiceover`.
271
+ 4. Add `callback_url` to receive a POST when the job completes or fails.
272
+ 5. Poll `GET https://YOUR-SPACE.hf.space/status/{{$json.job_id}}` only if you do not use callbacks.
273
+ 6. Download the final MP4 from the signed `download_url`.
274
+
275
+ Multipart flow:
276
+
277
+ 1. Use an HTTP Request node set to multipart form-data.
278
+ 2. Send `request_json` as a text field.
279
+ 3. Attach files under the `files` field.
280
+ 4. Reference those files in JSON as `upload://exact-filename.ext`.
281
+
282
+ Security defaults:
283
+
284
+ - Only `http` and `https` remote assets are accepted.
285
+ - Private, localhost, link-local, and multicast hosts are blocked by default.
286
+ - Set `ALLOW_PRIVATE_ASSET_URLS=true` only for trusted self-hosted deployments.
287
+ - `MAX_DOWNLOAD_BYTES` limits remote downloads and multipart upload totals.
288
+ - `WHISPER_MODEL_SIZE=tiny` and `WHISPER_COMPUTE_TYPE=int8` are the recommended CPU defaults.
289
+ - `BASYX_SIGNING_SECRET` should be set as a Space secret before public deployment.
290
 
291
  ## Operator Dashboard
292
 
 
313
  - Use `OUTPUT_CRF=23-28` to balance quality and file size.
314
  - Keep source assets near the target duration to reduce normalization work.
315
  - Prefer pre-trimmed voiceovers and assets for batch workloads.
316
+ - Start with `WHISPER_MODEL_SIZE=tiny` or `base` on free/CPU Spaces.
317
+ - Use `preview=true` for low-resolution draft renders.
318
+ - Use `audio_normalize=true` for voiceover or podcast-style content.
319
 
320
  ## Reliability
321
 
322
  - FFmpeg subprocesses are killed after `FFMPEG_TIMEOUT_SECONDS`.
323
+ - Remote asset downloads are killed after `DOWNLOAD_TIMEOUT_SECONDS`.
324
+ - Remote and multipart input size is capped by `MAX_DOWNLOAD_BYTES`.
325
+ - faster-whisper models are loaded lazily and cached under `WHISPER_MODEL_DIR`.
326
+ - Webhook callbacks are best-effort and callback failures are written to job logs.
327
+ - `export_target=local` copies completed renders to `STORAGE_DIR`.
328
+ - `export_target=https://...` uploads the completed MP4 with HTTP `PUT`, which works with presigned URLs from S3, Supabase Storage, and similar providers.
329
  - Jobs retry up to `MAX_RETRIES`.
330
  - Intermediate files are created under `TEMP_DIR` and removed after export.
331
  - Job records retain command history, logs, render time, output size, and failure reasons.