File size: 1,602 Bytes
d6f0309 ab9d20f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | V0_1: 2,800 more training steps compared to V0.
Current V0 went through :
- 100 initial steps at LR 5e-5 (batch size 8)
- 1100 more steps at LR 1e-4 (batch size 32)
- 600 more steps at LR 1e-4 (batch size 8)
- 600 more steps at LR 7.5e-5 (batch size 8)
Dataset has 30,000 examples covering:
- Text-to-video with no reference: 6,500
- Text-to-image with no reference: 3,500
- Still-image-to-video using:
- First frame from the target video: 750
- Middle frame from the target video: 750
- Last frame from the target video: 750
- Frame from the same source but outside the target interval: 750
- Face-reference-to-video using a close-up crop of the primary face: 3,000
- Image-to-image using a different image from the same gallery: 3,000
- Image restoration and inpainting:
- Masked image reference: 800
- Degraded image reference: 800
- Image outpainting using an aggressively cropped reference: 2,400
- Video outpainting:
- Static spatial crop: 501
- Tracked spatial crop: 499
- Temporal video completion:
- Reference video immediately before the target: 467
- Reference segment inside the complete target: 467
- Reference video immediately after the target: 466
- Audio-to-video using the target audio as conditioning: 2,000
- Video-to-audio / sound generation using a muted,
low-resolution video reference: 1,400
- Structural conditioning:
- Single-frame depth: 160
- Single-frame optical flow: 147
- Single-frame pose: 145
- Single-frame segmentation: 148
- Video depth: 146
- Video optical flow: 153
- Video pose: 149
- Video segmentation: 152 |