neonforestmist commited on
Commit
220ac54
·
verified ·
1 Parent(s): 59698fd

Add Modal training and inpainting model scaffold

Browse files
Files changed (1) hide show
  1. training/README-INPAINTING.md +61 -0
training/README-INPAINTING.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Clover Image Tiny Inpainting
2
+
3
+ The inpainting model keeps Clover’s text encoder, VAE, scheduler, and safety
4
+ checker, and replaces only the U-Net with a 9-channel fine-tuned version.
5
+
6
+ The U-Net input is ordered as:
7
+
8
+ ```text
9
+ [noisy_latent (4), mask (1), masked_image_latent (4)]
10
+ ```
11
+
12
+ White mask pixels are regenerated; black pixels are preserved. Masks are
13
+ sampled procedurally during training, so the dataset only needs image/caption
14
+ pairs. The pinned default dataset is `prithivMLmods/Caption3o-Opt` at revision
15
+ `17e893f785fcd3f5d6fc4a5d65a914b9f7b1ff5b` (Apache-2.0, `image` + `caption`).
16
+
17
+ ## Modal
18
+
19
+ The local Modal CLI is already configured for the `guccichungus69` profile.
20
+ Use that profile before running the commands below:
21
+
22
+ ```bash
23
+ modal profile activate guccichungus69
24
+ modal run modal_inpaint.py --smoke --steps 2
25
+ modal run modal_inpaint.py --steps 4000
26
+ ```
27
+
28
+ The job uses one A10G and writes the finished Diffusers pipeline to the
29
+ `clover-image-tiny-inpaint-output` Volume. No Hub token is required for the
30
+ default public dataset. The estimated compute ceiling for the default four-hour
31
+ job is about $4 at $1/hour; the actual charge depends on runtime.
32
+
33
+ Download a completed output directory with:
34
+
35
+ ```bash
36
+ modal volume get clover-image-tiny-inpaint-output \
37
+ clover-image-tiny-inpaint \
38
+ ./artifacts/clover-image-tiny-inpaint
39
+ ```
40
+
41
+ If a job stops before writing its final pipeline, the Volume retains any files
42
+ that were committed. The current trainer saves only at the end, so use a
43
+ separate `--output-name` for retries.
44
+
45
+ ## Local dry run
46
+
47
+ The same trainer can run locally when the model and dataset are available:
48
+
49
+ ```bash
50
+ python inpainting/train.py \
51
+ --pretrained_model_name_or_path . \
52
+ --dataset_name prithivMLmods/Caption3o-Opt \
53
+ --dataset_revision 17e893f785fcd3f5d6fc4a5d65a914b9f7b1ff5b \
54
+ --max_train_samples 4 \
55
+ --max_train_steps 2 \
56
+ --output_dir /tmp/clover-image-tiny-inpaint-smoke
57
+ ```
58
+
59
+ The complete model is large enough that Modal is the intended training
60
+ environment. The current repo’s Apple Core ML environment is for conversion,
61
+ not PyTorch training.