training_instructions.md

Instructions on Training a New Garment Checkpoint

Items Required

• A camera capable of recording video in 4K resolution (most smartphones are sufficient).
• The garment intended for virtual try-on.
• A person to wear the garment during recording.

Step 1

Recording a video of a person wearing the garment following the instruction of this paper. The predefined poses can be find here. Please note that you don't need to strictly follow the predefined poses (the more diverse the poses, the better). We provide an example of a recorded video here.

Step 2

Performing garment segmentation to the recorded video. There are many available methods, we recommend this repository, which requires only minimal interaction to achieve desirable garment segmentation results.

Exporting the segmentation results as a tar file. We provide an example of a tar file here.

Step 3

Create a directory and extract the tar file inside this directory. Run the following command to generate a per-garment dataset.

python DatasetGeneration/upperbody_dataset_generation.py --video_path <path to video> --mask_dir <path to mask dir> --dataset_name <name of this garment>

A dataset will be generated under ./PerGarmentDatasets.

Step 4

Train your own garment checkpoint using this command:

python Training/upperbody_training.py  --model pix2pixHD_RGBA --input_nc 6 --output_nc=4 --batchSize 4 --img_size 512 --dataset_path ./PerGarmentDatasets/<dataset name> --name <garment name> --niter 80 --niter_decay 80

A checkpoint will be generated under ./checkpoints.

Step 5

Run this command to use your trained checkpoint for virtual try-on. We provide an example video as input here.

python Inference/upperbody_inference.py --input_video <input video path> --garment_name <garment name>

The output video will be saved as ./output.mp4.