train_enc_dc_f32c32_EqM/main.log

[2025-10-26 11:19:20,467][main][INFO] - Will write tensorboard logs inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/tensorboard_logs
[2025-10-26 11:19:20,470][main][INFO] - Runtime at /workspace/DC_SSDAE
[2025-10-26 11:19:20,472][main][INFO] - Running inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM
[2025-10-26 11:19:20,472][main][INFO] - Running args: ['main.py', 'run_name=train_enc_dc_f32c32_EqM', 'dataset.im_size=128', 'dataset.aug_scale=2', 'training.epochs=60', 'dc_ssdae.encoder_train=true']
[2025-10-26 11:19:20,473][main][INFO] - Command: 'main.py' 'run_name=train_enc_dc_f32c32_EqM' 'dataset.im_size=128' 'dataset.aug_scale=2' 'training.epochs=60' 'dc_ssdae.encoder_train=true'
[2025-10-26 11:19:20,473][main][INFO] - Accelerator with 8 processes, running on cuda:0
[2025-10-26 11:19:20,478][main][INFO] - Hydra configuration:
seed: 0
task: train
runtime_path: ${hydra:runtime.cwd}
ckpt_dir: ${runtime_path}/runs
run_name: train_enc_dc_f32c32_EqM
cache_dir: ${ckpt_dir}/cache
run_dir: ${ckpt_dir}/jobs/${run_name}
checkpoint_path: ${run_dir}/checkpoints
dataset:
  imagenet_root: imagenet_data
  im_size: 128
  batch_size: 192
  aug_scale: 2
  limit: null
distill_teacher: false
dc_ssdae:
  compile: false
  checkpoint: null
  encoder: f32c32
  encoder_checkpoint: null
  encoder_train: true
  decoder: S
  trainer_type: FM
  encoder_type: dc
  sampler:
    steps: 10
  ema:
    decay: 0.999
    start_iter: 50000
aux_losses:
  compile: ${dc_ssdae.compile}
  repa:
    i_extract: 4
    n_layers: 2
  lpips: true
training:
  sdpa_kernel: 2
  mixed_precision: bf16
  grad_accumulate: 1
  grad_clip: 0.1
  epochs: 60
  eval_freq: 1
  save_on_best: FID
  log_freq: 100
  lr: 0.0003
  weight_decay: 0.001
losses:
  diffusion: 1
  repa: 0.25
  lpips: 0.5
  kl: 1.0e-06
show_samples: 8



[2025-10-26 11:19:33,933][main][INFO] - Loaded ImageNet dataset: {'train': Dataset ImageNet
    Number of datapoints: 1279867
    Root location: ../../../imagenet_data
    Split: train
    StandardTransform
Transform: Compose(
                 RandomResize(min_size=128, max_size=256, interpolation=InterpolationMode.LANCZOS, antialias=True)
                 RandomCrop(size=(128, 128), pad_if_needed=False, fill=0, padding_mode=constant)
                 RandomHorizontalFlip(p=0.5)
                 ToImage()
                 ToDtype(scale=True)
                 Normalize(mean=[0.5], std=[0.5], inplace=False)
           ), 'test': Dataset ImageNet
    Number of datapoints: 49950
    Root location: ../../../imagenet_data
    Split: validation
    StandardTransform
Transform: Compose(
                 Resize(size=[128], interpolation=InterpolationMode.BILINEAR, antialias=True)
                 CenterCrop(size=(128, 128))
                 ToImage()
                 ToDtype(scale=True)
                 Normalize(mean=[0.5], std=[0.5], inplace=False)
           )}
[2025-10-26 11:19:49,801][main][INFO] - ae parameters count:
[2025-10-26 11:19:49,807][main][INFO] - Total: #230.9M   (trainable: #230.9M)
[2025-10-26 11:19:49,808][main][INFO] - - encoder: #217.4M   (trainable: #217.4M)
[2025-10-26 11:19:49,809][main][INFO] -      - project_in: #1.8K   (trainable: #1.8K)
[2025-10-26 11:19:49,810][main][INFO] -      - stages: #216.9M   (trainable: #216.9M)
[2025-10-26 11:19:49,811][main][INFO] -      - project_out: #576.1K   (trainable: #576.1K)
[2025-10-26 11:19:49,813][main][INFO] - - decoder: #13.5M   (trainable: #13.5M)
[2025-10-26 11:19:49,813][main][INFO] -      - conv_in_img: #896   (trainable: #896)
[2025-10-26 11:19:49,814][main][INFO] -      - conv_in_z: #9.0K   (trainable: #9.0K)
[2025-10-26 11:19:49,814][main][INFO] -      - conv_in: #36.1K   (trainable: #36.1K)
[2025-10-26 11:19:49,815][main][INFO] -      - batch_norm_z: #64   (trainable: #64)
[2025-10-26 11:19:49,815][main][INFO] -      - time_proj: #0   (trainable: #0)
[2025-10-26 11:19:49,817][main][INFO] -      - time_embedding: #80.5K   (trainable: #80.5K)
[2025-10-26 11:19:49,818][main][INFO] -      - ada_ctx_proj: #54.1K   (trainable: #54.1K)
[2025-10-26 11:19:49,819][main][INFO] -      - down_blocks: #3.0M   (trainable: #3.0M)
[2025-10-26 11:19:49,820][main][INFO] -      - mid_block: #3.4M   (trainable: #3.4M)
[2025-10-26 11:19:49,820][main][INFO] -      - up_blocks: #6.9M   (trainable: #6.9M)
[2025-10-26 11:19:49,821][main][INFO] -      - conv_norm_out: #128   (trainable: #128)
[2025-10-26 11:19:49,821][main][INFO] -      - conv_out_act: #0   (trainable: #0)
[2025-10-26 11:19:49,822][main][INFO] -      - conv_out: #1.7K   (trainable: #1.7K)
[2025-10-26 11:19:49,825][main][INFO] - ae: EMAWrapper(
  (model): DistributedDataParallel(
    (module): DC_SSDAE(
      (encoder): DCEncoder(
        (project_in): ConvPixelUnshuffleDownSampleLayer(
          (conv): ConvLayer(
            (conv): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          )
        )
        (stages): ModuleList(
          (0): OpSequential(
            (op_list): ModuleList()
          )
          (1): OpSequential(
            (op_list): ModuleList(
              (0-4): 5 x ResidualBlock(
                (main): ResBlock(
                  (conv1): ConvLayer(
                    (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                    (act): SiLU()
                  )
                  (conv2): ConvLayer(
                    (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  )
                )
                (shortcut): IdentityLayer()
              )
              (5): ResidualBlock(
                (main): ConvPixelUnshuffleDownSampleLayer(
                  (conv): ConvLayer(
                    (conv): Conv2d(256, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                  )
                )
                (shortcut): PixelUnshuffleChannelAveragingDownSampleLayer()
              )
            )
          )
          (2): OpSequential(
            (op_list): ModuleList(
              (0-9): 10 x ResidualBlock(
                (main): ResBlock(
                  (conv1): ConvLayer(
                    (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                    (act): SiLU()
                  )
                  (conv2): ConvLayer(
                    (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  )
                )
                (shortcut): IdentityLayer()
              )
              (10): ResidualBlock(
                (main): ConvPixelUnshuffleDownSampleLayer(
                  (conv): ConvLayer(
                    (conv): Conv2d(512, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                  )
                )
                (shortcut): PixelUnshuffleChannelAveragingDownSampleLayer()
              )
            )
          )
          (3): OpSequential(
            (op_list): ModuleList(
              (0-3): 4 x ResidualBlock(
                (main): ResBlock(
                  (conv1): ConvLayer(
                    (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                    (act): SiLU()
                  )
                  (conv2): ConvLayer(
                    (conv): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  )
                )
                (shortcut): IdentityLayer()
              )
              (4): ResidualBlock(
                (main): ConvPixelUnshuffleDownSampleLayer(
                  (conv): ConvLayer(
                    (conv): Conv2d(512, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                  )
                )
                (shortcut): PixelUnshuffleChannelAveragingDownSampleLayer()
              )
            )
          )
          (4): OpSequential(
            (op_list): ModuleList(
              (0-3): 4 x ResidualBlock(
                (main): ResBlock(
                  (conv1): ConvLayer(
                    (conv): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                    (act): SiLU()
                  )
                  (conv2): ConvLayer(
                    (conv): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  )
                )
                (shortcut): IdentityLayer()
              )
              (4): ResidualBlock(
                (main): ConvPixelUnshuffleDownSampleLayer(
                  (conv): ConvLayer(
                    (conv): Conv2d(1024, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                  )
                )
                (shortcut): PixelUnshuffleChannelAveragingDownSampleLayer()
              )
            )
          )
          (5): OpSequential(
            (op_list): ModuleList(
              (0-3): 4 x ResidualBlock(
                (main): ResBlock(
                  (conv1): ConvLayer(
                    (conv): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                    (act): SiLU()
                  )
                  (conv2): ConvLayer(
                    (conv): Conv2d(1024, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  )
                )
                (shortcut): IdentityLayer()
              )
            )
          )
        )
        (project_out): OpSequential(
          (op_list): ModuleList(
            (0): ConvLayer(
              (conv): Conv2d(1024, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
            )
          )
        )
      )
      (decoder): UViTDecoder(
        (conv_in_img): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (conv_in_z): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (conv_in): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (batch_norm_z): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (time_proj): Timesteps()
        (time_embedding): TimestepEmbedding(
          (linear_1): Linear(in_features=64, out_features=256, bias=True)
          (act): SiLU()
          (linear_2): Linear(in_features=256, out_features=256, bias=True)
        )
        (ada_ctx_proj): Sequential(
          (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): SiLU()
          (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (down_blocks): ModuleList(
          (0): DownBlock2D(
            (resnets): ModuleList(
              (0-1): 2 x ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=128, bias=True)
                (norm2): GroupNorm(32, 64, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
              )
            )
            (downsamplers): ModuleList(
              (0): Downsample2D(
                (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
              )
            )
          )
          (1): DownBlock2D(
            (resnets): ModuleList(
              (0): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=192, bias=True)
                (norm2): GroupNorm(32, 96, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(64, 96, kernel_size=(1, 1), stride=(1, 1))
              )
              (1): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 192, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=192, bias=True)
                (norm2): GroupNorm(32, 96, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
              )
            )
            (downsamplers): ModuleList(
              (0): Downsample2D(
                (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
              )
            )
          )
          (2): DownBlock2D(
            (resnets): ModuleList(
              (0): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 192, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(96, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(96, 160, kernel_size=(1, 1), stride=(1, 1))
              )
              (1): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 320, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
              )
            )
            (downsamplers): ModuleList(
              (0): Downsample2D(
                (conv): Conv2d(160, 160, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
              )
            )
          )
          (3): DownBlock2D(
            (resnets): ModuleList(
              (0-1): 2 x ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 320, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
              )
            )
          )
        )
        (mid_block): UViTMiddleTransformer(
          (proj_in): Linear(in_features=160, out_features=160, bias=True)
          (transformer_blocks): ModuleList(
            (0-7): 8 x TransformerBlock(
              (norm1): AdaLayerNorm(
                (silu): SiLU()
                (linear): Linear(in_features=64, out_features=320, bias=True)
                (norm): LayerNorm((160,), eps=1e-05, elementwise_affine=False)
              )
              (attn1): Attention(
                (to_q): Linear(in_features=160, out_features=160, bias=False)
                (to_k): Linear(in_features=160, out_features=160, bias=False)
                (to_v): Linear(in_features=160, out_features=160, bias=False)
                (out_proj): Linear(in_features=160, out_features=160, bias=True)
                (out_drop): Dropout(p=0.0, inplace=False)
              )
              (norm2): LayerNorm((160,), eps=1e-05, elementwise_affine=True)
              (ff): FeedForward(
                (proj_in_act): GEGLU(
                  (proj): Linear(in_features=160, out_features=1280, bias=True)
                )
                (drop): Dropout(p=0.0, inplace=False)
                (proj_out): Linear(in_features=640, out_features=160, bias=True)
              )
              (relative_position_bias): RelativePositionBias()
            )
          )
          (proj_out): Linear(in_features=160, out_features=160, bias=True)
          (norm): GroupNorm(32, 160, eps=1e-06, affine=True)
        )
        (up_blocks): ModuleList(
          (0): UpBlock2D(
            (resnets): ModuleList(
              (0-2): 3 x ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 640, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(320, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(320, 160, kernel_size=(1, 1), stride=(1, 1))
              )
            )
            (upsamplers): ModuleList(
              (0): Upsample2D(
                (conv): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
              )
            )
          )
          (1): UpBlock2D(
            (resnets): ModuleList(
              (0-1): 2 x ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 640, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(320, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(320, 160, kernel_size=(1, 1), stride=(1, 1))
              )
              (2): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 512, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(256, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=320, bias=True)
                (norm2): GroupNorm(32, 160, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(256, 160, kernel_size=(1, 1), stride=(1, 1))
              )
            )
            (upsamplers): ModuleList(
              (0): Upsample2D(
                (conv): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
              )
            )
          )
          (2): UpBlock2D(
            (resnets): ModuleList(
              (0): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 512, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(256, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=192, bias=True)
                (norm2): GroupNorm(32, 96, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(256, 96, kernel_size=(1, 1), stride=(1, 1))
              )
              (1): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(192, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=192, bias=True)
                (norm2): GroupNorm(32, 96, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(192, 96, kernel_size=(1, 1), stride=(1, 1))
              )
              (2): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 320, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(160, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=192, bias=True)
                (norm2): GroupNorm(32, 96, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(160, 96, kernel_size=(1, 1), stride=(1, 1))
              )
            )
            (upsamplers): ModuleList(
              (0): Upsample2D(
                (conv): Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
              )
            )
          )
          (3): UpBlock2D(
            (resnets): ModuleList(
              (0): ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 320, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(160, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=128, bias=True)
                (norm2): GroupNorm(32, 64, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(160, 64, kernel_size=(1, 1), stride=(1, 1))
              )
              (1-2): 2 x ResnetBlock2D(
                (norm1): AdaGroupNorm2D(
                  (ctx_proj): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1))
                )
                (conv1): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (time_emb_proj): Linear(in_features=256, out_features=128, bias=True)
                (norm2): GroupNorm(32, 64, eps=1e-05, affine=True)
                (dropout): Dropout(p=0.0, inplace=False)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                (nonlinearity): SiLU()
                (conv_shortcut): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1))
              )
            )
          )
        )
        (conv_norm_out): GroupNorm(32, 64, eps=1e-05, affine=True)
        (conv_out_act): SiLU()
        (conv_out): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      )
    )
  )
  (ema): EMA(ema_model=DC_SSDAE, decay=0.999, start_iter=50000)
)
[2025-10-26 11:19:49,825][main][INFO] - aux_losses parameters count:
[2025-10-26 11:19:49,826][main][INFO] - Total: #96.7M   (trainable: #145.9K)
[2025-10-26 11:19:49,827][main][INFO] - - repa_loss: #82.7M   (trainable: #145.9K)
[2025-10-26 11:19:49,828][main][INFO] - - lpips_loss: #14.0M   (trainable: #0)
[2025-10-26 11:19:49,828][main][INFO] - aux_losses: DistributedDataParallel(
  (module): SSDDLosses(
    (repa_loss): REPALoss(
      (features_extractor): Frozen(DinoEncoder/Dinov2Model)
      (repa_mlp): Sequential(
        (0): Linear(in_features=160, out_features=160, bias=True)
        (1): SiLU()
        (2): Linear(in_features=160, out_features=768, bias=True)
      )
      (repa_loss): CosineSimilarity()
    )
    (lpips_loss): Frozen(LPIPS)
  )
)
[2025-10-26 11:19:49,833][main][INFO] - Optimizer for autoencoder: RAdamScheduleFree (
Parameter Group 0
    betas: (0.9, 0.999)
    eps: 1e-08
    foreach: True
    k: 0
    lr: 0.0003
    lr_max: -1.0
    r: 0.0
    scheduled_lr: 0.0
    silent_sgd_phase: True
    train_mode: False
    weight_decay: 0.001
    weight_lr_power: 2.0
    weight_sum: 0.0

Parameter Group 1
    betas: (0.9, 0.999)
    eps: 1e-08
    foreach: True
    k: 0
    lr: 0.0003
    lr_max: -1.0
    r: 0.0
    scheduled_lr: 0.0
    silent_sgd_phase: True
    train_mode: False
    weight_decay: 0.0
    weight_lr_power: 2.0
    weight_sum: 0.0
)
[2025-10-26 11:19:49,843][main][INFO] - No training state found to resume from None
[2025-10-26 11:19:49,844][main][INFO] - ====================== RUNNING TASK train
[2025-10-26 11:19:49,844][main][INFO] - Starting training
[2025-10-26 11:19:49,845][main][INFO] - Batch size of 192 (24 per GPU, 1 acumulation step(s) 8 process(es))
[2025-10-26 11:19:49,853][main][INFO] - ---


[2025-10-26 11:19:49,854][main][INFO] - [T_total=00:00:29 | T_train=00:00:00] Start epoch 0
[2025-10-26 14:25:01,522][main][INFO] - [T_total=03:05:41 | T_train=03:05:11 | T_epoch=03:05:11] End of epoch 0 (6666 steps) train loss 67151
[2025-10-26 14:25:01,524][main][INFO] - [Epoch 0] All losses: [[diffusion=0.124278 ; kl=6.71505e+10 ; lpips=0.360362 ; repa=0.667823]]
[2025-10-26 14:28:30,738][main][INFO] - [Epoch 1] Test metrics: [[MSE=47.45 | MAE=0.161 | LPIPS=0.4364 | PSNR=13.24 | SSIM=0.2403 | dreamsim=0.6167 | FID=113.3]]
[2025-10-26 14:28:30,740][main][INFO] - [Epoch 1] Best metrics: [[min_MSE=47.45 | min_MAE=0.161 | min_LPIPS=0.4364 | max_PSNR=13.24 | max_SSIM=0.2403 | min_dreamsim=0.6167 | min_FID=113.3]]
[2025-10-26 14:28:30,741][main][DEBUG] - Writing images to disk...
[2025-10-26 14:28:31,622][main][DEBUG] - Image(s) saved on disk
[2025-10-26 14:28:31,831][main][INFO] - End of epoch timers: [T_train=03:05:11 | T_epoch=03:05:11 | T_eval=00:03:30 | T_total=03:09:11]
[2025-10-26 14:28:31,832][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-26 14:28:43,727][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-26 14:28:54,887][main][INFO] - ---


[2025-10-26 14:28:54,888][main][INFO] - [T_total=03:09:34 | T_train=03:05:11] Start epoch 1
[2025-10-26 17:33:46,084][main][INFO] - [T_total=06:14:25 | T_train=06:10:02 | T_epoch=03:04:51] End of epoch 1 (13332 steps) train loss 4110.26
[2025-10-26 17:33:46,086][main][INFO] - [Epoch 1] All losses: [[diffusion=0.0919295 ; kl=4.10988e+09 ; lpips=0.275692 ; repa=0.588433]]
[2025-10-26 17:37:12,979][main][INFO] - [Epoch 2] Test metrics: [[MSE=46.7 | MAE=0.1611 | LPIPS=0.3256 | PSNR=13.31 | SSIM=0.2891 | dreamsim=0.496 | FID=78.54]]
[2025-10-26 17:37:12,981][main][INFO] - [Epoch 2] Best metrics: [[min_MSE=46.7 | min_MAE=0.161 | min_LPIPS=0.3256 | max_PSNR=13.31 | max_SSIM=0.2891 | min_dreamsim=0.496 | min_FID=78.54]]
[2025-10-26 17:37:12,982][main][DEBUG] - Writing images to disk...
[2025-10-26 17:37:13,796][main][DEBUG] - Image(s) saved on disk
[2025-10-26 17:37:14,011][main][INFO] - End of epoch timers: [T_train=06:10:02 | T_epoch=03:04:51 | T_eval=00:06:58 | T_total=06:17:53]
[2025-10-26 17:37:14,012][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-26 17:37:25,273][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-26 17:37:35,581][main][INFO] - ---


[2025-10-26 17:37:35,582][main][INFO] - [T_total=06:18:15 | T_train=06:10:02] Start epoch 2
[2025-10-26 20:42:34,608][main][INFO] - [T_total=09:23:14 | T_train=09:15:01 | T_epoch=03:04:59] End of epoch 2 (19998 steps) train loss 1112.41
[2025-10-26 20:42:34,609][main][INFO] - [Epoch 2] All losses: [[diffusion=0.0875515 ; kl=1.11206e+09 ; lpips=0.238805 ; repa=0.559219]]
[2025-10-26 20:46:02,005][main][INFO] - [Epoch 3] Test metrics: [[MSE=39.34 | MAE=0.1462 | LPIPS=0.2609 | PSNR=14.05 | SSIM=0.3195 | dreamsim=0.4047 | FID=56.04]]
[2025-10-26 20:46:02,007][main][INFO] - [Epoch 3] Best metrics: [[min_MSE=39.34 | min_MAE=0.1462 | min_LPIPS=0.2609 | max_PSNR=14.05 | max_SSIM=0.3195 | min_dreamsim=0.4047 | min_FID=56.04]]
[2025-10-26 20:46:02,007][main][DEBUG] - Writing images to disk...
[2025-10-26 20:46:02,818][main][DEBUG] - Image(s) saved on disk
[2025-10-26 20:46:03,028][main][INFO] - End of epoch timers: [T_train=09:15:01 | T_epoch=03:04:59 | T_eval=00:10:26 | T_total=09:26:42]
[2025-10-26 20:46:03,029][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-26 20:46:14,286][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-26 20:46:24,572][main][INFO] - ---


[2025-10-26 20:46:24,573][main][INFO] - [T_total=09:27:04 | T_train=09:15:01] Start epoch 3
[2025-10-26 23:51:05,689][main][INFO] - [T_total=12:31:45 | T_train=12:19:43 | T_epoch=03:04:41] End of epoch 3 (26664 steps) train loss 5.02755
[2025-10-26 23:51:05,690][main][INFO] - [Epoch 3] All losses: [[diffusion=0.0849642 ; kl=4.69653e+06 ; lpips=0.22171 ; repa=0.540818]]
[2025-10-26 23:54:33,185][main][INFO] - [Epoch 4] Test metrics: [[MSE=35.97 | MAE=0.1387 | LPIPS=0.2313 | PSNR=14.44 | SSIM=0.3346 | dreamsim=0.3568 | FID=45.03]]
[2025-10-26 23:54:33,187][main][INFO] - [Epoch 4] Best metrics: [[min_MSE=35.97 | min_MAE=0.1387 | min_LPIPS=0.2313 | max_PSNR=14.44 | max_SSIM=0.3346 | min_dreamsim=0.3568 | min_FID=45.03]]
[2025-10-26 23:54:33,188][main][DEBUG] - Writing images to disk...
[2025-10-26 23:54:34,013][main][DEBUG] - Image(s) saved on disk
[2025-10-26 23:54:34,260][main][INFO] - End of epoch timers: [T_train=12:19:43 | T_epoch=03:04:41 | T_eval=00:13:54 | T_total=12:35:13]
[2025-10-26 23:54:34,261][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-26 23:54:45,885][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-26 23:54:56,831][main][INFO] - ---


[2025-10-26 23:54:56,832][main][INFO] - [T_total=12:35:36 | T_train=12:19:43] Start epoch 4
[2025-10-27 03:00:25,624][main][INFO] - [T_total=15:41:05 | T_train=15:25:11 | T_epoch=03:05:28] End of epoch 4 (33330 steps) train loss 166.439
[2025-10-27 03:00:25,626][main][INFO] - [Epoch 4] All losses: [[diffusion=0.0838747 ; kl=1.66118e+08 ; lpips=0.211539 ; repa=0.528085]]
[2025-10-27 03:03:52,781][main][INFO] - [Epoch 5] Test metrics: [[MSE=31.78 | MAE=0.129 | LPIPS=0.2131 | PSNR=14.98 | SSIM=0.3511 | dreamsim=0.3263 | FID=38.77]]
[2025-10-27 03:03:52,782][main][INFO] - [Epoch 5] Best metrics: [[min_MSE=31.78 | min_MAE=0.129 | min_LPIPS=0.2131 | max_PSNR=14.98 | max_SSIM=0.3511 | min_dreamsim=0.3263 | min_FID=38.77]]
[2025-10-27 03:03:52,783][main][DEBUG] - Writing images to disk...
[2025-10-27 03:03:53,617][main][DEBUG] - Image(s) saved on disk
[2025-10-27 03:03:53,821][main][INFO] - End of epoch timers: [T_train=15:25:11 | T_epoch=03:05:28 | T_eval=00:17:22 | T_total=15:44:33]
[2025-10-27 03:03:53,821][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 03:04:05,149][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 03:04:14,375][main][INFO] - ---


[2025-10-27 03:04:14,376][main][INFO] - [T_total=15:44:53 | T_train=15:25:11] Start epoch 5
[2025-10-27 06:10:26,456][main][INFO] - [T_total=18:51:05 | T_train=18:31:23 | T_epoch=03:06:12] End of epoch 5 (39996 steps) train loss 39272
[2025-10-27 06:10:26,457][main][INFO] - [Epoch 5] All losses: [[diffusion=0.0828694 ; kl=3.92717e+10 ; lpips=0.205352 ; repa=0.518688]]
[2025-10-27 06:13:53,521][main][INFO] - [Epoch 6] Test metrics: [[MSE=29.02 | MAE=0.1223 | LPIPS=0.2011 | PSNR=15.37 | SSIM=0.3643 | dreamsim=0.3048 | FID=34.65]]
[2025-10-27 06:13:53,524][main][INFO] - [Epoch 6] Best metrics: [[min_MSE=29.02 | min_MAE=0.1223 | min_LPIPS=0.2011 | max_PSNR=15.37 | max_SSIM=0.3643 | min_dreamsim=0.3048 | min_FID=34.65]]
[2025-10-27 06:13:53,525][main][DEBUG] - Writing images to disk...
[2025-10-27 06:13:54,357][main][DEBUG] - Image(s) saved on disk
[2025-10-27 06:13:54,564][main][INFO] - End of epoch timers: [T_train=18:31:23 | T_epoch=03:06:12 | T_eval=00:20:50 | T_total=18:54:34]
[2025-10-27 06:13:54,565][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 06:14:09,698][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 06:14:20,685][main][INFO] - ---


[2025-10-27 06:14:20,686][main][INFO] - [T_total=18:55:00 | T_train=18:31:23] Start epoch 6
[2025-10-27 09:20:11,071][main][INFO] - [T_total=22:00:50 | T_train=21:37:14 | T_epoch=03:05:50] End of epoch 6 (46662 steps) train loss 38.9426
[2025-10-27 09:20:11,072][main][INFO] - [Epoch 6] All losses: [[diffusion=0.0819753 ; kl=3.86332e+07 ; lpips=0.199326 ; repa=0.510736]]
[2025-10-27 09:23:38,395][main][INFO] - [Epoch 7] Test metrics: [[MSE=27.01 | MAE=0.1173 | LPIPS=0.191 | PSNR=15.68 | SSIM=0.3792 | dreamsim=0.2861 | FID=30.48]]
[2025-10-27 09:23:38,397][main][INFO] - [Epoch 7] Best metrics: [[min_MSE=27.01 | min_MAE=0.1173 | min_LPIPS=0.191 | max_PSNR=15.68 | max_SSIM=0.3792 | min_dreamsim=0.2861 | min_FID=30.48]]
[2025-10-27 09:23:38,398][main][DEBUG] - Writing images to disk...
[2025-10-27 09:23:39,236][main][DEBUG] - Image(s) saved on disk
[2025-10-27 09:23:39,501][main][INFO] - End of epoch timers: [T_train=21:37:14 | T_epoch=03:05:50 | T_eval=00:24:19 | T_total=22:04:19]
[2025-10-27 09:23:39,505][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 09:23:49,578][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 09:23:58,982][main][INFO] - ---


[2025-10-27 09:23:58,983][main][INFO] - [T_total=22:04:38 | T_train=21:37:14] Start epoch 7
[2025-10-27 12:29:38,691][main][INFO] - [T_total=25:10:18 | T_train=24:42:53 | T_epoch=03:05:39] End of epoch 7 (53328 steps) train loss 21.8781
[2025-10-27 12:29:38,692][main][INFO] - [Epoch 7] All losses: [[diffusion=0.0808239 ; kl=2.15753e+07 ; lpips=0.192459 ; repa=0.50294]]
[2025-10-27 12:33:06,379][main][INFO] - [Epoch 8] Test metrics: [[MSE=25.89 | MAE=0.1143 | LPIPS=0.1838 | PSNR=15.87 | SSIM=0.3878 | dreamsim=0.273 | FID=27.51]]
[2025-10-27 12:33:06,381][main][INFO] - [Epoch 8] Best metrics: [[min_MSE=25.89 | min_MAE=0.1143 | min_LPIPS=0.1838 | max_PSNR=15.87 | max_SSIM=0.3878 | min_dreamsim=0.273 | min_FID=27.51]]
[2025-10-27 12:33:06,382][main][DEBUG] - Writing images to disk...
[2025-10-27 12:33:07,208][main][DEBUG] - Image(s) saved on disk
[2025-10-27 12:33:07,412][main][INFO] - End of epoch timers: [T_train=24:42:53 | T_epoch=03:05:39 | T_eval=00:27:47 | T_total=25:13:46]
[2025-10-27 12:33:07,414][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 12:33:18,072][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 12:33:28,529][main][INFO] - ---


[2025-10-27 12:33:28,530][main][INFO] - [T_total=25:14:08 | T_train=24:42:53] Start epoch 8
[2025-10-27 15:39:17,310][main][INFO] - [T_total=28:19:56 | T_train=27:48:42 | T_epoch=03:05:48] End of epoch 8 (59994 steps) train loss 56.025
[2025-10-27 15:39:17,312][main][INFO] - [Epoch 8] All losses: [[diffusion=0.0821982 ; kl=5.5718e+07 ; lpips=0.19805 ; repa=0.503114]]
[2025-10-27 15:42:44,898][main][INFO] - [Epoch 9] Test metrics: [[MSE=25.28 | MAE=0.1125 | LPIPS=0.1792 | PSNR=15.97 | SSIM=0.3941 | dreamsim=0.2633 | FID=25.14]]
[2025-10-27 15:42:44,902][main][INFO] - [Epoch 9] Best metrics: [[min_MSE=25.28 | min_MAE=0.1125 | min_LPIPS=0.1792 | max_PSNR=15.97 | max_SSIM=0.3941 | min_dreamsim=0.2633 | min_FID=25.14]]
[2025-10-27 15:42:44,903][main][DEBUG] - Writing images to disk...
[2025-10-27 15:42:45,999][main][DEBUG] - Image(s) saved on disk
[2025-10-27 15:42:46,279][main][INFO] - End of epoch timers: [T_train=27:48:42 | T_epoch=03:05:48 | T_eval=00:31:16 | T_total=28:23:25]
[2025-10-27 15:42:46,281][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 15:42:57,387][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 15:43:08,322][main][INFO] - ---


[2025-10-27 15:43:08,323][main][INFO] - [T_total=28:23:47 | T_train=27:48:42] Start epoch 9
[2025-10-27 18:48:50,025][main][INFO] - [T_total=31:29:29 | T_train=30:54:24 | T_epoch=03:05:41] End of epoch 9 (66660 steps) train loss 66.697
[2025-10-27 18:48:50,026][main][INFO] - [Epoch 9] All losses: [[diffusion=0.0810491 ; kl=6.63959e+07 ; lpips=0.191886 ; repa=0.496548]]
[2025-10-27 18:52:17,550][main][INFO] - [Epoch 10] Test metrics: [[MSE=24.51 | MAE=0.1103 | LPIPS=0.1742 | PSNR=16.11 | SSIM=0.4006 | dreamsim=0.2544 | FID=23.1]]
[2025-10-27 18:52:17,551][main][INFO] - [Epoch 10] Best metrics: [[min_MSE=24.51 | min_MAE=0.1103 | min_LPIPS=0.1742 | max_PSNR=16.11 | max_SSIM=0.4006 | min_dreamsim=0.2544 | min_FID=23.1]]
[2025-10-27 18:52:17,552][main][DEBUG] - Writing images to disk...
[2025-10-27 18:52:18,382][main][DEBUG] - Image(s) saved on disk
[2025-10-27 18:52:18,587][main][INFO] - End of epoch timers: [T_train=30:54:24 | T_epoch=03:05:41 | T_eval=00:34:44 | T_total=31:32:58]
[2025-10-27 18:52:18,589][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 18:52:30,619][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 18:52:40,962][main][INFO] - ---


[2025-10-27 18:52:40,963][main][INFO] - [T_total=31:33:20 | T_train=30:54:24] Start epoch 10
[2025-10-27 21:58:04,023][main][INFO] - [T_total=34:38:43 | T_train=33:59:47 | T_epoch=03:05:23] End of epoch 10 (73326 steps) train loss 5.94657
[2025-10-27 21:58:04,024][main][INFO] - [Epoch 10] All losses: [[diffusion=0.0795436 ; kl=5.65143e+06 ; lpips=0.186013 ; repa=0.490351]]
[2025-10-27 22:01:31,334][main][INFO] - [Epoch 11] Test metrics: [[MSE=24.04 | MAE=0.109 | LPIPS=0.1708 | PSNR=16.19 | SSIM=0.4055 | dreamsim=0.2477 | FID=21.54]]
[2025-10-27 22:01:31,336][main][INFO] - [Epoch 11] Best metrics: [[min_MSE=24.04 | min_MAE=0.109 | min_LPIPS=0.1708 | max_PSNR=16.19 | max_SSIM=0.4055 | min_dreamsim=0.2477 | min_FID=21.54]]
[2025-10-27 22:01:31,337][main][DEBUG] - Writing images to disk...
[2025-10-27 22:01:32,162][main][DEBUG] - Image(s) saved on disk
[2025-10-27 22:01:32,367][main][INFO] - End of epoch timers: [T_train=33:59:47 | T_epoch=03:05:23 | T_eval=00:38:12 | T_total=34:42:11]
[2025-10-27 22:01:32,368][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-27 22:01:42,990][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-27 22:01:52,731][main][INFO] - ---


[2025-10-27 22:01:52,731][main][INFO] - [T_total=34:42:32 | T_train=33:59:47] Start epoch 11
[2025-10-28 01:06:41,641][main][INFO] - [T_total=37:47:21 | T_train=37:04:36 | T_epoch=03:04:48] End of epoch 11 (79992 steps) train loss 794.458
[2025-10-28 01:06:41,643][main][INFO] - [Epoch 11] All losses: [[diffusion=0.0796192 ; kl=7.94164e+08 ; lpips=0.1859 ; repa=0.488066]]
[2025-10-28 01:10:08,942][main][INFO] - [Epoch 12] Test metrics: [[MSE=23.46 | MAE=0.1073 | LPIPS=0.1673 | PSNR=16.3 | SSIM=0.4107 | dreamsim=0.2413 | FID=20.23]]
[2025-10-28 01:10:08,944][main][INFO] - [Epoch 12] Best metrics: [[min_MSE=23.46 | min_MAE=0.1073 | min_LPIPS=0.1673 | max_PSNR=16.3 | max_SSIM=0.4107 | min_dreamsim=0.2413 | min_FID=20.23]]
[2025-10-28 01:10:08,945][main][DEBUG] - Writing images to disk...
[2025-10-28 01:10:09,790][main][DEBUG] - Image(s) saved on disk
[2025-10-28 01:10:09,998][main][INFO] - End of epoch timers: [T_train=37:04:36 | T_epoch=03:04:48 | T_eval=00:41:41 | T_total=37:50:49]
[2025-10-28 01:10:09,999][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 01:10:20,640][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 01:10:31,337][main][INFO] - ---


[2025-10-28 01:10:31,338][main][INFO] - [T_total=37:51:10 | T_train=37:04:36] Start epoch 12
[2025-10-28 04:15:54,812][main][INFO] - [T_total=40:56:34 | T_train=40:09:59 | T_epoch=03:05:23] End of epoch 12 (86658 steps) train loss 4.51982
[2025-10-28 04:15:54,814][main][INFO] - [Epoch 12] All losses: [[diffusion=0.0793437 ; kl=4.22649e+06 ; lpips=0.185086 ; repa=0.485754]]
[2025-10-28 04:19:22,090][main][INFO] - [Epoch 13] Test metrics: [[MSE=23.14 | MAE=0.1062 | LPIPS=0.1647 | PSNR=16.36 | SSIM=0.414 | dreamsim=0.2363 | FID=19.18]]
[2025-10-28 04:19:22,092][main][INFO] - [Epoch 13] Best metrics: [[min_MSE=23.14 | min_MAE=0.1062 | min_LPIPS=0.1647 | max_PSNR=16.36 | max_SSIM=0.414 | min_dreamsim=0.2363 | min_FID=19.18]]
[2025-10-28 04:19:22,093][main][DEBUG] - Writing images to disk...
[2025-10-28 04:19:22,937][main][DEBUG] - Image(s) saved on disk
[2025-10-28 04:19:23,137][main][INFO] - End of epoch timers: [T_train=40:09:59 | T_epoch=03:05:23 | T_eval=00:45:09 | T_total=41:00:02]
[2025-10-28 04:19:23,138][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 04:19:35,064][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 04:19:46,497][main][INFO] - ---


[2025-10-28 04:19:46,498][main][INFO] - [T_total=41:00:26 | T_train=40:09:59] Start epoch 13
[2025-10-28 07:25:31,821][main][INFO] - [T_total=44:06:11 | T_train=43:15:45 | T_epoch=03:05:45] End of epoch 13 (93324 steps) train loss 5.05172
[2025-10-28 07:25:31,823][main][INFO] - [Epoch 13] All losses: [[diffusion=0.0796848 ; kl=4.75808e+06 ; lpips=0.185526 ; repa=0.484762]]
[2025-10-28 07:28:58,913][main][INFO] - [Epoch 14] Test metrics: [[MSE=23.03 | MAE=0.106 | LPIPS=0.1633 | PSNR=16.38 | SSIM=0.4158 | dreamsim=0.2328 | FID=18.39]]
[2025-10-28 07:28:58,915][main][INFO] - [Epoch 14] Best metrics: [[min_MSE=23.03 | min_MAE=0.106 | min_LPIPS=0.1633 | max_PSNR=16.38 | max_SSIM=0.4158 | min_dreamsim=0.2328 | min_FID=18.39]]
[2025-10-28 07:28:58,916][main][DEBUG] - Writing images to disk...
[2025-10-28 07:28:59,747][main][DEBUG] - Image(s) saved on disk
[2025-10-28 07:28:59,993][main][INFO] - End of epoch timers: [T_train=43:15:45 | T_epoch=03:05:45 | T_eval=00:48:37 | T_total=44:09:39]
[2025-10-28 07:28:59,994][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 07:29:10,392][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 07:29:20,783][main][INFO] - ---


[2025-10-28 07:29:20,784][main][INFO] - [T_total=44:10:00 | T_train=43:15:45] Start epoch 14
[2025-10-28 10:35:12,763][main][INFO] - [T_total=47:15:52 | T_train=46:21:37 | T_epoch=03:05:51] End of epoch 14 (99990 steps) train loss 2.56252
[2025-10-28 10:35:12,765][main][INFO] - [Epoch 14] All losses: [[diffusion=0.0787443 ; kl=2.27203e+06 ; lpips=0.182851 ; repa=0.481282]]
[2025-10-28 10:38:39,849][main][INFO] - [Epoch 15] Test metrics: [[MSE=22.95 | MAE=0.1058 | LPIPS=0.1619 | PSNR=16.39 | SSIM=0.4198 | dreamsim=0.2294 | FID=17.64]]
[2025-10-28 10:38:39,851][main][INFO] - [Epoch 15] Best metrics: [[min_MSE=22.95 | min_MAE=0.1058 | min_LPIPS=0.1619 | max_PSNR=16.39 | max_SSIM=0.4198 | min_dreamsim=0.2294 | min_FID=17.64]]
[2025-10-28 10:38:39,853][main][DEBUG] - Writing images to disk...
[2025-10-28 10:38:40,690][main][DEBUG] - Image(s) saved on disk
[2025-10-28 10:38:40,939][main][INFO] - End of epoch timers: [T_train=46:21:37 | T_epoch=03:05:51 | T_eval=00:52:05 | T_total=47:19:20]
[2025-10-28 10:38:40,940][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 10:38:52,080][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 10:39:03,800][main][INFO] - ---


[2025-10-28 10:39:03,801][main][INFO] - [T_total=47:19:43 | T_train=46:21:37] Start epoch 15
[2025-10-28 13:44:56,004][main][INFO] - [T_total=50:25:35 | T_train=49:27:29 | T_epoch=03:05:52] End of epoch 15 (106656 steps) train loss 150.527
[2025-10-28 13:44:56,005][main][INFO] - [Epoch 15] All losses: [[diffusion=0.0778382 ; kl=1.50242e+08 ; lpips=0.177149 ; repa=0.475895]]
[2025-10-28 13:48:23,438][main][INFO] - [Epoch 16] Test metrics: [[MSE=22.91 | MAE=0.1058 | LPIPS=0.1609 | PSNR=16.4 | SSIM=0.4219 | dreamsim=0.2269 | FID=17.15]]
[2025-10-28 13:48:23,439][main][INFO] - [Epoch 16] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1609 | max_PSNR=16.4 | max_SSIM=0.4219 | min_dreamsim=0.2269 | min_FID=17.15]]
[2025-10-28 13:48:23,440][main][DEBUG] - Writing images to disk...
[2025-10-28 13:48:24,270][main][DEBUG] - Image(s) saved on disk
[2025-10-28 13:48:24,506][main][INFO] - End of epoch timers: [T_train=49:27:29 | T_epoch=03:05:52 | T_eval=00:55:33 | T_total=50:29:04]
[2025-10-28 13:48:24,507][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 13:48:36,131][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 13:48:46,875][main][INFO] - ---


[2025-10-28 13:48:46,876][main][INFO] - [T_total=50:29:26 | T_train=49:27:29] Start epoch 16
[2025-10-28 16:55:33,548][main][INFO] - [T_total=53:36:13 | T_train=52:34:16 | T_epoch=03:06:46] End of epoch 16 (113322 steps) train loss 254.554
[2025-10-28 16:55:33,549][main][INFO] - [Epoch 16] All losses: [[diffusion=0.0785965 ; kl=2.54265e+08 ; lpips=0.18152 ; repa=0.477904]]
[2025-10-28 16:59:00,697][main][INFO] - [Epoch 17] Test metrics: [[MSE=23.1 | MAE=0.1065 | LPIPS=0.16 | PSNR=16.36 | SSIM=0.425 | dreamsim=0.2245 | FID=16.63]]
[2025-10-28 16:59:00,699][main][INFO] - [Epoch 17] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.16 | max_PSNR=16.4 | max_SSIM=0.425 | min_dreamsim=0.2245 | min_FID=16.63]]
[2025-10-28 16:59:00,703][main][DEBUG] - Writing images to disk...
[2025-10-28 16:59:01,784][main][DEBUG] - Image(s) saved on disk
[2025-10-28 16:59:02,034][main][INFO] - End of epoch timers: [T_train=52:34:16 | T_epoch=03:06:46 | T_eval=00:59:02 | T_total=53:39:41]
[2025-10-28 16:59:02,035][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 16:59:13,349][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 16:59:24,416][main][INFO] - ---


[2025-10-28 16:59:24,417][main][INFO] - [T_total=53:40:03 | T_train=52:34:16] Start epoch 17
[2025-10-28 20:04:55,554][main][INFO] - [T_total=56:45:35 | T_train=55:39:47 | T_epoch=03:05:31] End of epoch 17 (119988 steps) train loss 3.54179
[2025-10-28 20:04:55,556][main][INFO] - [Epoch 17] All losses: [[diffusion=0.0778652 ; kl=3.25619e+06 ; lpips=0.178335 ; repa=0.474268]]
[2025-10-28 20:08:22,775][main][INFO] - [Epoch 18] Test metrics: [[MSE=23.09 | MAE=0.1066 | LPIPS=0.159 | PSNR=16.37 | SSIM=0.4268 | dreamsim=0.2221 | FID=16.09]]
[2025-10-28 20:08:22,777][main][INFO] - [Epoch 18] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.159 | max_PSNR=16.4 | max_SSIM=0.4268 | min_dreamsim=0.2221 | min_FID=16.09]]
[2025-10-28 20:08:22,778][main][DEBUG] - Writing images to disk...
[2025-10-28 20:08:23,864][main][DEBUG] - Image(s) saved on disk
[2025-10-28 20:08:24,074][main][INFO] - End of epoch timers: [T_train=55:39:47 | T_epoch=03:05:31 | T_eval=01:02:30 | T_total=56:49:03]
[2025-10-28 20:08:24,075][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 20:08:35,565][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 20:08:47,122][main][INFO] - ---


[2025-10-28 20:08:47,124][main][INFO] - [T_total=56:49:26 | T_train=55:39:47] Start epoch 18
[2025-10-28 23:13:47,257][main][INFO] - [T_total=59:54:26 | T_train=58:44:47 | T_epoch=03:05:00] End of epoch 18 (126654 steps) train loss 8.77081
[2025-10-28 23:13:47,258][main][INFO] - [Epoch 18] All losses: [[diffusion=0.0783209 ; kl=8.4835e+06 ; lpips=0.180506 ; repa=0.474953]]
[2025-10-28 23:17:14,615][main][INFO] - [Epoch 19] Test metrics: [[MSE=23.25 | MAE=0.1072 | LPIPS=0.1586 | PSNR=16.34 | SSIM=0.4269 | dreamsim=0.2204 | FID=15.71]]
[2025-10-28 23:17:14,617][main][INFO] - [Epoch 19] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1586 | max_PSNR=16.4 | max_SSIM=0.4269 | min_dreamsim=0.2204 | min_FID=15.71]]
[2025-10-28 23:17:14,618][main][DEBUG] - Writing images to disk...
[2025-10-28 23:17:15,450][main][DEBUG] - Image(s) saved on disk
[2025-10-28 23:17:15,693][main][INFO] - End of epoch timers: [T_train=58:44:47 | T_epoch=03:05:00 | T_eval=01:05:58 | T_total=59:57:55]
[2025-10-28 23:17:15,694][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-28 23:17:25,883][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-28 23:17:35,331][main][INFO] - ---


[2025-10-28 23:17:35,332][main][INFO] - [T_total=59:58:14 | T_train=58:44:47] Start epoch 19
[2025-10-29 02:22:52,012][main][INFO] - [T_total=63:03:31 | T_train=61:50:04 | T_epoch=03:05:16] End of epoch 19 (133320 steps) train loss 65.3624
[2025-10-29 02:22:52,014][main][INFO] - [Epoch 19] All losses: [[diffusion=0.0772616 ; kl=6.50794e+07 ; lpips=0.176332 ; repa=0.470315]]
[2025-10-29 02:26:19,233][main][INFO] - [Epoch 20] Test metrics: [[MSE=23.28 | MAE=0.1074 | LPIPS=0.1579 | PSNR=16.33 | SSIM=0.4276 | dreamsim=0.2187 | FID=15.33]]
[2025-10-29 02:26:19,250][main][INFO] - [Epoch 20] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1579 | max_PSNR=16.4 | max_SSIM=0.4276 | min_dreamsim=0.2187 | min_FID=15.33]]
[2025-10-29 02:26:19,251][main][DEBUG] - Writing images to disk...
[2025-10-29 02:26:20,086][main][DEBUG] - Image(s) saved on disk
[2025-10-29 02:26:20,336][main][INFO] - End of epoch timers: [T_train=61:50:04 | T_epoch=03:05:16 | T_eval=01:09:26 | T_total=63:06:59]
[2025-10-29 02:26:20,337][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 02:26:31,555][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 02:26:42,308][main][INFO] - ---


[2025-10-29 02:26:42,309][main][INFO] - [T_total=63:07:21 | T_train=61:50:04] Start epoch 20
[2025-10-29 05:32:23,347][main][INFO] - [T_total=66:13:02 | T_train=64:55:45 | T_epoch=03:05:41] End of epoch 20 (139986 steps) train loss 0.949199
[2025-10-29 05:32:23,349][main][INFO] - [Epoch 20] All losses: [[diffusion=0.07719 ; kl=667540 ; lpips=0.174758 ; repa=0.468358]]
[2025-10-29 05:35:50,507][main][INFO] - [Epoch 21] Test metrics: [[MSE=23.35 | MAE=0.1078 | LPIPS=0.1576 | PSNR=16.32 | SSIM=0.4284 | dreamsim=0.2173 | FID=15.02]]
[2025-10-29 05:35:50,509][main][INFO] - [Epoch 21] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1576 | max_PSNR=16.4 | max_SSIM=0.4284 | min_dreamsim=0.2173 | min_FID=15.02]]
[2025-10-29 05:35:50,510][main][DEBUG] - Writing images to disk...
[2025-10-29 05:35:51,347][main][DEBUG] - Image(s) saved on disk
[2025-10-29 05:35:51,557][main][INFO] - End of epoch timers: [T_train=64:55:45 | T_epoch=03:05:41 | T_eval=01:12:55 | T_total=66:16:31]
[2025-10-29 05:35:51,558][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 05:36:02,284][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 05:36:12,666][main][INFO] - ---


[2025-10-29 05:36:12,667][main][INFO] - [T_total=66:16:52 | T_train=64:55:45] Start epoch 21
[2025-10-29 08:42:54,495][main][INFO] - [T_total=69:23:34 | T_train=68:02:26 | T_epoch=03:06:41] End of epoch 21 (146652 steps) train loss 289.216
[2025-10-29 08:42:54,496][main][INFO] - [Epoch 21] All losses: [[diffusion=0.0776409 ; kl=2.88933e+08 ; lpips=0.176763 ; repa=0.469242]]
[2025-10-29 08:46:22,241][main][INFO] - [Epoch 22] Test metrics: [[MSE=23.57 | MAE=0.1086 | LPIPS=0.1573 | PSNR=16.28 | SSIM=0.4288 | dreamsim=0.2159 | FID=14.7]]
[2025-10-29 08:46:22,243][main][INFO] - [Epoch 22] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1573 | max_PSNR=16.4 | max_SSIM=0.4288 | min_dreamsim=0.2159 | min_FID=14.7]]
[2025-10-29 08:46:22,244][main][DEBUG] - Writing images to disk...
[2025-10-29 08:46:23,102][main][DEBUG] - Image(s) saved on disk
[2025-10-29 08:46:23,340][main][INFO] - End of epoch timers: [T_train=68:02:26 | T_epoch=03:06:41 | T_eval=01:16:23 | T_total=69:27:02]
[2025-10-29 08:46:23,341][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 08:46:34,825][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 08:46:45,943][main][INFO] - ---


[2025-10-29 08:46:45,944][main][INFO] - [T_total=69:27:25 | T_train=68:02:26] Start epoch 22
[2025-10-29 11:52:25,547][main][INFO] - [T_total=72:33:05 | T_train=71:08:06 | T_epoch=03:05:39] End of epoch 22 (153318 steps) train loss 6.70619e+06
[2025-10-29 11:52:25,549][main][INFO] - [Epoch 22] All losses: [[diffusion=0.0782495 ; kl=6.70619e+12 ; lpips=0.180401 ; repa=0.471485]]
[2025-10-29 11:55:52,869][main][INFO] - [Epoch 23] Test metrics: [[MSE=23.7 | MAE=0.109 | LPIPS=0.157 | PSNR=16.25 | SSIM=0.4303 | dreamsim=0.2147 | FID=14.49]]
[2025-10-29 11:55:52,871][main][INFO] - [Epoch 23] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.157 | max_PSNR=16.4 | max_SSIM=0.4303 | min_dreamsim=0.2147 | min_FID=14.49]]
[2025-10-29 11:55:52,872][main][DEBUG] - Writing images to disk...
[2025-10-29 11:55:53,703][main][DEBUG] - Image(s) saved on disk
[2025-10-29 11:55:53,941][main][INFO] - End of epoch timers: [T_train=71:08:06 | T_epoch=03:05:39 | T_eval=01:19:51 | T_total=72:36:33]
[2025-10-29 11:55:53,943][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 11:56:05,150][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 11:56:16,440][main][INFO] - ---


[2025-10-29 11:56:16,441][main][INFO] - [T_total=72:36:55 | T_train=71:08:06] Start epoch 23
[2025-10-29 15:01:26,171][main][INFO] - [T_total=75:42:05 | T_train=74:13:16 | T_epoch=03:05:09] End of epoch 23 (159984 steps) train loss 168.462
[2025-10-29 15:01:26,172][main][INFO] - [Epoch 23] All losses: [[diffusion=0.0781783 ; kl=1.68175e+08 ; lpips=0.18157 ; repa=0.471573]]
[2025-10-29 15:04:53,505][main][INFO] - [Epoch 24] Test metrics: [[MSE=23.89 | MAE=0.1096 | LPIPS=0.1566 | PSNR=16.22 | SSIM=0.4302 | dreamsim=0.2137 | FID=14.27]]
[2025-10-29 15:04:53,507][main][INFO] - [Epoch 24] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1566 | max_PSNR=16.4 | max_SSIM=0.4303 | min_dreamsim=0.2137 | min_FID=14.27]]
[2025-10-29 15:04:53,508][main][DEBUG] - Writing images to disk...
[2025-10-29 15:04:54,339][main][DEBUG] - Image(s) saved on disk
[2025-10-29 15:04:54,543][main][INFO] - End of epoch timers: [T_train=74:13:16 | T_epoch=03:05:09 | T_eval=01:23:20 | T_total=75:45:34]
[2025-10-29 15:04:54,544][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 15:05:05,591][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 15:05:16,848][main][INFO] - ---


[2025-10-29 15:05:16,849][main][INFO] - [T_total=75:45:56 | T_train=74:13:16] Start epoch 24
[2025-10-29 18:10:51,689][main][INFO] - [T_total=78:51:31 | T_train=77:18:51 | T_epoch=03:05:34] End of epoch 24 (166650 steps) train loss 1.72129
[2025-10-29 18:10:51,690][main][INFO] - [Epoch 24] All losses: [[diffusion=0.0766599 ; kl=1.44158e+06 ; lpips=0.173953 ; repa=0.464302]]
[2025-10-29 18:14:18,759][main][INFO] - [Epoch 25] Test metrics: [[MSE=24.04 | MAE=0.1101 | LPIPS=0.1564 | PSNR=16.19 | SSIM=0.4306 | dreamsim=0.2126 | FID=13.99]]
[2025-10-29 18:14:18,761][main][INFO] - [Epoch 25] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1564 | max_PSNR=16.4 | max_SSIM=0.4306 | min_dreamsim=0.2126 | min_FID=13.99]]
[2025-10-29 18:14:18,762][main][DEBUG] - Writing images to disk...
[2025-10-29 18:14:19,594][main][DEBUG] - Image(s) saved on disk
[2025-10-29 18:14:19,796][main][INFO] - End of epoch timers: [T_train=77:18:51 | T_epoch=03:05:34 | T_eval=01:26:48 | T_total=78:54:59]
[2025-10-29 18:14:19,797][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 18:14:31,104][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 18:14:42,095][main][INFO] - ---


[2025-10-29 18:14:42,095][main][INFO] - [T_total=78:55:21 | T_train=77:18:51] Start epoch 25
[2025-10-29 21:19:56,417][main][INFO] - [T_total=82:00:35 | T_train=80:24:05 | T_epoch=03:05:14] End of epoch 25 (173316 steps) train loss 2.83654e+07
[2025-10-29 21:19:56,418][main][INFO] - [Epoch 25] All losses: [[diffusion=0.0773898 ; kl=2.83654e+13 ; lpips=0.17699 ; repa=0.466391]]
[2025-10-29 21:23:23,926][main][INFO] - [Epoch 26] Test metrics: [[MSE=24.22 | MAE=0.1107 | LPIPS=0.1564 | PSNR=16.16 | SSIM=0.4312 | dreamsim=0.2119 | FID=13.81]]
[2025-10-29 21:23:23,928][main][INFO] - [Epoch 26] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1564 | max_PSNR=16.4 | max_SSIM=0.4312 | min_dreamsim=0.2119 | min_FID=13.81]]
[2025-10-29 21:23:23,929][main][DEBUG] - Writing images to disk...
[2025-10-29 21:23:25,021][main][DEBUG] - Image(s) saved on disk
[2025-10-29 21:23:25,253][main][INFO] - End of epoch timers: [T_train=80:24:05 | T_epoch=03:05:14 | T_eval=01:30:16 | T_total=82:04:04]
[2025-10-29 21:23:25,254][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-29 21:23:36,113][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-29 21:23:47,503][main][INFO] - ---


[2025-10-29 21:23:47,504][main][INFO] - [T_total=82:04:27 | T_train=80:24:05] Start epoch 26
[2025-10-30 00:29:03,175][main][INFO] - [T_total=85:09:42 | T_train=83:29:21 | T_epoch=03:05:15] End of epoch 26 (179982 steps) train loss 2.92682
[2025-10-30 00:29:03,176][main][INFO] - [Epoch 26] All losses: [[diffusion=0.076385 ; kl=2.6494e+06 ; lpips=0.171497 ; repa=0.46114]]
[2025-10-30 00:32:30,822][main][INFO] - [Epoch 27] Test metrics: [[MSE=24.25 | MAE=0.1109 | LPIPS=0.1556 | PSNR=16.15 | SSIM=0.4313 | dreamsim=0.2106 | FID=13.64]]
[2025-10-30 00:32:30,823][main][INFO] - [Epoch 27] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1556 | max_PSNR=16.4 | max_SSIM=0.4313 | min_dreamsim=0.2106 | min_FID=13.64]]
[2025-10-30 00:32:30,825][main][DEBUG] - Writing images to disk...
[2025-10-30 00:32:31,903][main][DEBUG] - Image(s) saved on disk
[2025-10-30 00:32:32,151][main][INFO] - End of epoch timers: [T_train=83:29:21 | T_epoch=03:05:15 | T_eval=01:33:45 | T_total=85:13:11]
[2025-10-30 00:32:32,152][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 00:32:42,864][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 00:32:54,008][main][INFO] - ---


[2025-10-30 00:32:54,009][main][INFO] - [T_total=85:13:33 | T_train=83:29:21] Start epoch 27
[2025-10-30 03:37:54,148][main][INFO] - [T_total=88:18:33 | T_train=86:34:21 | T_epoch=03:05:00] End of epoch 27 (186648 steps) train loss 0.940592
[2025-10-30 03:37:54,149][main][INFO] - [Epoch 27] All losses: [[diffusion=0.0759206 ; kl=665087 ; lpips=0.169737 ; repa=0.458863]]
[2025-10-30 03:41:21,260][main][INFO] - [Epoch 28] Test metrics: [[MSE=24.31 | MAE=0.1112 | LPIPS=0.1553 | PSNR=16.14 | SSIM=0.4322 | dreamsim=0.2096 | FID=13.42]]
[2025-10-30 03:41:21,262][main][INFO] - [Epoch 28] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1553 | max_PSNR=16.4 | max_SSIM=0.4322 | min_dreamsim=0.2096 | min_FID=13.42]]
[2025-10-30 03:41:21,263][main][DEBUG] - Writing images to disk...
[2025-10-30 03:41:22,104][main][DEBUG] - Image(s) saved on disk
[2025-10-30 03:41:22,371][main][INFO] - End of epoch timers: [T_train=86:34:21 | T_epoch=03:05:00 | T_eval=01:37:13 | T_total=88:22:01]
[2025-10-30 03:41:22,372][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 03:41:34,752][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 03:41:46,656][main][INFO] - ---


[2025-10-30 03:41:46,657][main][INFO] - [T_total=88:22:26 | T_train=86:34:21] Start epoch 28
[2025-10-30 06:47:29,123][main][INFO] - [T_total=91:28:08 | T_train=89:40:03 | T_epoch=03:05:42] End of epoch 28 (193314 steps) train loss 37.7653
[2025-10-30 06:47:29,124][main][INFO] - [Epoch 28] All losses: [[diffusion=0.0768294 ; kl=3.74862e+07 ; lpips=0.173765 ; repa=0.461866]]
[2025-10-30 06:50:55,982][main][INFO] - [Epoch 29] Test metrics: [[MSE=24.39 | MAE=0.1115 | LPIPS=0.1553 | PSNR=16.13 | SSIM=0.4326 | dreamsim=0.209 | FID=13.31]]
[2025-10-30 06:50:55,983][main][INFO] - [Epoch 29] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1553 | max_PSNR=16.4 | max_SSIM=0.4326 | min_dreamsim=0.209 | min_FID=13.31]]
[2025-10-30 06:50:55,984][main][DEBUG] - Writing images to disk...
[2025-10-30 06:50:56,817][main][DEBUG] - Image(s) saved on disk
[2025-10-30 06:50:57,019][main][INFO] - End of epoch timers: [T_train=89:40:03 | T_epoch=03:05:42 | T_eval=01:40:41 | T_total=91:31:36]
[2025-10-30 06:50:57,021][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 06:51:07,339][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 06:51:16,820][main][INFO] - ---


[2025-10-30 06:51:16,821][main][INFO] - [T_total=91:31:56 | T_train=89:40:03] Start epoch 29
[2025-10-30 09:56:41,969][main][INFO] - [T_total=94:37:21 | T_train=92:45:28 | T_epoch=03:05:25] End of epoch 29 (199980 steps) train loss 422
[2025-10-30 09:56:41,970][main][INFO] - [Epoch 29] All losses: [[diffusion=0.0763346 ; kl=4.21723e+08 ; lpips=0.171742 ; repa=0.459557]]
[2025-10-30 10:00:09,392][main][INFO] - [Epoch 30] Test metrics: [[MSE=24.44 | MAE=0.1117 | LPIPS=0.1547 | PSNR=16.12 | SSIM=0.4334 | dreamsim=0.2078 | FID=13.08]]
[2025-10-30 10:00:09,394][main][INFO] - [Epoch 30] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1547 | max_PSNR=16.4 | max_SSIM=0.4334 | min_dreamsim=0.2078 | min_FID=13.08]]
[2025-10-30 10:00:09,396][main][DEBUG] - Writing images to disk...
[2025-10-30 10:00:10,246][main][DEBUG] - Image(s) saved on disk
[2025-10-30 10:00:10,452][main][INFO] - End of epoch timers: [T_train=92:45:28 | T_epoch=03:05:25 | T_eval=01:44:09 | T_total=94:40:49]
[2025-10-30 10:00:10,453][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 10:00:21,119][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 10:00:31,248][main][INFO] - ---


[2025-10-30 10:00:31,249][main][INFO] - [T_total=94:41:10 | T_train=92:45:28] Start epoch 30
[2025-10-30 13:06:42,974][main][INFO] - [T_total=97:47:22 | T_train=95:51:40 | T_epoch=03:06:11] End of epoch 30 (206646 steps) train loss 255193
[2025-10-30 13:06:42,975][main][INFO] - [Epoch 30] All losses: [[diffusion=0.0769152 ; kl=2.55193e+11 ; lpips=0.174071 ; repa=0.460975]]
[2025-10-30 13:10:10,113][main][INFO] - [Epoch 31] Test metrics: [[MSE=24.51 | MAE=0.112 | LPIPS=0.1546 | PSNR=16.11 | SSIM=0.4338 | dreamsim=0.2072 | FID=12.95]]
[2025-10-30 13:10:10,116][main][INFO] - [Epoch 31] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1546 | max_PSNR=16.4 | max_SSIM=0.4338 | min_dreamsim=0.2072 | min_FID=12.95]]
[2025-10-30 13:10:10,117][main][DEBUG] - Writing images to disk...
[2025-10-30 13:10:10,957][main][DEBUG] - Image(s) saved on disk
[2025-10-30 13:10:11,195][main][INFO] - End of epoch timers: [T_train=95:51:40 | T_epoch=03:06:11 | T_eval=01:47:37 | T_total=97:50:50]
[2025-10-30 13:10:11,197][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 13:10:21,570][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 13:10:31,290][main][INFO] - ---


[2025-10-30 13:10:31,291][main][INFO] - [T_total=97:51:10 | T_train=95:51:40] Start epoch 31
[2025-10-30 16:15:53,375][main][INFO] - [T_total=100:56:32 | T_train=98:57:02 | T_epoch=03:05:22] End of epoch 31 (213312 steps) train loss 29024
[2025-10-30 16:15:53,376][main][INFO] - [Epoch 31] All losses: [[diffusion=0.0770869 ; kl=2.90237e+10 ; lpips=0.1765 ; repa=0.462709]]
[2025-10-30 16:19:20,850][main][INFO] - [Epoch 32] Test metrics: [[MSE=24.52 | MAE=0.1121 | LPIPS=0.1545 | PSNR=16.1 | SSIM=0.4351 | dreamsim=0.2065 | FID=12.85]]
[2025-10-30 16:19:20,852][main][INFO] - [Epoch 32] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1545 | max_PSNR=16.4 | max_SSIM=0.4351 | min_dreamsim=0.2065 | min_FID=12.85]]
[2025-10-30 16:19:20,854][main][DEBUG] - Writing images to disk...
[2025-10-30 16:19:21,697][main][DEBUG] - Image(s) saved on disk
[2025-10-30 16:19:21,942][main][INFO] - End of epoch timers: [T_train=98:57:02 | T_epoch=03:05:22 | T_eval=01:51:06 | T_total=101:00:01]
[2025-10-30 16:19:21,942][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 16:19:33,022][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 16:19:43,573][main][INFO] - ---


[2025-10-30 16:19:43,574][main][INFO] - [T_total=101:00:23 | T_train=98:57:02] Start epoch 32
[2025-10-30 19:24:58,751][main][INFO] - [T_total=104:05:38 | T_train=102:02:17 | T_epoch=03:05:15] End of epoch 32 (219978 steps) train loss 20.2597
[2025-10-30 19:24:58,752][main][INFO] - [Epoch 32] All losses: [[diffusion=0.0767827 ; kl=1.99806e+07 ; lpips=0.174456 ; repa=0.460265]]
[2025-10-30 19:28:26,469][main][INFO] - [Epoch 33] Test metrics: [[MSE=24.6 | MAE=0.1125 | LPIPS=0.1544 | PSNR=16.09 | SSIM=0.4346 | dreamsim=0.206 | FID=12.76]]
[2025-10-30 19:28:26,471][main][INFO] - [Epoch 33] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1544 | max_PSNR=16.4 | max_SSIM=0.4351 | min_dreamsim=0.206 | min_FID=12.76]]
[2025-10-30 19:28:26,473][main][DEBUG] - Writing images to disk...
[2025-10-30 19:28:27,304][main][DEBUG] - Image(s) saved on disk
[2025-10-30 19:28:27,556][main][INFO] - End of epoch timers: [T_train=102:02:17 | T_epoch=03:05:15 | T_eval=01:54:34 | T_total=104:09:07]
[2025-10-30 19:28:27,558][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 19:28:38,944][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 19:28:50,181][main][INFO] - ---


[2025-10-30 19:28:50,182][main][INFO] - [T_total=104:09:29 | T_train=102:02:17] Start epoch 33
[2025-10-30 22:34:36,461][main][INFO] - [T_total=107:15:16 | T_train=105:08:04 | T_epoch=03:05:46] End of epoch 33 (226644 steps) train loss 176.126
[2025-10-30 22:34:36,462][main][INFO] - [Epoch 33] All losses: [[diffusion=0.0773723 ; kl=1.75845e+08 ; lpips=0.176956 ; repa=0.462503]]
[2025-10-30 22:38:03,690][main][INFO] - [Epoch 34] Test metrics: [[MSE=24.58 | MAE=0.1124 | LPIPS=0.1542 | PSNR=16.09 | SSIM=0.4358 | dreamsim=0.2053 | FID=12.61]]
[2025-10-30 22:38:03,692][main][INFO] - [Epoch 34] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1542 | max_PSNR=16.4 | max_SSIM=0.4358 | min_dreamsim=0.2053 | min_FID=12.61]]
[2025-10-30 22:38:03,692][main][DEBUG] - Writing images to disk...
[2025-10-30 22:38:04,540][main][DEBUG] - Image(s) saved on disk
[2025-10-30 22:38:04,752][main][INFO] - End of epoch timers: [T_train=105:08:04 | T_epoch=03:05:46 | T_eval=01:58:02 | T_total=107:18:44]
[2025-10-30 22:38:04,753][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-30 22:38:16,516][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-30 22:38:27,966][main][INFO] - ---


[2025-10-30 22:38:27,967][main][INFO] - [T_total=107:19:07 | T_train=105:08:04] Start epoch 34
[2025-10-31 01:43:52,689][main][INFO] - [T_total=110:24:32 | T_train=108:13:28 | T_epoch=03:05:24] End of epoch 34 (233310 steps) train loss 3098.35
[2025-10-31 01:43:52,690][main][INFO] - [Epoch 34] All losses: [[diffusion=0.0765349 ; kl=3.09807e+09 ; lpips=0.174148 ; repa=0.459381]]
[2025-10-31 01:47:20,181][main][INFO] - [Epoch 35] Test metrics: [[MSE=24.56 | MAE=0.1125 | LPIPS=0.1538 | PSNR=16.1 | SSIM=0.4369 | dreamsim=0.2045 | FID=12.45]]
[2025-10-31 01:47:20,183][main][INFO] - [Epoch 35] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1538 | max_PSNR=16.4 | max_SSIM=0.4369 | min_dreamsim=0.2045 | min_FID=12.45]]
[2025-10-31 01:47:20,184][main][DEBUG] - Writing images to disk...
[2025-10-31 01:47:21,284][main][DEBUG] - Image(s) saved on disk
[2025-10-31 01:47:21,496][main][INFO] - End of epoch timers: [T_train=108:13:28 | T_epoch=03:05:24 | T_eval=02:01:31 | T_total=110:28:01]
[2025-10-31 01:47:21,497][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 01:47:32,897][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 01:47:43,716][main][INFO] - ---


[2025-10-31 01:47:43,717][main][INFO] - [T_total=110:28:23 | T_train=108:13:28] Start epoch 35
[2025-10-31 04:53:48,886][main][INFO] - [T_total=113:34:28 | T_train=111:19:33 | T_epoch=03:06:05] End of epoch 35 (239976 steps) train loss 44306.2
[2025-10-31 04:53:48,887][main][INFO] - [Epoch 35] All losses: [[diffusion=0.0759655 ; kl=4.43059e+10 ; lpips=0.170242 ; repa=0.455204]]
[2025-10-31 04:57:16,489][main][INFO] - [Epoch 36] Test metrics: [[MSE=24.6 | MAE=0.1126 | LPIPS=0.1535 | PSNR=16.09 | SSIM=0.4367 | dreamsim=0.2038 | FID=12.31]]
[2025-10-31 04:57:16,491][main][INFO] - [Epoch 36] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1535 | max_PSNR=16.4 | max_SSIM=0.4369 | min_dreamsim=0.2038 | min_FID=12.31]]
[2025-10-31 04:57:16,493][main][DEBUG] - Writing images to disk...
[2025-10-31 04:57:17,591][main][DEBUG] - Image(s) saved on disk
[2025-10-31 04:57:17,835][main][INFO] - End of epoch timers: [T_train=111:19:33 | T_epoch=03:06:05 | T_eval=02:05:00 | T_total=113:37:57]
[2025-10-31 04:57:17,836][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 04:57:28,300][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 04:57:38,419][main][INFO] - ---


[2025-10-31 04:57:38,420][main][INFO] - [T_total=113:38:17 | T_train=111:19:33] Start epoch 36
[2025-10-31 08:03:40,896][main][INFO] - [T_total=116:44:20 | T_train=114:25:36 | T_epoch=03:06:02] End of epoch 36 (246642 steps) train loss 13991
[2025-10-31 08:03:40,897][main][INFO] - [Epoch 36] All losses: [[diffusion=0.0760433 ; kl=1.39907e+10 ; lpips=0.169438 ; repa=0.454577]]
[2025-10-31 08:07:07,981][main][INFO] - [Epoch 37] Test metrics: [[MSE=24.59 | MAE=0.1126 | LPIPS=0.1533 | PSNR=16.09 | SSIM=0.4386 | dreamsim=0.2031 | FID=12.18]]
[2025-10-31 08:07:07,982][main][INFO] - [Epoch 37] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1533 | max_PSNR=16.4 | max_SSIM=0.4386 | min_dreamsim=0.2031 | min_FID=12.18]]
[2025-10-31 08:07:07,983][main][DEBUG] - Writing images to disk...
[2025-10-31 08:07:08,817][main][DEBUG] - Image(s) saved on disk
[2025-10-31 08:07:09,048][main][INFO] - End of epoch timers: [T_train=114:25:36 | T_epoch=03:06:02 | T_eval=02:08:28 | T_total=116:47:48]
[2025-10-31 08:07:09,049][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 08:07:19,835][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 08:07:30,639][main][INFO] - ---


[2025-10-31 08:07:30,640][main][INFO] - [T_total=116:48:10 | T_train=114:25:36] Start epoch 37
[2025-10-31 11:12:53,371][main][INFO] - [T_total=119:53:32 | T_train=117:30:59 | T_epoch=03:05:22] End of epoch 37 (253308 steps) train loss 13991.7
[2025-10-31 11:12:53,372][main][INFO] - [Epoch 37] All losses: [[diffusion=0.0767062 ; kl=1.39915e+10 ; lpips=0.174108 ; repa=0.4581]]
[2025-10-31 11:16:19,970][main][INFO] - [Epoch 38] Test metrics: [[MSE=24.53 | MAE=0.1125 | LPIPS=0.1528 | PSNR=16.1 | SSIM=0.4386 | dreamsim=0.2024 | FID=12.07]]
[2025-10-31 11:16:19,972][main][INFO] - [Epoch 38] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1528 | max_PSNR=16.4 | max_SSIM=0.4386 | min_dreamsim=0.2024 | min_FID=12.07]]
[2025-10-31 11:16:19,973][main][DEBUG] - Writing images to disk...
[2025-10-31 11:16:20,809][main][DEBUG] - Image(s) saved on disk
[2025-10-31 11:16:21,014][main][INFO] - End of epoch timers: [T_train=117:30:59 | T_epoch=03:05:22 | T_eval=02:11:55 | T_total=119:57:00]
[2025-10-31 11:16:21,015][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 11:16:32,831][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 11:16:44,606][main][INFO] - ---


[2025-10-31 11:16:44,607][main][INFO] - [T_total=119:57:24 | T_train=117:30:59] Start epoch 38
[2025-10-31 14:21:53,789][main][INFO] - [T_total=123:02:33 | T_train=120:36:08 | T_epoch=03:05:09] End of epoch 38 (259974 steps) train loss 957.097
[2025-10-31 14:21:53,790][main][INFO] - [Epoch 38] All losses: [[diffusion=0.0761426 ; kl=9.56822e+08 ; lpips=0.170405 ; repa=0.454693]]
[2025-10-31 14:25:20,866][main][INFO] - [Epoch 39] Test metrics: [[MSE=24.69 | MAE=0.1129 | LPIPS=0.1528 | PSNR=16.07 | SSIM=0.4388 | dreamsim=0.2019 | FID=11.95]]
[2025-10-31 14:25:20,868][main][INFO] - [Epoch 39] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1528 | max_PSNR=16.4 | max_SSIM=0.4388 | min_dreamsim=0.2019 | min_FID=11.95]]
[2025-10-31 14:25:20,869][main][DEBUG] - Writing images to disk...
[2025-10-31 14:25:21,702][main][DEBUG] - Image(s) saved on disk
[2025-10-31 14:25:21,953][main][INFO] - End of epoch timers: [T_train=120:36:08 | T_epoch=03:05:09 | T_eval=02:15:23 | T_total=123:06:01]
[2025-10-31 14:25:21,955][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 14:25:32,079][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 14:25:43,132][main][INFO] - ---


[2025-10-31 14:25:43,136][main][INFO] - [T_total=123:06:22 | T_train=120:36:08] Start epoch 39
[2025-10-31 17:30:32,200][main][INFO] - [T_total=126:11:11 | T_train=123:40:57 | T_epoch=03:04:49] End of epoch 39 (266640 steps) train loss 2134.48
[2025-10-31 17:30:32,202][main][INFO] - [Epoch 39] All losses: [[diffusion=0.076013 ; kl=2.1342e+09 ; lpips=0.169987 ; repa=0.453857]]
[2025-10-31 17:33:59,741][main][INFO] - [Epoch 40] Test metrics: [[MSE=24.76 | MAE=0.1132 | LPIPS=0.1528 | PSNR=16.06 | SSIM=0.4394 | dreamsim=0.2017 | FID=11.91]]
[2025-10-31 17:33:59,743][main][INFO] - [Epoch 40] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1528 | max_PSNR=16.4 | max_SSIM=0.4394 | min_dreamsim=0.2017 | min_FID=11.91]]
[2025-10-31 17:33:59,744][main][DEBUG] - Writing images to disk...
[2025-10-31 17:34:00,581][main][DEBUG] - Image(s) saved on disk
[2025-10-31 17:34:00,789][main][INFO] - End of epoch timers: [T_train=123:40:57 | T_epoch=03:04:49 | T_eval=02:18:52 | T_total=126:14:40]
[2025-10-31 17:34:00,790][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 17:34:11,470][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 17:34:20,752][main][INFO] - ---


[2025-10-31 17:34:20,753][main][INFO] - [T_total=126:15:00 | T_train=123:40:57] Start epoch 40
[2025-10-31 20:39:19,195][main][INFO] - [T_total=129:19:58 | T_train=126:45:55 | T_epoch=03:04:58] End of epoch 40 (273306 steps) train loss 257946
[2025-10-31 20:39:19,196][main][INFO] - [Epoch 40] All losses: [[diffusion=0.0760113 ; kl=2.57946e+11 ; lpips=0.170735 ; repa=0.454041]]
[2025-10-31 20:42:46,266][main][INFO] - [Epoch 41] Test metrics: [[MSE=24.79 | MAE=0.1133 | LPIPS=0.1526 | PSNR=16.06 | SSIM=0.4406 | dreamsim=0.2009 | FID=11.78]]
[2025-10-31 20:42:46,268][main][INFO] - [Epoch 41] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1526 | max_PSNR=16.4 | max_SSIM=0.4406 | min_dreamsim=0.2009 | min_FID=11.78]]
[2025-10-31 20:42:46,269][main][DEBUG] - Writing images to disk...
[2025-10-31 20:42:47,095][main][DEBUG] - Image(s) saved on disk
[2025-10-31 20:42:47,343][main][INFO] - End of epoch timers: [T_train=126:45:55 | T_epoch=03:04:58 | T_eval=02:22:20 | T_total=129:23:26]
[2025-10-31 20:42:47,345][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 20:42:58,982][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 20:43:08,773][main][INFO] - ---


[2025-10-31 20:43:08,774][main][INFO] - [T_total=129:23:48 | T_train=126:45:55] Start epoch 41
[2025-10-31 23:47:55,833][main][INFO] - [T_total=132:28:35 | T_train=129:50:42 | T_epoch=03:04:47] End of epoch 41 (279972 steps) train loss 14.3203
[2025-10-31 23:47:55,839][main][INFO] - [Epoch 41] All losses: [[diffusion=0.0761964 ; kl=1.40445e+07 ; lpips=0.171818 ; repa=0.454763]]
[2025-10-31 23:51:24,059][main][INFO] - [Epoch 42] Test metrics: [[MSE=24.9 | MAE=0.1136 | LPIPS=0.1528 | PSNR=16.04 | SSIM=0.4402 | dreamsim=0.2009 | FID=11.76]]
[2025-10-31 23:51:24,061][main][INFO] - [Epoch 42] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1526 | max_PSNR=16.4 | max_SSIM=0.4406 | min_dreamsim=0.2009 | min_FID=11.76]]
[2025-10-31 23:51:24,063][main][DEBUG] - Writing images to disk...
[2025-10-31 23:51:24,900][main][DEBUG] - Image(s) saved on disk
[2025-10-31 23:51:25,140][main][INFO] - End of epoch timers: [T_train=129:50:42 | T_epoch=03:04:47 | T_eval=02:25:49 | T_total=132:32:04]
[2025-10-31 23:51:25,141][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-10-31 23:51:35,044][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-10-31 23:51:44,524][main][INFO] - ---


[2025-10-31 23:51:44,525][main][INFO] - [T_total=132:32:24 | T_train=129:50:42] Start epoch 42
[2025-11-01 02:57:08,828][main][INFO] - [T_total=135:37:48 | T_train=132:56:07 | T_epoch=03:05:24] End of epoch 42 (286638 steps) train loss 27710.1
[2025-11-01 02:57:08,829][main][INFO] - [Epoch 42] All losses: [[diffusion=0.0763487 ; kl=2.77098e+10 ; lpips=0.172949 ; repa=0.455514]]
[2025-11-01 03:00:36,341][main][INFO] - [Epoch 43] Test metrics: [[MSE=24.85 | MAE=0.1135 | LPIPS=0.1524 | PSNR=16.05 | SSIM=0.4412 | dreamsim=0.2003 | FID=11.67]]
[2025-11-01 03:00:36,343][main][INFO] - [Epoch 43] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1524 | max_PSNR=16.4 | max_SSIM=0.4412 | min_dreamsim=0.2003 | min_FID=11.67]]
[2025-11-01 03:00:36,344][main][DEBUG] - Writing images to disk...
[2025-11-01 03:00:37,429][main][DEBUG] - Image(s) saved on disk
[2025-11-01 03:00:37,669][main][INFO] - End of epoch timers: [T_train=132:56:07 | T_epoch=03:05:24 | T_eval=02:29:17 | T_total=135:41:17]
[2025-11-01 03:00:37,671][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-11-01 03:00:48,523][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-11-01 03:00:59,238][main][INFO] - ---


[2025-11-01 03:00:59,239][main][INFO] - [T_total=135:41:38 | T_train=132:56:07] Start epoch 43
[2025-11-01 06:06:17,238][main][INFO] - [T_total=138:46:56 | T_train=136:01:25 | T_epoch=03:05:17] End of epoch 43 (293304 steps) train loss 321.888
[2025-11-01 06:06:17,239][main][INFO] - [Epoch 43] All losses: [[diffusion=0.075849 ; kl=3.21614e+08 ; lpips=0.170029 ; repa=0.452597]]
[2025-11-01 06:09:44,706][main][INFO] - [Epoch 44] Test metrics: [[MSE=24.88 | MAE=0.1137 | LPIPS=0.1522 | PSNR=16.04 | SSIM=0.4413 | dreamsim=0.1998 | FID=11.54]]
[2025-11-01 06:09:44,708][main][INFO] - [Epoch 44] Best metrics: [[min_MSE=22.91 | min_MAE=0.1058 | min_LPIPS=0.1522 | max_PSNR=16.4 | max_SSIM=0.4413 | min_dreamsim=0.1998 | min_FID=11.54]]
[2025-11-01 06:09:44,709][main][DEBUG] - Writing images to disk...
[2025-11-01 06:09:45,799][main][DEBUG] - Image(s) saved on disk
[2025-11-01 06:09:46,004][main][INFO] - End of epoch timers: [T_train=136:01:25 | T_epoch=03:05:17 | T_eval=02:32:46 | T_total=138:50:25]
[2025-11-01 06:09:46,005][main][INFO] - Storing model checkpoint inside /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/last
[2025-11-01 06:09:57,354][main][INFO] - Best FID so far, storing a copy of the model checkpoint to /workspace/DC_SSDAE/runs/jobs/train_enc_dc_f32c32_EqM/checkpoints/best
[2025-11-01 06:10:08,846][main][INFO] - ---


[2025-11-01 06:10:08,847][main][INFO] - [T_total=138:50:48 | T_train=136:01:25] Start epoch 44