ssl_dino/swin_tiny/train.log

[2025-08-15 05:14:14,328][INFO] Args: {
    "accelerator": "auto",
    "batch_size": 32,
    "callbacks": null,
    "checkpoint": null,
    "data": "data/kyucapsule",
    "devices": "auto",
    "embed_dim": null,
    "epochs": 300,
    "loader_args": null,
    "loggers": {
        "wandb": {
            "project": "ent-endoscopy-ssl"
        }
    },
    "method": "dino",
    "method_args": null,
    "model": "SwinTransformer",
    "model_args": null,
    "num_nodes": 1,
    "num_workers": 64,
    "optim": "auto",
    "optim_args": null,
    "out": "outputs/ssl_dino/swin_tiny",
    "overwrite": true,
    "precision": "32-true",
    "resume": true,
    "seed": 0,
    "strategy": "auto",
    "trainer_args": null,
    "transform_args": {
        "image_size": [
            224,
            224
        ],
        "local_view": {
            "num_views": 0
        }
    }
}
[2025-08-15 05:14:14,328][INFO] Using output directory '/workspace/ent-labotary/outputs/ssl_dino/swin_tiny'.
[2025-08-15 05:14:14,563][DEBUG] '/usr/local/lib/python3.12/dist-packages/lightly_train' is not a git repository.
[2025-08-15 05:14:14,573][DEBUG] Platform: Linux-6.5.0-18-generic-x86_64-with-glibc2.39
[2025-08-15 05:14:14,574][DEBUG] Python: 3.12.3
[2025-08-15 05:14:14,574][DEBUG] LightlyTrain: 0.6.1
[2025-08-15 05:14:14,574][DEBUG] LightlyTrain Git Information:
[2025-08-15 05:14:14,574][DEBUG]  LightlyTrain is not installed from a git repository.
[2025-08-15 05:14:14,574][DEBUG] Run directory Git Information:
[2025-08-15 05:14:14,574][DEBUG]  Branch: feat/hfup
[2025-08-15 05:14:14,574][DEBUG]  Commit: a47f8b8122fc2efd40b0fdb826c07983a260b074
[2025-08-15 05:14:14,574][DEBUG]  Uncommitted changes: ?? src/experiment/.ssl_dino.py.swp
[2025-08-15 05:14:14,574][DEBUG] Dependencies:
[2025-08-15 05:14:14,574][DEBUG]  - torch                       2.8.0
[2025-08-15 05:14:14,574][DEBUG]  - torchvision                0.23.0
[2025-08-15 05:14:14,574][DEBUG]  - pytorch-lightning           2.5.3
[2025-08-15 05:14:14,574][DEBUG]  - Pillow                     11.3.0
[2025-08-15 05:14:14,574][DEBUG]  - pillow-simd                     x
[2025-08-15 05:14:14,574][DEBUG] Optional dependencies:
[2025-08-15 05:14:14,574][DEBUG]  - super-gradients                 x
[2025-08-15 05:14:14,574][DEBUG]  - timm                       1.0.19
[2025-08-15 05:14:14,574][DEBUG]  - ultralytics                     x
[2025-08-15 05:14:14,574][DEBUG]  - wandb                      0.21.1
[2025-08-15 05:14:14,574][DEBUG] CPUs: 384
[2025-08-15 05:14:14,574][DEBUG] GPUs: 1
[2025-08-15 05:14:14,574][DEBUG]  - NVIDIA GeForce RTX 5090 12.0 (33669251072)
[2025-08-15 05:14:14,574][DEBUG] Environment variables:
[2025-08-15 05:14:14,575][DEBUG] Getting transform args for method 'dino'.
[2025-08-15 05:14:14,575][DEBUG] Using additional transform arguments {'image_size': (224, 224), 'local_view': {'num_views': 0}}.
[2025-08-15 05:14:14,576][DEBUG] Getting transform for method 'dino'.
[2025-08-15 05:14:14,582][DEBUG] Making sure data directory '/workspace/ent-labotary/data/kyucapsule' exists and is not empty.
[2025-08-15 05:14:14,582][INFO] Initializing dataset from '/workspace/ent-labotary/data/kyucapsule'.
[2025-08-15 05:14:14,582][DEBUG] Writing filenames to '/tmp/tmp1kte4dge' (chunk_size=10000)
[2025-08-15 05:14:15,124][DEBUG] Creating memory mapped sequence with 18481 'filenames'.
[2025-08-15 05:14:15,124][DEBUG] Found dataset size 18481.
[2025-08-15 05:14:15,124][DEBUG] Getting embedding model with embedding dimension None.
[2025-08-15 05:14:15,125][DEBUG] Using jsonl logger with args flush_logs_every_n_steps=100
[2025-08-15 05:14:15,126][DEBUG] Using tensorboard logger with args name='' version='' log_graph=False default_hp_metric=True prefix='' sub_dir=None
[2025-08-15 05:14:15,127][DEBUG] Using wandb logger with args name=None version=None offline=False anonymous=None project='ent-endoscopy-ssl' log_model=False prefix='' checkpoint_name=None
[2025-08-15 05:14:15,127][DEBUG] Using loggers ['JSONLLogger', 'TensorBoardLogger', 'WandbLogger'].
[2025-08-15 05:14:15,128][DEBUG] Getting accelerator for 'auto'.
[2025-08-15 05:14:15,128][DEBUG] CUDA is available, defaulting to CUDA.
[2025-08-15 05:14:15,128][DEBUG] Detected 1 devices.
[2025-08-15 05:14:15,128][DEBUG] Using strategy 'auto'.
[2025-08-15 05:14:15,128][DEBUG] Getting trainer.
[2025-08-15 05:14:15,128][DEBUG] Using sync_batchnorm 'True'.
[2025-08-15 05:14:15,140][INFO] GPU available: True (cuda), used: True
[2025-08-15 05:14:15,141][INFO] TPU available: False, using: 0 TPU cores
[2025-08-15 05:14:15,141][INFO] HPU available: False, using: 0 HPUs
[2025-08-15 05:14:15,141][DEBUG] Detected 1 nodes and 1 devices per node.
[2025-08-15 05:14:15,141][DEBUG] Total number of devices: 1.
[2025-08-15 05:14:15,141][DEBUG] Detected dataset size 18481.
[2025-08-15 05:14:15,141][DEBUG] Using batch size per device 32.
[2025-08-15 05:14:15,141][DEBUG] Using optimizer 'OptimizerType.SGD'.
[2025-08-15 05:14:15,141][DEBUG] Getting method args for 'DINO'
[2025-08-15 05:14:15,142][DEBUG] Getting method for 'DINO'
[2025-08-15 05:14:15,194][WARNING] /usr/local/lib/python3.12/dist-packages/torch/nn/utils/weight_norm.py:144: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)

[2025-08-15 05:14:15,783][INFO] Resolved configuration:
{
    "accelerator": "CUDAAccelerator",
    "batch_size": 32,
    "callbacks": {
        "device_stats_monitor": {},
        "early_stopping": {
            "check_finite": true,
            "monitor": "train_loss",
            "patience": 1000000000000
        },
        "learning_rate_monitor": {},
        "model_checkpoint": {
            "enable_version_counter": false,
            "every_n_epochs": null,
            "save_last": true,
            "save_top_k": 1
        }
    },
    "checkpoint": null,
    "data": "data/kyucapsule",
    "devices": 1,
    "embed_dim": null,
    "epochs": 300,
    "loader_args": null,
    "loggers": {
        "jsonl": {
            "flush_logs_every_n_steps": 100
        },
        "tensorboard": {
            "default_hp_metric": true,
            "log_graph": false,
            "name": "",
            "prefix": "",
            "sub_dir": null,
            "version": ""
        },
        "wandb": {
            "anonymous": null,
            "checkpoint_name": null,
            "log_model": false,
            "name": null,
            "offline": false,
            "prefix": "",
            "project": "ent-endoscopy-ssl",
            "version": null
        }
    },
    "method": "dino",
    "method_args": {
        "batch_norm": false,
        "bottleneck_dim": 256,
        "center_momentum": 0.9,
        "hidden_dim": 2048,
        "momentum_end": 1.0,
        "momentum_start": 0.99,
        "norm_last_layer": true,
        "output_dim": 1024,
        "student_freeze_last_layer_epochs": 1,
        "student_temp": 0.1,
        "teacher_temp": 0.02,
        "warmup_teacher_temp": 0.02,
        "warmup_teacher_temp_epochs": 30,
        "weight_decay_end": 0.0001,
        "weight_decay_start": 0.0001
    },
    "model": "SwinTransformer",
    "model_args": null,
    "num_nodes": 1,
    "num_workers": 64,
    "optim": "sgd",
    "optim_args": {
        "lr": 0.03,
        "momentum": 0.9,
        "weight_decay": 0.0001
    },
    "out": "outputs/ssl_dino/swin_tiny",
    "overwrite": true,
    "precision": "32-true",
    "resume": true,
    "seed": 0,
    "strategy": "SingleDeviceStrategy",
    "trainer_args": null,
    "transform_args": {
        "color_jitter": {
            "brightness": 0.8,
            "contrast": 0.8,
            "hue": 0.2,
            "prob": 0.8,
            "saturation": 0.4,
            "strength": 0.5
        },
        "gaussian_blur": {
            "blur_limit": 0,
            "prob": 1.0,
            "sigmas": [
                0.1,
                2.0
            ]
        },
        "global_view_1": {
            "gaussian_blur": {
                "blur_limit": 0,
                "prob": 0.1,
                "sigmas": [
                    0.1,
                    2.0
                ]
            },
            "solarize": {
                "prob": 0.2,
                "threshold": 0.5
            }
        },
        "image_size": [
            224,
            224
        ],
        "local_view": {
            "gaussian_blur": {
                "blur_limit": 0,
                "prob": 0.5,
                "sigmas": [
                    0.1,
                    2.0
                ]
            },
            "num_views": 0,
            "random_resize": {
                "max_scale": 0.14,
                "min_scale": 0.05
            },
            "view_size": [
                96,
                96
            ]
        },
        "normalize": {
            "mean": [
                0.485,
                0.456,
                0.406
            ],
            "std": [
                0.229,
                0.224,
                0.225
            ]
        },
        "random_flip": {
            "horizontal_prob": 0.5,
            "vertical_prob": 0.0
        },
        "random_gray_scale": 0.2,
        "random_resize": {
            "max_scale": 1.0,
            "min_scale": 0.14
        },
        "random_rotation": null,
        "solarize": null
    }
}
[2025-08-15 05:14:18,140][WARNING] /usr/local/lib/python3.12/dist-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py:190: .fit(ckpt_path="last") is set, but there is no last checkpoint available. No checkpoint will be loaded. HINT: Set `ModelCheckpoint(..., save_last=True)`.

[2025-08-15 05:14:18,141][INFO] You are using a CUDA device ('NVIDIA GeForce RTX 5090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[2025-08-15 05:14:18,297][INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
[2025-08-15 05:14:18,331][INFO] Loading `train_dataloader` to estimate number of stepping batches.
[2025-08-15 05:14:19,711][INFO] 
  | Name                    | Type               | Params | Mode 
-----------------------------------------------------------------------
0 | teacher_embedding_model | EmbeddingModel     | 28.3 M | train
1 | teacher_projection_head | DINOProjectionHead | 6.6 M  | train
2 | student_embedding_model | EmbeddingModel     | 28.3 M | train
3 | student_projection_head | DINOProjectionHead | 6.6 M  | train
4 | flatten                 | Flatten            | 0      | train
5 | criterion               | DINOLoss           | 0      | train
-----------------------------------------------------------------------
69.8 M    Trainable params
2.0 K     Non-trainable params
69.8 M    Total params
279.101   Total estimated model params size (MB)
520       Modules in train mode
0         Modules in eval mode
[2025-08-15 11:33:38,060][INFO] `Trainer.fit` stopped: `max_epochs=300` reached.
[2025-08-15 11:33:39,373][INFO] Training completed.
[2025-08-15 11:33:39,374][DEBUG] Exporting model to '/workspace/ent-labotary/outputs/ssl_dino/swin_tiny/exported_models/exported_last.pt' in format 'ModelFormat.PACKAGE_DEFAULT'.
[2025-08-15 11:33:39,460][INFO] Example: How to use the exported model
----------------------------------------------------------------------------------------
import timm

# Load the pretrained model
model = timm.create_model(
    model_name='swin_s3_tiny_224',
    checkpoint_path='/workspace/ent-labotary/outputs/ssl_dino/swin_tiny/exported_models/exported_last.pt',
)

# Finetune or evaluate the model
...
----------------------------------------------------------------------------------------

[2025-08-15 11:33:39,460][INFO] Model exported.