Add files using upload-large-folder tool

NASH_steato/README.md

@@ -0,0 +1,29 @@
+# NASH_steato
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `NASH_steato_JZ_download`
+- Local checkpoint subfolder: `saved_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["NASH_steato/*"],
+)
+```

NASH_steato/source.txt

@@ -0,0 +1,3 @@
+local_run: NASH_steato_JZ_download
+checkpoint_subdir: saved_model
+hf_checkpoint_name: NASH_steato

NASH_steato/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 125000, "best_metric_to_date": 20.38945737155447}

README.md

@@ -0,0 +1,57 @@
+---
+license: agpl-3.0
+library_name: pytorch
+tags:
+  - static2dynamic
+  - diffusion
+  - generative-modeling
+  - biology
+  - microscopy
+  - image-to-video
+  - trajectory-inference
+pipeline_tag: image-to-video
+---
+
+# Static2Dynamic
+
+Pretrained checkpoints for **Static2Dynamic**, a method for reconstructing videos of unobservable cellular, developmental, and disease processes from static, unpaired snapshots.
+
+Code repository: `https://github.com/biocompibens/static2dynamic`
+
+Paper/preprint: `https://www.biorxiv.org/content/10.64898/2026.05.18.725860v1`
+
+## Checkpoints
+
+Each checkpoint subfolder contains:
+
+- `net/`: diffusion generative model config and weights
+- `video_time_encoder/`: time conditioning encoder config and weights
+- `dynamic/`: scheduler configuration
+- `my_conf/`: run configuration needed to instantiate/evaluate the model
+- `training_state_info.json`: checkpoint/training metadata, when available
+- `source.txt`: local provenance marker for this release
+
+| Checkpoint | Local source run | Local checkpoint |
+|---|---|---|
+| `chromalive` | `chromalive_high_doses` | `best_model` |
+| `docetaxel_skip_half_doses` | `bbbc021_docetaxel_skip_half_doses` | `best_model` |
+| `docetaxel` | `bbbc021_JZ_download` | `saved_model` |
+| `nocodazole` | `bbbc021_nocodazole_no_xattn_no_reweight` | `best_model` |
+| `cell_cycle` | `bbbc048` | `best_model` |
+| `biotine` | `biotine_png_JZ_download` | `saved_model` |
+| `biotine_unpaired` | `biotine_unpaired` | `best_model` |
+| `ependymal` | `ependymal_cutout_noised0.1_fc_augs` | `last_model` |
+| `NASH_steato` | `NASH_steato_JZ_download` | `saved_model` |
+| `diabetic_retinopathy` | `retino_fc_precrop_2048_noreweight_6M` | `best_model` |
+
+## Python download example
+
+Download one checkpoint folder:
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["chromalive/*"],
+)

biotine/README.md

@@ -0,0 +1,29 @@
+# biotine
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `biotine_png_JZ_download`
+- Local checkpoint subfolder: `saved_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["biotine/*"],
+)
+```

biotine/source.txt

@@ -0,0 +1,3 @@
+local_run: biotine_png_JZ_download
+checkpoint_subdir: saved_model
+hf_checkpoint_name: biotine

biotine/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 100000, "best_metric_to_date": 23.474280357337324}

biotine_unpaired/README.md

@@ -0,0 +1,29 @@
+# biotine_unpaired
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `biotine_unpaired`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["biotine_unpaired/*"],
+)
+```

biotine_unpaired/source.txt

@@ -0,0 +1,3 @@
+local_run: biotine_unpaired
+checkpoint_subdir: best_model
+hf_checkpoint_name: biotine_unpaired

biotine_unpaired/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 50000, "best_metric_to_date": 31.223219652098997}

cell_cycle/README.md

@@ -0,0 +1,29 @@
+# cell_cycle
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `bbbc048`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["cell_cycle/*"],
+)
+```

cell_cycle/source.txt

@@ -0,0 +1,3 @@
+local_run: bbbc048
+checkpoint_subdir: best_model
+hf_checkpoint_name: cell_cycle

cell_cycle/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 40000, "best_metric_to_date": 13.254866885791186}

chromalive/README.md

@@ -0,0 +1,29 @@
+# chromalive
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `chromalive_high_doses`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["chromalive/*"],
+)
+```

chromalive/my_conf/dataset/ChromaLive6h/ChromaLive6h_3ch_png_hard_aug.yaml

@@ -0,0 +1,16 @@
+name: chromaLive6h_3ch_png_patches_380px_hard_aug
+path: /projects/static2dynamic/datasets/20231017ChromaLive_6hr_4ch/MIP_normalized/paired_dataset/patches_hard_augmented
+data_shape: [ 3, 128, 128 ]
+transforms:
+  _target_: torchvision.transforms.transforms.Compose
+  transforms:
+  - _target_: torchvision.transforms.transforms.Resize
+    size: 128
+  - _target_: torchvision.transforms.ConvertImageDtype # this also scales to [0; 1]
+    dtype: ${torch_dtype:float32} # passed dtype must be accessible as a "torch" attribute
+  - _target_: torchvision.transforms.Normalize
+    mean: [ 0.5, 0.5, 0.5 ] # move to [-1:1]
+    std: [ 0.5, 0.5, 0.5 ]
+expected_initial_data_range: [ 0, 255 ]
+expected_dtype: torch.uint8
+selected_dists:

chromalive/my_conf/dataset/ChromaLiveTL24h/ChromaLiveTL24h.yaml

@@ -0,0 +1,19 @@
+name: chromalive_tl_24h_380px
+path: /projects/static2dynamic/datasets/20230920ChromaLiveTL_24hr4ch/ch_4_3_1___norm_whole_ds_per_channel_per_zslice_0_99perc___patches_380
+data_shape: [3, 256, 256]
+transforms:
+  _target_: torchvision.transforms.transforms.Compose
+  transforms:
+  - _target_: torchvision.transforms.transforms.Resize
+    size: 256
+  - _target_: torchvision.transforms.ConvertImageDtype # this also scales to [0; 1]
+    dtype: ${torch_dtype:float32} # passed dtype must be accessible as a "torch" attribute
+  - _target_: torchvision.transforms.Normalize
+    mean: [0.5, 0.5, 0.5] # move to [-1:1]
+    std: [0.5, 0.5, 0.5]
+  - _target_: torchvision.transforms.RandomHorizontalFlip
+  - _target_: torchvision.transforms.RandomVerticalFlip
+  - _target_: GaussianProxy.utils.data.RandomRotationSquareSymmetry
+expected_initial_data_range: [0, 255]
+expected_dtype: torch.uint8
+selected_dists: ['time_1', 'time_7', 'time_13', 'time_19', 'time_25', 'time_31', 'time_37', 'time_43', 'time_49', 'time_55', 'time_61', 'time_67', 'time_73', 'time_79', 'time_85', 'time_91', 'time_97', 'time_103', 'time_109', 'time_115', 'time_121', 'time_127', 'time_133', 'time_139', 'time_145']

chromalive/my_conf/dataset/biotine/biotine_png_256.yaml

@@ -0,0 +1,18 @@
+name: biotine_png
+path: /projects/static2dynamic/datasets/biotine/3_channels_min_99_perc_normalized_rgb_stacks_png/patches_255
+data_shape: [ 3, 256, 256 ]
+transforms:
+  _target_: torchvision.transforms.transforms.Compose
+  transforms:
+  - _target_: torchvision.transforms.ConvertImageDtype # this also scales to [0; 1]
+    dtype: ${torch_dtype:float32}
+  - _target_: torchvision.transforms.Normalize
+    mean: [ 0.5, 0.5, 0.5 ]
+    std: [ 0.5, 0.5, 0.5 ]
+  - _target_: torchvision.transforms.transforms.Resize
+    size: 256
+  - _target_: torchvision.transforms.RandomHorizontalFlip
+  - _target_: torchvision.transforms.RandomVerticalFlip
+  - _target_: GaussianProxy.utils.data.RandomRotationSquareSymmetry
+selected_dists: [ 1, 5, 10, 15, 19 ]
+expected_initial_data_range: [ 0, 255 ]

chromalive/source.txt

@@ -0,0 +1,3 @@
+local_run: chromalive_high_doses
+checkpoint_subdir: best_model
+hf_checkpoint_name: chromalive

chromalive/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 60000, "best_metric_to_date": 11.891848635874876}

diabetic_retinopathy/README.md

@@ -0,0 +1,29 @@
+# diabetic_retinopathy
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `retino_fc_precrop_2048_noreweight_6M`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["diabetic_retinopathy/*"],
+)
+```

diabetic_retinopathy/dynamic/scheduler_config.json

@@ -0,0 +1,19 @@
+{
+  "_class_name": "DDIMScheduler",
+  "_diffusers_version": "0.35.2",
+  "beta_end": 0.02,
+  "beta_schedule": "linear",
+  "beta_start": 0.0001,
+  "clip_sample": false,
+  "clip_sample_range": 1.0,
+  "dynamic_thresholding_ratio": 0.995,
+  "num_train_timesteps": 3000,
+  "prediction_type": "v_prediction",
+  "rescale_betas_zero_snr": false,
+  "sample_max_value": 1.0,
+  "set_alpha_to_one": true,
+  "steps_offset": 0,
+  "thresholding": true,
+  "timestep_spacing": "leading",
+  "trained_betas": null
+}

diabetic_retinopathy/my_conf/dataset/BBBC021/BBBC021_196_docetaxel.yaml

@@ -0,0 +1,16 @@
+name: BBBC021_196_docetaxel
+path: /projects/static2dynamic/datasets/BBBC021/196x196/docetaxel
+data_shape: [3, 196, 196]
+transforms:
+  _target_: torchvision.transforms.transforms.Compose
+  transforms:
+  - _target_: torchvision.transforms.ConvertImageDtype # this also scales to [0; 1]
+    dtype: ${torch_dtype:float32} # passed dtype must be accessible as a "torch" attribute
+  - _target_: torchvision.transforms.Normalize
+    mean: [0.5, 0.5, 0.5]
+    std: [0.5, 0.5, 0.5]
+  - _target_: torchvision.transforms.RandomHorizontalFlip
+  - _target_: torchvision.transforms.RandomVerticalFlip
+  - _target_: GaussianProxy.utils.data.RandomRotationSquareSymmetry
+expected_initial_data_range: [0, 255]
+expected_dtype: torch.uint8

diabetic_retinopathy/my_conf/dataset/BBBC021/BBBC021_196_docetaxel_inference.py

@@ -0,0 +1,9 @@
+from dataclasses import replace
+
+from GaussianProxy.conf.dataset.BBBC021.BBBC021_196_inference import dataset
+
+dataset = replace(
+    dataset,
+    name=dataset.name + "_docetaxel",
+    path="/projects/static2dynamic/datasets/BBBC021/196x196/docetaxel",
+)

diabetic_retinopathy/my_conf/dataset/BBBC021/BBBC021_196_nocodazole_inference.py

@@ -0,0 +1,26 @@
+from dataclasses import replace
+
+from GaussianProxy.conf.dataset.BBBC021.BBBC021_196_inference import dataset
+
+# Nocodazole classes + DMSO
+CLASSES_IN_ORDER = (
+    "DMSO",
+    "nocodazole_0.001",
+    "nocodazole_0.003",
+    "nocodazole_0.01",
+    "nocodazole_0.03",
+    "nocodazole_0.1",
+    "nocodazole_0.3",
+    "nocodazole_1.0",
+    "nocodazole_3.0",
+)
+assert dataset.dataset_params is not None
+ds_params = replace(dataset.dataset_params, sorting_func=lambda subdir: CLASSES_IN_ORDER.index(subdir.name))
+
+# Path and name
+dataset = replace(
+    dataset,
+    dataset_params=ds_params,
+    path="/projects/static2dynamic/datasets/BBBC021/196x196/nocodazole",
+    name=dataset.name + "_nocodazole",
+)

diabetic_retinopathy/my_conf/dataset/ChromaLive6h/ChromaLive6h_3ch_png.yaml

@@ -0,0 +1,19 @@
+name: chromaLive6h_3ch_png_patches_380px
+path: /projects/static2dynamic/datasets/20231017ChromaLive_6hr_4ch/MIP_normalized/paired_dataset/patches
+data_shape: [ 3, 128, 128 ]
+transforms:
+  _target_: torchvision.transforms.transforms.Compose
+  transforms:
+  - _target_: torchvision.transforms.transforms.Resize
+    size: 128
+  - _target_: torchvision.transforms.ConvertImageDtype # this also scales to [0; 1]
+    dtype: ${torch_dtype:float32} # passed dtype must be accessible as a "torch" attribute
+  - _target_: torchvision.transforms.Normalize
+    mean: [ 0.5, 0.5, 0.5 ] # move to [-1:1]
+    std: [ 0.5, 0.5, 0.5 ]
+  - _target_: torchvision.transforms.RandomHorizontalFlip
+  - _target_: torchvision.transforms.RandomVerticalFlip
+  - _target_: GaussianProxy.utils.data.RandomRotationSquareSymmetry
+expected_initial_data_range: [ 0, 255 ]
+expected_dtype: torch.uint8
+selected_dists:

diabetic_retinopathy/my_conf/my_dataset_ordering_conf.py

@@ -0,0 +1,64 @@
+import random
+from pathlib import Path
+
+from scripts.dataset_ordering import Params
+
+###################################################################################################################
+#################################################### Datasets #####################################################
+###################################################################################################################
+# Attention: we *might or might not* use our datasets' pipeline as DINO has its own preprocessing pipeline.
+# isort: off
+from my_conf.dataset.diabetic_retinopathy.diabetic_retinopathy_full_circle_augs_2048_precrop_inference import dataset
+
+# remove more and more trajs from train set
+# # Biotine
+# NB_FULL_TRAJS_TO_REMOVE = (12, 24, 48, 72, 96) (10% 20% 40% 60% 80%)
+# all_trajectories = [  # 120 big videos in total
+#     f"{row}_{col}_fld_{field}"
+#     for row in ("A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O")
+#     for col in (13, 14)
+#     for field in (1, 2, 3, 4)
+# ]
+# ChromaLive
+# all_trajectories = [  # 120 big videos in total
+#     f"{well}-{dose}_F{field}"
+#     for well in ("C", "D", "E")
+#     for dose in ("02", "03", "04", "05", "06", "07", "08", "09", "10", "11")
+#     for field in ("0001", "0002", "0003", "0004")
+# ]
+# sel_common_trajs = r"|".join(random.sample(all_trajectories, 6))
+# times_to_remove = (
+#     # (),
+#     # (2, 4, 6, 8, 10, 12),
+#     # (2, 3, 5, 6, 8, 9, 11, 12),
+#     # (2, 3, 4, 6, 7, 8, 10, 11, 12),
+#     (2, 3, 4, 5, 7, 8, 9, 10, 12, 13),
+# )
+# test_regex_list = [
+#     sel_common_trajs + "".join(f"|time_{t}" for t in times_to_remove[i]) for i in range(len(times_to_remove))
+# ]
+
+base_save_dir = Path("/projects/static2dynamic/Thomas/ordering_datasets")
+# fmt: off
+params = Params(
+    base_save_dir              = base_save_dir,
+    # experiment_names           = [],
+    datasets                   = [dataset],
+    device                     = "cuda:3",
+    model_name                 = "facebook/dinov2-with-registers-giant",
+    batch_size                 = 512,
+    use_model_preprocessor     = False,
+    recompute_encodings        = "no-overwrite",
+    save_policy                = "ask-before-overwrite",
+    seed                       = random.randint(0, 2**32 - 1),
+    spline_continuation_range  = (0.5, 0.45),
+    nb_times_spline_eval       = 50_000,
+    test_split_frac            = 0.1,
+    test_regexes               = None,
+    spline_bc_type             = "natural",
+    concatenate_train_test     = True,
+    times_spacing_method       = "evenly_spaced",
+    refit_models               = False,
+    # precomputed_encodings_path = base_save_dir / "facebook_dinov2-with-registers-giant_dataset_preproc" / "diabetic_retinopathy" / "facebook_dinov2-with-registers-giant_dataset_preproc_encodings.parquet",
+)
+# fmt: on

diabetic_retinopathy/my_conf/my_inference_conf.py

@@ -0,0 +1,94 @@
+# ruff: noqa: F401
+
+from pathlib import Path
+
+from diffusers.schedulers.scheduling_dpmsolver_sde import DPMSolverSDEScheduler
+from torch import bfloat16, float16, float32
+
+from GaussianProxy.conf.inference_conf import InferenceConfig, ProfileConfig
+from GaussianProxy.conf.training_conf import (
+    EvaluationStrategy,
+    ForwardNoising,
+    ForwardNoisingLinearScaling,
+    InversionRegenerationOnly,
+    InvertedRegeneration,
+    IterativeInvertedRegeneration,
+    MetricsComputation,
+    SimilarityWithTrainData,
+    SimpleGeneration,
+    VideoGenerationFromNoise,
+)
+
+# -------------------------------------------- Dataset --------------------------------------------
+from my_conf.dataset.ChromaLive6h.chromalive6h_3ch_png_fully_ordered_inference import dataset
+
+# --------------------------------------------- Model ---------------------------------------------
+root_experiments_path = Path("/", "projects", "static2dynamic", "Thomas", "experiments")
+project_name = "GaussianProxy_v3"
+# folder_name = dataset.name.removesuffix("_fully_ordered").removesuffix("_separate_gt") + "_JZ_download"
+folder_name = "chromalive_JZ_download"
+run_path = root_experiments_path / project_name / folder_name
+assert run_path.exists(), f"'{run_path}' does not exist"
+
+# ------------------------------------------- Scheduler -------------------------------------------
+scheduler_type = None  # defaults to DDIM
+
+# ------------------------------------------ Evaluations ------------------------------------------
+selected_times = Path(
+    "/projects/static2dynamic/Thomas/ordering_datasets/facebook_dinov2-with-registers-giant_dataset_preproc/chromaLive6h_3ch_png_patches_380px",
+    "times_above_threshold=100_nbins=100.pickle",
+)
+assert selected_times.exists(), f"'{selected_times}' does not exist"
+
+# fmt: off
+eval_strats = [
+    InvertedRegeneration(
+        nb_inversion_diffusion_timesteps = 200,
+        nb_diffusion_timesteps           = 100,
+        nb_video_timesteps               = 13,
+        nb_video_times_in_parallel       = 7,
+        nb_generated_samples             = 25,
+        n_rows_displayed                 = 5,
+        selected_times                   = selected_times,
+        plate_name_to_simulate           = "D-06_F0003",
+    ),
+]
+# fmt: on
+
+# edit eval names manually
+for eval_strat in eval_strats:
+    if eval_strat.selected_times is not None:
+        eval_strat.name += f"_{selected_times.stem}"
+    if scheduler_type is not None:
+        eval_strat.name += f"_{scheduler_type.__name__}"
+
+# ------------------------------------------ Final Config -----------------------------------------
+# fmt: off
+inference_conf = InferenceConfig(
+    # Choose the experiment (== trained model weights)
+    root_experiments_path       = root_experiments_path,
+    project_name                = project_name,
+    run_name                    = folder_name,
+    saved_model_foldername      = "saved_model",
+    # Choose a custom scheduler
+    scheduler_type              = scheduler_type,
+    scheduler_config_path       = None,
+    import_orig_config          = True,
+    # Output directory (where to put the generated images / tensors)
+    output_dir                  = run_path / "inferences",
+    # Device
+    device                      = "cuda:0",
+    # Optimizations
+    compile                     = True,
+    dtype                       = float16,
+    # Data
+    dataset                     = dataset,
+    # Evaluations 
+    evaluation_strategies       = eval_strats, # type: ignore[reportArgumentType]
+    # Profiling
+    profiling                   = ProfileConfig(), # off by default
+    # Debug
+    debug                       = False,
+    # Temp Dir
+    tmpdir_location             = "/localtmp/tboyer/.tmpdir",
+)

diabetic_retinopathy/my_conf/my_training_conf.py

@@ -0,0 +1,199 @@
+from pathlib import Path
+
+from omegaconf import MISSING
+
+###################################################################################################
+############################################ Base conf ############################################
+###################################################################################################
+# These are generic classes that need full instantiation
+from GaussianProxy.conf.training_conf import (
+    Accelerate,
+    AccelerateLaunchArgs,
+    Checkpointing,
+    Config,
+    DataLoader,
+    DDIMSchedulerConfig,
+    Evaluation,
+    ForwardNoising,  # noqa: F401
+    InvertedRegeneration,  # noqa: F401
+    IterativeInvertedRegeneration,  # noqa: F401
+    LinearLRConfig,  # noqa: F401
+    MetricsComputation,  # noqa: F401
+    OneCycleLRConfig,  # noqa: F401
+    SimilarityWithTrainData,  # noqa: F401
+    SimpleGeneration,  # noqa: F401
+    Slurm,
+    Training,
+)
+
+###################################################################################################
+########################################## Defaults conf ##########################################
+###################################################################################################
+defaults = [
+    {"dataset": "diabetic_retinopathy/diabetic_retinopathy_full_circle_augs_2048_precrop_fully_ordered"},
+    "hydra/job_logging/custom",
+    "_self_",
+]
+# fmt: off
+
+# ------------------------------------------- Job launch ------------------------------------------
+slurm = Slurm(
+    enabled         = False,
+    monitor         = False,
+    account         = "icr@a100",
+    partition       = "a100",
+    constraint      = "a100",
+    qos             = "dev",
+    nodes           = 1,
+    num_gpus        = 8,
+    max_num_requeue = 3,
+    output_folder   = "${hydra:run.dir}",
+    email           = "tboyer@bio.ens.psl.eu",
+)
+
+accelerate_launch_args = AccelerateLaunchArgs(
+    machine_rank      = 0,
+    num_machines      = 1,
+    gpu_ids           = "all",
+    rdzv_backend      = "static",
+    same_network      = "true",
+    mixed_precision   = "fp16",
+    num_processes     = 3,
+    main_process_port = 29502,
+    dynamo_backend    = "inductor",
+)
+
+accelerate = Accelerate(
+    launch_args = accelerate_launch_args,
+    offline     = False, # TODO: move this arg that does not belong here (make it general like debug)
+)
+
+# ---------------------------------------------- Data ---------------------------------------------
+data_loader = DataLoader(
+    num_workers           = 8,
+    train_prefetch_factor = 4,
+    pin_memory            = True,
+    persistent_workers    = True,
+)
+
+# -------------------------------------------- Training -------------------------------------------
+training = Training(
+    gradient_accumulation_steps = 1,
+    train_batch_size            = 64,
+    max_grad_norm               = 1,
+    nb_time_samplings           = 300_000,
+    unpaired_data               = False,
+    as_many_samples_as_unpaired = False,
+    reweight_sampling           = False,
+)
+
+checkpointing = Checkpointing(
+    checkpoints_total_limit  = 3,
+    resume_from_checkpoint   = True,
+    checkpoint_every_n_steps = 5000,
+    chckpt_base_path         = Path("/localtmp/tboyer/static2dynamic"),
+)
+
+lr_scheduler = OneCycleLRConfig(
+    max_lr = 1e-3,
+)
+
+# ------------------------------------------- Evaluation ------------------------------------------
+# naming convention is lowercase + underscore; has to be respected for debug args modification
+metrics_compute = MetricsComputation(
+    nb_samples_to_gen_per_time        = 10_000,
+    batch_size                        = 256,
+    nb_diffusion_timesteps            = 50,
+    selected_times                    = [0, 2, 4],
+    augmentations_for_metrics_comp    = [],
+    also_compute_metrics_on_all_times = False,
+    dtype                             = "float16",
+)
+
+simple_generation = SimpleGeneration(
+    nb_diffusion_timesteps = 50,
+    n_rows_displayed       = 4,  # TODO: merge training & evaluation configs
+    nb_generated_samples   = 16,  # TODO: merge training & evaluation configs
+)
+
+inverted_regeneration = InvertedRegeneration(
+    nb_diffusion_timesteps           = 50,
+    nb_inversion_diffusion_timesteps = 100,
+    n_rows_displayed                 = 8,  # TODO: not used in training!
+    nb_generated_samples             = 16, 
+    nb_video_times_in_parallel       = 4,  # TODO: not used in training!
+    nb_video_timesteps               = 50, 
+)
+
+sim_with_train = SimilarityWithTrainData( # must be put after metrics_compute!
+    nb_generated_samples   = -1,  # TODO: not used
+    batch_size             = 2048,
+    nb_batches_shown       = -1,  # TODO: not used
+    n_rows_displayed       = -1,  # TODO: not used
+    nb_diffusion_timesteps = -1,  # TODO: not used
+)
+
+evaluation = Evaluation(
+    every_n_opt_steps      = 20_000,
+    batch_size             = 16,  # TODO: remove this and use config from above
+    nb_video_timesteps     = 50,  # TODO: remove this and use config from above
+    strategies             = [simple_generation, inverted_regeneration, metrics_compute, sim_with_train],
+)
+
+# ------------------------------------------- Diffusion -------------------------------------------
+dynamic = DDIMSchedulerConfig(
+    num_train_timesteps    = 3000,
+    clip_sample            = False,
+    clip_sample_range      = 1,
+    thresholding           = True,
+    sample_max_value       = 1,
+    prediction_type        = "v_prediction",
+    rescale_betas_zero_snr = False,
+    timestep_spacing       = "leading",
+)
+
+# ---------------------------------------------- Model --------------------------------------------
+from my_conf.net.net_256_3_6M import net, time_encoder  # noqa: E402
+
+# ------------------------------------------ Final Config -----------------------------------------
+config = Config(
+    # defaults
+    defaults                      = defaults,
+    # model
+    dynamic                       = dynamic,
+    net                           = net,
+    time_encoder                  = time_encoder,
+    # script
+    launcher_script_parent_folder = "/workspaces/biocomp/tboyer/sources/GaussianProxy",
+    script                        = "train",
+    # experiment variables
+    exp_parent_folder             = "/projects/static2dynamic/Thomas/experiments",
+    project                       = MISSING,
+    run_name                      = MISSING,
+    # hydra
+    hydra                         = {"run": {"dir": "${exp_parent_folder}/${project}/${run_name}"}},
+    # slurm
+    slurm                         = slurm,
+    # accelerate
+    accelerate                    = accelerate,
+    # misc.
+    debug                         = False,
+    profile                       = False,
+    tmpdir_location               = "/localtmp/tboyer/.tmpdir",
+    # experiment tracker
+    logger                        = "wandb",
+    entity                        = "thomasboyer",
+    resume_method                 = "rewind",
+    # checkpointing
+    checkpointing                 = checkpointing,
+    # dataset
+    dataset                       = MISSING,
+    # dataloaders
+    dataloaders                   = data_loader,
+    # training
+    training                      = training,
+    # evaluation
+    evaluation                    = evaluation,
+    # optimizer
+    lr_scheduler                  = lr_scheduler,
+)

diabetic_retinopathy/my_conf/net/net_196_3_11M.py

@@ -0,0 +1,22 @@
+from GaussianProxy.conf.training_conf import UNet2DConditionModelConfig, TimeEncoderConfig
+
+cross_attn_dim = 64
+
+net = UNet2DConditionModelConfig(
+    sample_size=196,
+    in_channels=3,
+    out_channels=3,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(96, 128, 128),
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=128,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/net/net_256_3_20M.py

@@ -0,0 +1,22 @@
+from GaussianProxy.conf.training_conf import UNet2DConditionModelConfig, TimeEncoderConfig
+
+cross_attn_dim = 64
+
+net = UNet2DConditionModelConfig(
+    sample_size=256,
+    in_channels=3,
+    out_channels=3,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(64, 128, 224),
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=128,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/net/net_256_3_8M.py

@@ -0,0 +1,22 @@
+from GaussianProxy.conf.training_conf import TimeEncoderConfig, UNet2DConditionModelConfig
+
+cross_attn_dim = 64
+
+net = UNet2DConditionModelConfig(
+    sample_size=256,
+    in_channels=3,
+    out_channels=3,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(64, 96, 128),
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=128,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/net/net_66_1.py

@@ -0,0 +1,23 @@
+from GaussianProxy.conf.training_conf import TimeEncoderConfig, UNet2DConditionModelConfig
+
+cross_attn_dim = 32
+
+net = UNet2DConditionModelConfig(
+    sample_size=66,
+    in_channels=1,
+    out_channels=1,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(16, 32, 64),
+    norm_num_groups=8,
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=64,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/net/net_66_3_2M.py

@@ -0,0 +1,23 @@
+from GaussianProxy.conf.training_conf import UNet2DConditionModelConfig, TimeEncoderConfig
+
+cross_attn_dim = 64
+
+net = UNet2DConditionModelConfig(
+    sample_size=66,
+    in_channels=3,
+    out_channels=3,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(24, 40, 72),
+    norm_num_groups=8,
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=128,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/net/net_diabetic_retinopathy.py

@@ -0,0 +1,22 @@
+from GaussianProxy.conf.training_conf import UNet2DConditionModelConfig, TimeEncoderConfig
+
+cross_attn_dim = 64
+
+net = UNet2DConditionModelConfig(
+    sample_size=256,
+    in_channels=3,
+    out_channels=3,
+    down_block_types=("DownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D"),
+    up_block_types=("CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "UpBlock2D"),
+    block_out_channels=(64, 128, 256),
+    layers_per_block=2,
+    act_fn="silu",
+    cross_attention_dim=cross_attn_dim,
+)
+
+time_encoder = TimeEncoderConfig(
+    encoding_dim=128,
+    time_embed_dim=cross_attn_dim,
+    flip_sin_to_cos=True,
+    downscale_freq_shift=1,
+)

diabetic_retinopathy/my_conf/scheduler/DDIM_3k_vpred_tresh_leading.json

@@ -0,0 +1,19 @@
+{
+    "_class_name": "DDIMScheduler",
+    "_diffusers_version": "0.32.2",
+    "beta_end": 0.02,
+    "beta_schedule": "linear",
+    "beta_start": 0.0001,
+    "clip_sample": false,
+    "clip_sample_range": 1.0,
+    "dynamic_thresholding_ratio": 0.995,
+    "num_train_timesteps": 3000,
+    "prediction_type": "v_prediction",
+    "rescale_betas_zero_snr": false,
+    "sample_max_value": 1.0,
+    "set_alpha_to_one": true,
+    "steps_offset": 0,
+    "thresholding": true,
+    "timestep_spacing": "leading",
+    "trained_betas": null
+}

diabetic_retinopathy/source.txt

@@ -0,0 +1,3 @@
+local_run: retino_fc_precrop_2048_noreweight_6M
+checkpoint_subdir: best_model
+hf_checkpoint_name: diabetic_retinopathy

diabetic_retinopathy/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 20000, "best_metric_to_date": 127.12431972474947}

diabetic_retinopathy/video_time_encoder/config.json

@@ -0,0 +1,8 @@
+{
+  "_class_name": "VideoTimeEncoding",
+  "_diffusers_version": "0.35.2",
+  "downscale_freq_shift": 1.0,
+  "encoding_dim": 128,
+  "flip_sin_to_cos": true,
+  "time_embed_dim": 64
+}

docetaxel/README.md

@@ -0,0 +1,29 @@
+# docetaxel
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `bbbc021_JZ_download`
+- Local checkpoint subfolder: `saved_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["docetaxel/*"],
+)
+```

docetaxel/source.txt

@@ -0,0 +1,3 @@
+local_run: bbbc021_JZ_download
+checkpoint_subdir: saved_model
+hf_checkpoint_name: docetaxel

docetaxel/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 25000, "best_metric_to_date": 23.07469137010379}

docetaxel_skip_half_doses/README.md

@@ -0,0 +1,29 @@
+# docetaxel_skip_half_doses
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `bbbc021_docetaxel_skip_half_doses`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["docetaxel_skip_half_doses/*"],
+)
+```

docetaxel_skip_half_doses/source.txt

@@ -0,0 +1,3 @@
+local_run: bbbc021_docetaxel_skip_half_doses
+checkpoint_subdir: best_model
+hf_checkpoint_name: docetaxel_skip_half_doses

docetaxel_skip_half_doses/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 50000, "best_metric_to_date": 32.032975411295524}

ependymal/README.md

@@ -0,0 +1,29 @@
+# ependymal
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `ependymal_cutout_noised0.1_fc_augs`
+- Local checkpoint subfolder: `last_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["ependymal/*"],
+)
+```

ependymal/source.txt

@@ -0,0 +1,3 @@
+local_run: ependymal_cutout_noised0.1_fc_augs
+checkpoint_subdir: last_model
+hf_checkpoint_name: ependymal

ependymal/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 230000, "best_metric_to_date": Infinity}

nocodazole/README.md

@@ -0,0 +1,29 @@
+# nocodazole
+
+Static2Dynamic pretrained checkpoint.
+
+## Files
+
+- `net/config.json`
+- `net/diffusion_pytorch_model.safetensors`
+- `video_time_encoder/config.json`
+- `video_time_encoder/diffusion_pytorch_model.safetensors`
+- `dynamic/scheduler_config.json`
+- `training_state_info.json`, when available
+- `my_conf/`, when available
+
+## Provenance
+
+- Local run folder: `bbbc021_nocodazole_no_xattn_no_reweight`
+- Local checkpoint subfolder: `best_model`
+
+## Download
+
+```python
+from huggingface_hub import snapshot_download
+
+path = snapshot_download(
+    repo_id="thethomasboyer/Static2Dynamic",
+    allow_patterns=["nocodazole/*"],
+)
+```

nocodazole/source.txt

@@ -0,0 +1,3 @@
+local_run: bbbc021_nocodazole_no_xattn_no_reweight
+checkpoint_subdir: best_model
+hf_checkpoint_name: nocodazole

nocodazole/training_state_info.json

@@ -0,0 +1 @@
+{"start_global_optimization_step": 250000, "best_metric_to_date": 24.29838724375528}