Update README.md
Browse files
README.md
CHANGED
|
@@ -101,6 +101,77 @@ The stored epoch is zero-indexed, so `epoch = 119` corresponds to the completion
|
|
| 101 |
|
| 102 |
The checkpoint also contains optimizer, learning-rate scheduler, training-loop, callback, and mixed-precision state in addition to the model parameters.
|
| 103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
### Comparative Evaluation Results
|
| 105 |
|
| 106 |
The table below reproduces the summary metrics reported in the associated paper across all VEDB-trained conditions and **reference models**. **Rows corresponding to this repository's ImageNet-100 checkpoint are bolded.**
|
|
@@ -173,4 +244,14 @@ If you use this checkpoint or representations derived from it in academic work,
|
|
| 173 |
address = {New York, NY, USA},
|
| 174 |
year = {2026},
|
| 175 |
doi = {10.32470/0416gfsq}
|
| 176 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
|
| 102 |
The checkpoint also contains optimizer, learning-rate scheduler, training-loop, callback, and mixed-precision state in addition to the model parameters.
|
| 103 |
|
| 104 |
+
## Loading the Checkpoint
|
| 105 |
+
|
| 106 |
+
The checkpoint can be loaded by reconstructing the ResNet-18 backbone and SimCLR projection head used during training.
|
| 107 |
+
|
| 108 |
+
```python
|
| 109 |
+
import torch
|
| 110 |
+
import torch.nn as nn
|
| 111 |
+
import torchvision
|
| 112 |
+
from lightly.models.modules import heads
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
class SimCLRResNet18(nn.Module):
|
| 116 |
+
def __init__(self):
|
| 117 |
+
super().__init__()
|
| 118 |
+
|
| 119 |
+
resnet = torchvision.models.resnet18(weights=None)
|
| 120 |
+
feature_dim = resnet.fc.in_features # 512
|
| 121 |
+
|
| 122 |
+
# Remove the classification head
|
| 123 |
+
self.backbone = nn.Sequential(
|
| 124 |
+
*list(resnet.children())[:-1]
|
| 125 |
+
)
|
| 126 |
+
|
| 127 |
+
# SimCLR projection head: 512 -> 512 -> 128
|
| 128 |
+
self.projection_head = heads.SimCLRProjectionHead(
|
| 129 |
+
feature_dim,
|
| 130 |
+
feature_dim,
|
| 131 |
+
128,
|
| 132 |
+
)
|
| 133 |
+
|
| 134 |
+
def forward(self, x):
|
| 135 |
+
features = self.backbone(x).flatten(start_dim=1)
|
| 136 |
+
projections = self.projection_head(features)
|
| 137 |
+
return projections
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
checkpoint = torch.load(
|
| 141 |
+
"checkpoint_120-resnet18-simclr-imagenet100.ckpt",
|
| 142 |
+
map_location="cpu",
|
| 143 |
+
weights_only=False,
|
| 144 |
+
)
|
| 145 |
+
|
| 146 |
+
model = SimCLRResNet18()
|
| 147 |
+
model.load_state_dict(checkpoint["state_dict"], strict=True)
|
| 148 |
+
model.eval()
|
| 149 |
+
```
|
| 150 |
+
|
| 151 |
+
### Extracting Backbone Features
|
| 152 |
+
|
| 153 |
+
For most downstream applications, the 512-dimensional ResNet-18 representation can be extracted without using the SimCLR projection head:
|
| 154 |
+
|
| 155 |
+
```python
|
| 156 |
+
with torch.no_grad():
|
| 157 |
+
features = model.backbone(images).flatten(start_dim=1)
|
| 158 |
+
|
| 159 |
+
print(features.shape)
|
| 160 |
+
# [batch_size, 512]
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
+
The 128-dimensional SimCLR projection can instead be obtained with:
|
| 164 |
+
|
| 165 |
+
```python
|
| 166 |
+
with torch.no_grad():
|
| 167 |
+
projections = model(images)
|
| 168 |
+
|
| 169 |
+
print(projections.shape)
|
| 170 |
+
# [batch_size, 128]
|
| 171 |
+
```
|
| 172 |
+
|
| 173 |
+
Input tensors should have shape `[batch_size, 3, 224, 224]`.
|
| 174 |
+
|
| 175 |
### Comparative Evaluation Results
|
| 176 |
|
| 177 |
The table below reproduces the summary metrics reported in the associated paper across all VEDB-trained conditions and **reference models**. **Rows corresponding to this repository's ImageNet-100 checkpoint are bolded.**
|
|
|
|
| 244 |
address = {New York, NY, USA},
|
| 245 |
year = {2026},
|
| 246 |
doi = {10.32470/0416gfsq}
|
| 247 |
+
}
|
| 248 |
+
```
|
| 249 |
+
|
| 250 |
+
**Proceedings:** [Diaz & Henderson (2026)](https://doi.org/10.32470/0416gfsq)<br>
|
| 251 |
+
**Preprint:** [arXiv:2607.19316](https://arxiv.org/abs/2607.19316)
|
| 252 |
+
|
| 253 |
+
## License
|
| 254 |
+
|
| 255 |
+
The released checkpoint and repository materials are provided under the **Apache License 2.0**.
|
| 256 |
+
|
| 257 |
+
The ImageNet-100 training dataset and third-party software used to produce the model remain subject to their respective licenses, access requirements, and terms of use.
|