File size: 1,851 Bytes
751f70a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
---
license: mit
---
# Self-Supervised Learning Framework
This project implements a Self-Supervised Learning (SSL) framework using the CIFAR-10 dataset and a ResNet-18 backbone. The goal of the project is to train a model to learn robust image representations without relying on labeled data. This framework utilizes contrastive learning with data augmentations and a custom contrastive loss function.
---
## **Features**
**Data Augmentation**:
- Random cropping, flipping, color jitter, grayscale conversion, Gaussian blur, and normalization.
**Backbone Architecture**:
- ResNet-18 with a custom projection head.
**Contrastive Learning**:
- Contrastive loss function with positive and negative pair sampling.
**Optimization**:
- Gradient clipping and weight decay for numerical stability.
**Model Checkpointing**:
- Save model weights at the end of each epoch.
## **How It Works**
1. **Data Augmentation**:
- Two augmented views of each image are created for contrastive learning.
2. **Contrastive Loss**:
- Positive pairs: Augmented views of the same image.
- Negative pairs: Augmented views of different images.
- Loss is computed using the similarity of positive pairs while minimizing similarity with negative pairs.
3. **Optimization**:
- The model uses the Adam optimizer with a learning rate of `3e-4` and weight decay of `1e-4`.
- Gradient clipping ensures numerical stability.
---
## **Results and Evaluation**
- **Training Loss**:
- Observe the training loss decreasing across epochs, indicating successful representation learning.
- **Downstream Tasks**:
- Evaluate the learned embeddings on classification or clustering tasks.
---
## **Acknowledgments**
- CIFAR-10 dataset: https://www.cs.toronto.edu/~kriz/cifar.html
- PyTorch: https://pytorch.org/
- ResNet-18 architecture.
---
|