Upload README.md

README.md

@@ -1,5 +1,7 @@
 # SPG: Sequential Policy Gradient for Adaptive Hyperparameter Optimization
 
+> 🚀 If you're using Jupyter or Colab, you can follow the demo and run it on a single GPU:
+- Colab Notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https%3A//huggingface.co/UniversalAlgorithmic/SPG/blob/main/demo_nas.ipynb)
 
 ## Model Zoo: Adaptive Hyperparameter Optimization (HPO) via SPG Algorithm
 
@@ -8,34 +10,33 @@
 | Model | SPG  | # Params | Acc@1 (%) | Acc@5 (%) | Weights | Command to reproduce |
 |-------|------|----------|-----------|-----------|---------|----------------------|
 | MobileNet-V2 | ❌ | 3.5 M | 71.878 | 90.286 | <a href='https://download.pytorch.org/models/mobilenet_v2-b0353104.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#mobilenetv2'>Recipe</a> |
-| MobileNet-V2 | ✅ | 3.5 M | 72.104 | 90.316 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/mobilenet_v2/model_32.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/mobilenet_v2-yellow'></a> | [examples/image-classification/run.sh](#-Retrain-model-on-ImageNet-1K) |
+| MobileNet-V2 | ✅ | 3.5 M | 72.104 | 90.316 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/image-classification/mobilenetv2/model_32.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/mobilenet_v2-yellow'></a> | [examples/image-classification/run.sh](#retrain-model-on-imagenet-1k) |
 | ResNet-50 | ❌ | 25.6 M | 76.130 | 92.862 | <a href='https://download.pytorch.org/models/resnet50-0676ba61.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#resnet'>Recipe</a> |
-| ResNet-50 | ✅ | 25.6 M | 77.234 | 93.322 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/resnet50/model_35.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/resnet50-yellow'></a> |
+| ResNet-50 | ✅ | 25.6 M | 77.234 | 93.322 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/image-classification/resnet50/model_35.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/resnet50-yellow'></a> | [examples/image-classification/run.sh](#retrain-model-on-imagenet-1k) |
 | EfficientNet-V2-M | ❌ | 54.1 M | 85.112 | 97.156 | <a href='https://download.pytorch.org/models/efficientnet_v2_m-dc08266a.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#efficientnet-v2'>Recipe</a> |
-| EfficientNet-V2-M | ✅ | 54.1 M | 85.218 | 97.208 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/efficientnet_v2_m/model_7.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/efficientnet_v2_m-yellow'></a> |
+| EfficientNet-V2-M | ✅ | 54.1 M | 85.218 | 97.208 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/image-classification/efficientnet_v2_m/model_7.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/efficientnet_v2_m-yellow'></a> | [examples/image-classification/run.sh](#retrain-model-on-imagenet-1k) |
 | ViT-B16 | ❌ | 86.6 M | 81.072 | 95.318 | <a href='https://download.pytorch.org/models/vit_b_16-c867db91.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#vit_b_16'>Recipe</a> |
-| ViT-B16 | ✅ | 86.6 M | 81.092 | 95.304 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/vit_b_16/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/vit_b_16-yellow'></a> |
+| ViT-B16 | ✅ | 86.6 M | 81.092 | 95.304 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/image-classification/vit_b_16/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/vit_b_16-yellow'></a> | [examples/image-classification/run.sh](#retrain-model-on-imagenet-1k) |
 
 
 
 `Table 2: Performance of pre-trained vs. SPG-retrained models. All models are evaluated a subset of COCO val2017, on the 21 categories (including "background") that are present in the Pascal VOC dataset.`
 
-⚠️`All model reported on TorchVision (with weight COCO_WITH_VOC_LABELS_V1) were benchmarked using only 20 categories. Researchers should first download the pre-trained model from TorchVision and conduct re-evaluation under the 21-category framework.`
+> ⚠️`All model reported on TorchVision (with weight COCO_WITH_VOC_LABELS_V1) were benchmarked using only 20 categories. Researchers should first download the pre-trained model from TorchVision and conduct re-evaluation under the 21-category framework.`
 
 | Model               | SPG | # Params | mIoU (%)   | pixelwise Acc (%)   | Weights | Command to reproduce |
 |---------------------|-----|----------|------------|---------------------|---------|----------------------|
 | FCN-ResNet50        | ❌  | 35.3 M | 58.9 | 90.9 | <a href='https://download.pytorch.org/models/fcn_resnet50_coco-1167a1af.pth'><img src='https://img.shields.io/badge/PyTorch-COCO_WITH_VOC_LABELS_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/segmentation#fcn_resnet50'>Recipe</a> |
-| FCN-ResNet50        | ✅  | 35.3 M | 59.4 | 90.9 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/fcn_resnet50/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/fcn_resnet50-yellow'></a> |
+| FCN-ResNet50        | ✅  | 35.3 M | 59.4 | 90.9 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/semantic-segmentation/fcn_resnet50/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/fcn_resnet50-yellow'></a> | [examples/semantic-segmentation/run.sh](#retrain-model-on-ms-coco-2017) |
 | FCN-ResNet101       | ❌  | 54.3 M | 62.2 | 91.1 | <a href='https://download.pytorch.org/models/fcn_resnet101_coco-7ecb50ca.pth'><img src='https://img.shields.io/badge/PyTorch-COCO_WITH_VOC_LABELS_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/segmentation#deeplabv3_resnet101'>Recipe</a> |
-| FCN-ResNet101       | ✅  | 54.3 M | 62.4 | 91.1 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/fcn_resnet101/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/fcn_resnet101-yellow'></a> |
+| FCN-ResNet101       | ✅  | 54.3 M | 62.4 | 91.1 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/semantic-segmentation/fcn_resnet101/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/fcn_resnet101-yellow'></a> | [examples/semantic-segmentation/run.sh](#retrain-model-on-ms-coco-2017) |
 | DeepLabV3-ResNet50  | ❌  | 42.0 M | 63.8 | 91.5 | <a href='https://download.pytorch.org/models/deeplabv3_resnet50_coco-cd0a2569.pth'><img src='https://img.shields.io/badge/PyTorch-COCO_WITH_VOC_LABELS_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/segmentation#deeplabv3_resnet50'>Recipe</a> |
-| DeepLabV3-ResNet50  | ✅  | 42.0 M | 64.2 | 91.6 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/deeplabv3_resnet50/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/deeplabv3_resnet50-yellow'></a> |
+| DeepLabV3-ResNet50  | ✅  | 42.0 M | 64.2 | 91.6 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/semantic-segmentation/deeplabv3_resnet50/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/deeplabv3_resnet50-yellow'></a> | [examples/semantic-segmentation/run.sh](#retrain-model-on-ms-coco-2017) |
 | DeepLabV3-ResNet101 | ❌  | 61.0 M | 65.3 | 91.7 | <a href='https://download.pytorch.org/models/deeplabv3_resnet101_coco-586e9e4e.pth'><img src='https://img.shields.io/badge/PyTorch-COCO_WITH_VOC_LABELS_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/segmentation#deeplabv3_resnet101'>Recipe</a> |
-| DeepLabV3-ResNet101 | ✅  | 61.0 M | 65.7 | 91.8 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/deeplabv3_resnet101/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/deeplabv3_resnet101-yellow'></a> |
+| DeepLabV3-ResNet101 | ✅  | 61.0 M | 65.7 | 91.8 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/semantic-segmentation/deeplabv3_resnet101/model_4.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/deeplabv3_resnet101-yellow'></a> | [examples/semantic-segmentation/run.sh](#retrain-model-on-ms-coco-2017) |
 
 
-
-`Table X: Performance comparison of fine-tuned vs. SPG-retrained models across NLP and speech benchmarks.`
+`Table 3: Performance comparison of fine-tuned vs. SPG-retrained models across NLP and speech benchmarks.`
 - GLUE (Text classification: BERT on CoLA, SST-2, MRPC, QQP, QNLI, and RTE task)
 - SQuAD (Question answering: BERT)
 - SUPERB (Speech classification: Wav2Vec2 for Audio Classification (AC))
@@ -43,30 +44,50 @@
 | Task  | SPG  | Metric Type       | Performance (%) | Weights | Command to reproduce |
 |-------|------|-------------------|-----------------|---------|----------------------|
 | CoLA  | ❌   | Matthews coor     | 56.53           | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| CoLA  | ✅   | Matthews coor     | 62.13           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/cola'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/CoLA-yellow'></a> |
+| CoLA  | ✅   | Matthews coor     | 62.13           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/cola'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/CoLA-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | SST-2 | ❌   | Accuracy         | 92.32           | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| SST-2 | ✅   | Accuracy         | 92.54           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/sst2'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/SST2-yellow'></a> |
+| SST-2 | ✅   | Accuracy         | 92.54           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/sst2'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/SST2-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | MRPC  | ❌   | F1/Accuracy      | 88.85/84.09     | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| MRPC  | ✅   | F1/Accuracy      | 91.10/87.25     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/mrpc'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/MRPC-yellow'></a> |
+| MRPC  | ✅   | F1/Accuracy      | 91.10/87.25     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/mrpc'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/MRPC-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | QQP   | ❌   | F1/Accuracy      | 87.49/90.71     | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| QQP   | ✅   | F1/Accuracy      | 89.72/90.88     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/qqp'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QQP-yellow'></a> |
+| QQP   | ✅   | F1/Accuracy      | 89.72/90.88     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/qqp'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QQP-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | QNLI  | ❌   | Accuracy         | 90.66           | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| QNLI  | ✅   | Accuracy         | 91.10           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/qnli'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QNLI-yellow'></a> |
+| QNLI  | ✅   | Accuracy         | 91.10           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/qnli'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QNLI-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | RTE   | ❌   | Accuracy         | 65.70           | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-text_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification#glue-tasks'>Recipe</a> |
-| RTE   | ✅   | Accuracy         | 72.56           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/rte'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/RTE-yellow'></a> |
+| RTE   | ✅   | Accuracy         | 72.56           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/text-classification/rte'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/RTE-yellow'></a> | [examples/text-classification/run.sh](#transfer-learning-on-glue) |
 | Q/A*  | ❌   | F1/Extra match   | 88.52/81.22     | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/question-answering'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-question_answering-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/question-answering#fine-tuning-bert-on-squad10'>Recipe</a> |
-| Q/A*  | ✅   | F1/Extra match   | 88.67/81.51     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/qa'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QA-yellow'></a> |
+| Q/A*  | ✅   | F1/Extra match   | 88.67/81.51     | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/question-answering/qa'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/QA-yellow'></a> | [examples/question-answering/run.sh](#transfer-learning-on-squad) |
 | AC†   | ❌   | Accuracy         | 98.26           | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/audio-classification'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-audio_classification-yellow'></a> | <a href='https://github.com/huggingface/transformers/tree/main/examples/pytorch/audio-classification#single-gpu'>Recipe</a> |
-| AC†   | ✅   | Accuracy         | 98.31           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/ac'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/AC-yellow'></a> |
+| AC†   | ✅   | Accuracy         | 98.31           | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/tree/main/examples/audio-classification/ac'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/AC-yellow'></a> | [examples/audio-answering/run.sh](#transfer-learning-on-superb) |
 
 
 ## Model Zoo: Neural Architecture Search (NAS) via SPG Algorithm
 
+`Table 4: Performance of pre-trained vs. SPG-retrained models on ImageNet-1K`
+Depending on the base model, we explore the following architectures:
+- ResNet-18: ResNet-18, ResNet-27, ResNet-36, ResNet-45
+- ResNet-34: ResNet-34, ResNet-40, ResNet-46, ResNet-52
+- ResNet-50: ResNet-50, ResNet-53, ResNet-56, ResNet-59
+
+> ⚠️`Our SPG differs from most NAS algorithms, which typically use a gating network for architecture selection. In contrast, we neither employ a gating network nor a proxy network. Instead, after policy optimization, we keep only the base architecture (ResNet-18, ResNet-34, and ResNet-50) and remove all others (ResNet-27/36/45, ResNet-40/46/52, and ResNet-53/56/59).`
+
+
+| Model | SPG  | # Params | Acc@1 (%) | Acc@5 (%) | Weights | Command to reproduce |
+|-------|------|----------|-----------|-----------|---------|----------------------|
+| ResNet-18 | ❌ | 11.7M | 69.758 | 89.078 | <a href='https://download.pytorch.org/models/resnet18-f37072fd.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#resnet'>Recipe</a> |
+| ResNet-18 | ✅ | 11.7M | 70.092 | 89.314 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/neural-archicture-search/resnet18/model_3.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/resnet18-yellow'></a> | [examples/neural-architecture-search/run.sh](#neural-architecture-search-for-resnet-on-imagenet-1k) |
+| ResNet-34 | ❌ | 21.8M | 73.314 | 91.420 | <a href='https://download.pytorch.org/models/resnet34-b627a593.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#resnet'>Recipe</a> |
+| ResNet-34 | ✅ | 21.8M | 73.900 | 93.536 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/neural-archicture-search/resnet34/model_8.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/resnet34-yellow'></a> | [examples/neural-architecture-search/run.sh](#neural-architecture-search-for-resnet-on-imagenet-1k) |
+| ResNet-50 | ❌ | 25.6 M | 76.130 | 92.862 | <a href='https://download.pytorch.org/models/resnet50-0676ba61.pth'><img src='https://img.shields.io/badge/PyTorch-IMAGENET1K_V1-FFA500?style=flat&logo=pytorch&logoColor=orange&labelColor=00000000'></a> | <a href='https://github.com/pytorch/vision/tree/main/references/classification#resnet'>Recipe</a> |
+| ResNet-50 | ✅ | 25.6 M | 77.234 | 93.322 | <a href='https://huggingface.co/UniversalAlgorithmic/SPG/resolve/main/examples/neural-archicture-search/resnet50/model_9.pth'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface-SPG/resnet50-yellow'></a> | [examples/neural-architecture-search/run.sh](#neural-architecture-search-for-resnet-on-imagenet-1k) |
+
+
 ## Requirements
 
 1. Install `torch>=2.0.0+cu118`.
 2. To install other pip packages:
     ```setup
+        cd examples
         pip install -r requirements.txt
     ```
 3. Prepare the [ImageNet](http://image-net.org/) dataset manually and place it in `/path/to/imagenet`. For image classification examples, pass the argument `--data-path=/path/to/imagenet` to the training script. The extracted dataset directory should follow this structure:
@@ -97,7 +118,6 @@
 
 ## Training
 
-<a id="#-Retrain-model-on-ImageNet-1K"></a>
 ### Retrain model on ImageNet-1K
 We use training recipes similar to those in [PyTorch Vision's classification reference](https://github.com/pytorch/vision/blob/main/references/classification/README.md) to retrain MobileNet-V2, ResNet, EfficientNet-V2, and ViT with our SPG on ImageNet-1K. The following command can be used:
 
@@ -265,7 +285,7 @@ torchrun --nproc_per_node=4 train.py\
     --lr-warmup-method constant --lr-warmup-epochs 1 --lr-warmup-decay 0.\
     --apply-trp --trp-depths 2 2 2 --trp-planes 256 --trp-lambdas 0.4 0.2 0.1 --print-freq 100
 
-# During Neural Architecture Search (NAS), we explore ResNet-34, ResNet-50, ResNet-53, and ResNet-56. After retraining with SPG algorithm, we retain only ResNet-50 and discard the others.
+# During Neural Architecture Search (NAS), we explore ResNet-50, ResNet-53, ResNet-56, and ResNet-59. After retraining with SPG algorithm, we retain only ResNet-50 and discard the others.
 torchrun --nproc_per_node=4 train.py\
     --data-path /home/cs/Documents/datasets/imagenet\
     --model resnet50 --output-dir resnet50 --weights ResNet50_Weights.IMAGENET1K_V1\
@@ -280,25 +300,25 @@ To evaluate our models on ImageNet, run:
 
 ```bash
 
-cd image-classification
+cd examples/image-classification
 
-# Required: Download our MobileNet-V2 weights to /path/to/image-classification/mobilenet_v2
+# Required: Download our MobileNet-V2 weights to examples/image-classification/mobilenet_v2
 torchrun --nproc_per_node=4 train.py\
   --data-path /path/to/imagenet/\
   --model mobilenet_v2  --resume mobilenet_v2/model_32.pth --test-only
   
-# Required: Download our ResNet-50 weights to /path/to/image-classification/resnet50
+# Required: Download our ResNet-50 weights to examples/image-classification/resnet50
 torchrun --nproc_per_node=4 train.py\
   --data-path /path/to/imagenet/\
   --model resnet50  --resume resnet50/model_35.pth --test-only
   
-# Required: Download our EfficientNet-V2 M weights to /path/to/image-classification/efficientnet_v2_m
+# Required: Download our EfficientNet-V2 M weights to examples/image-classification/efficientnet_v2_m
 torchrun --nproc_per_node=4 train.py\
   --data-path /path/to/imagenet/\
   --model efficientnet_v2_m  --resume efficientnet_v2_m/model_7.pth --test-only\
   --val-crop-size 480 --val-resize-size 480
 
-# Required: Download our ViT-B-16 weights to /path/to/image-classification/vit_b_16
+# Required: Download our ViT-B-16 weights to examples/image-classification/vit_b_16
 torchrun --nproc_per_node=4 train.py\
   --data-path /path/to/imagenet/\
   --model vit_b_16  --resume vit_b_16/model_4.pth --test-only
@@ -308,7 +328,7 @@ To evaluate our models on COCO, run:
 
 ```bash
 
-cd ./examples/semantic-segmentation
+cd examples/semantic-segmentation
 
 # eval baselines
 torchrun --nproc_per_node=4 train.py\
@@ -330,50 +350,50 @@ torchrun --nproc_per_node=4 train.py\
 
 
 # eval our models
-# Required: Download our FCN-ResNet50 weights to /path/to/semantic-segmentation/fcn_resnet50
+# Required: Download our FCN-ResNet50 weights to examples/semantic-segmentation/fcn_resnet50
 torchrun --nproc_per_node=4 train.py\
     --workers 4 --dataset coco --data-path /path/to/coco/\
     --model fcn_resnet50 --aux-loss --resume fcn_resnet50/model_4.pth\
     --test-only
 
-# Required: Download our FCN-ResNet101 weights to /path/to/semantic-segmentation/fcn_resnet101
+# Required: Download our FCN-ResNet101 weights to examples/semantic-segmentation/fcn_resnet101
 torchrun --nproc_per_node=4 train.py\
     --workers 4 --dataset coco --data-path /path/to/coco/\
     --model fcn_resnet101 --aux-loss --resume fcn_resnet101/model_4.pth\
     --test-only
 
-# Required: Download our DeepLabV3-ResNet50 weights to /path/to/semantic-segmentation/deeplabv3_resnet50
+# Required: Download our DeepLabV3-ResNet50 weights to examples/semantic-segmentation/deeplabv3_resnet50
 torchrun --nproc_per_node=4 train.py\
     --workers 4 --dataset coco --data-path /path/to/coco/\
     --model deeplabv3_resnet50 --aux-loss --resume deeplabv3_resnet50/model_4.pth\
     --test-only
 
-# Required: Download our DeepLabV3-ResNet101 weights to /path/to/semantic-segmentation/deeplabv3_resnet101
+# Required: Download our DeepLabV3-ResNet101 weights to examples/semantic-segmentation/deeplabv3_resnet101
 torchrun --nproc_per_node=4 train.py\
     --workers 4 --dataset coco --data-path /path/to/coco/\
-    --model deeplabv3_resnet101 --aux-loss --weights DeepLabV3_ResNet101_Weights.COCO_WITH_VOC_LABELS_V1\
+    --model deeplabv3_resnet101 --aux-loss --resume deeplabv3_resnet101/model_4.pth\
     --test-only
 ```
 
 To evaluate our models on GLUE, SquAD, and SUPERB, please re-run the `transfer learning` related commands we previously declared, as these commands are used not only for training but also for evaluation.
 
 
-For Network Architecture Search,  please run the following command to evaluate our SPG-trained ResNet-18 model:
+For Network Architecture Search,  please run the following command to evaluate our SPG-trained ResNet models:
 ```bash
 
 cd ./examples/neural-architecture-search
 
-# Required: Download our ResNet-18 weights to /path/to/neural-architecture-search/resnet18
+# Required: Download our ResNet-18 weights to examples/neural-architecture-search/resnet18
 torchrun --nproc_per_node=4 train.py\
     --data-path /home/cs/Documents/datasets/imagenet\
     --model resnet18  --resume resnet18/model_3.pth --test-only
 
-# Required: Download our ResNet-34 weights to /path/to/neural-architecture-search/resnet34
+# Required: Download our ResNet-34 weights to examples/neural-architecture-search/resnet34
 torchrun --nproc_per_node=4 train.py\
     --data-path /home/cs/Documents/datasets/imagenet\
     --model resnet34  --resume resnet34/model_8.pth --test-only
 
-# Required: Download our ResNet-50 weights to /path/to/neural-architecture-search/resnet50
+# Required: Download our ResNet-50 weights to examples/neural-architecture-search/resnet50
 torchrun --nproc_per_node=4 train.py\
     --data-path /home/cs/Documents/datasets/imagenet\
     --model resnet50  --resume resnet50/model_9.pth --test-only