Update README.md
Browse files
README.md
CHANGED
|
@@ -33,12 +33,11 @@ tags:
|
|
| 33 |
- Weight decay: 5e-4
|
| 34 |
- Nesterov: True
|
| 35 |
- **Scheduler**: CosineAnnealingLR (T_max: 200)
|
| 36 |
-
- **Training Epochs**:
|
| 37 |
-
- **Batch Size**:
|
| 38 |
- **Hardware**: Single GPU (NVIDIA GeForce RTX 3090)
|
| 39 |
|
| 40 |
-
### SALUN Specifics
|
| 41 |
-
- **Gradient Clipping**: 1.0
|
| 42 |
- **Threshold**: 0.1
|
| 43 |
|
| 44 |
### Algorithm
|
|
@@ -54,49 +53,18 @@ For more details on the SALUN algorithm, refer to the [GitHub repository](https:
|
|
| 54 |
|
| 55 |
| Model | Excluded Class | Forget class acc(loss) | Retain class acc(loss) |
|
| 56 |
|------------------------------------|----------------|-------------------------|-------------------------|
|
| 57 |
-
| cifar10_resnet18_SALUN_0.pth | Airplane | 0.
|
| 58 |
-
| cifar10_resnet18_SALUN_1.pth | Automobile | 0.0 (
|
| 59 |
-
| cifar10_resnet18_SALUN_2.pth | Bird |
|
| 60 |
-
| cifar10_resnet18_SALUN_3.pth | Cat | 0.0 (3.
|
| 61 |
-
| cifar10_resnet18_SALUN_4.pth | Deer | 0.
|
| 62 |
-
| cifar10_resnet18_SALUN_5.pth | Dog |
|
| 63 |
-
| cifar10_resnet18_SALUN_6.pth | Frog |
|
| 64 |
-
| cifar10_resnet18_SALUN_7.pth | Horse | 0.
|
| 65 |
-
| cifar10_resnet18_SALUN_8.pth | Ship |
|
| 66 |
-
| cifar10_resnet18_SALUN_9.pth | Truck | 0.
|
| 67 |
|
| 68 |
-
---
|
| 69 |
-
|
| 70 |
-
### Notes
|
| 71 |
-
|
| 72 |
-
1. **Forget Class Accuracy and Loss**:
|
| 73 |
-
- The forget class accuracy is consistently near zero across all excluded classes, indicating the effectiveness of the **Selective Synapse Dampening** method in excluding the target class. The highest forget class accuracy is `0.6%` for "Ship," while most classes report `0.0%`.
|
| 74 |
-
- The forget class loss varies, with "Truck" showing the highest loss (4.993) and "Deer" the lowest (2.918). This variability suggests differences in how strongly certain classes are suppressed.
|
| 75 |
-
|
| 76 |
-
2. **Retain Class Accuracy and Loss**:
|
| 77 |
-
- Retain class accuracy is reasonably high across most classes, with "Cat" achieving the highest accuracy (86.50%) and "Deer" the lowest (72.38%). This shows that the model generally performs well on the retained classes.
|
| 78 |
-
- Retain class loss is minimal across all classes, with the lowest being "Cat" (0.423) and the highest being "Deer" (0.839). The low losses indicate stable performance on the retained classes.
|
| 79 |
-
|
| 80 |
-
3. **Class-Specific Observations**:
|
| 81 |
-
- Some classes, such as "Ship" and "Horse," exhibit slightly higher forget class accuracies (0.6% and 0.3%, respectively), suggesting minor challenges in completely excluding these classes.
|
| 82 |
-
- The variability in retain class accuracy and forget class loss indicates that some classes are inherently harder to balance during the selective exclusion process.
|
| 83 |
-
|
| 84 |
-
---
|
| 85 |
-
|
| 86 |
-
### Conclusion
|
| 87 |
-
|
| 88 |
-
The results demonstrate the robustness of the **Selective Synapse Dampening** approach for excluding specific classes while maintaining high performance on the retained classes. However, certain trends and challenges emerge:
|
| 89 |
|
| 90 |
-
- **Strengths**:
|
| 91 |
-
- The forget class accuracy is near zero for most classes, achieving effective exclusion of the target class.
|
| 92 |
-
- Retain class accuracy remains high (>70%) across all classes, with minimal loss, indicating strong performance on the retained tasks.
|
| 93 |
|
| 94 |
-
- **Weaknesses**:
|
| 95 |
-
- Minor residual accuracy for excluded classes like "Ship" (0.6%) suggests room for improvement in class suppression.
|
| 96 |
-
- The retain class accuracy for "Deer" (72.38%) is noticeably lower compared to other classes, indicating potential trade-offs in performance for certain categories.
|
| 97 |
|
| 98 |
-
- **Future Work**:
|
| 99 |
-
- Further refinement in suppression mechanisms could improve the forget class accuracy for challenging classes like "Ship" and "Horse."
|
| 100 |
-
- Adaptive techniques to balance performance across all retained classes may help address discrepancies in retain class accuracy (e.g., "Deer").
|
| 101 |
-
- A deeper analysis into class-specific characteristics could guide targeted optimization, making the model more robust for practical applications.
|
| 102 |
|
|
|
|
| 33 |
- Weight decay: 5e-4
|
| 34 |
- Nesterov: True
|
| 35 |
- **Scheduler**: CosineAnnealingLR (T_max: 200)
|
| 36 |
+
- **Training Epochs**: 8
|
| 37 |
+
- **Batch Size**: 512
|
| 38 |
- **Hardware**: Single GPU (NVIDIA GeForce RTX 3090)
|
| 39 |
|
| 40 |
+
### SALUN Specifics
|
|
|
|
| 41 |
- **Threshold**: 0.1
|
| 42 |
|
| 43 |
### Algorithm
|
|
|
|
| 53 |
|
| 54 |
| Model | Excluded Class | Forget class acc(loss) | Retain class acc(loss) |
|
| 55 |
|------------------------------------|----------------|-------------------------|-------------------------|
|
| 56 |
+
| cifar10_resnet18_SALUN_0.pth | Airplane | 0.7 (2.550) | 90.00 (0.347) |
|
| 57 |
+
| cifar10_resnet18_SALUN_1.pth | Automobile | 0.0 (2.976) | 88.73 (0.404) |
|
| 58 |
+
| cifar10_resnet18_SALUN_2.pth | Bird | 2.4 (2.862) | 89.13 (0.356) |
|
| 59 |
+
| cifar10_resnet18_SALUN_3.pth | Cat | 0.0 (3.640) | 92.13 (0.262) |
|
| 60 |
+
| cifar10_resnet18_SALUN_4.pth | Deer | 0.9 (2.749) | 89.74 (0.349) |
|
| 61 |
+
| cifar10_resnet18_SALUN_5.pth | Dog | 3.7 (2.870) | 88.92 (0.363) |
|
| 62 |
+
| cifar10_resnet18_SALUN_6.pth | Frog | 1.7 (3.236) | 86.23 (0.486) |
|
| 63 |
+
| cifar10_resnet18_SALUN_7.pth | Horse | 0.0 (3.119) | 90.03 (0.342) |
|
| 64 |
+
| cifar10_resnet18_SALUN_8.pth | Ship | 9.4 (2.685) | 90.86 (0.320) |
|
| 65 |
+
| cifar10_resnet18_SALUN_9.pth | Truck | 0.5 (2.748) | 89.88 (0.362) |
|
| 66 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
|
|
|
|
|
|
|
|
|
| 68 |
|
|
|
|
|
|
|
|
|
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|