Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,47 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: mit
|
| 4 |
+
library_name: pytorch
|
| 5 |
+
tags:
|
| 6 |
+
- continual-learning
|
| 7 |
+
- catastrophic-forgetting
|
| 8 |
+
- information-geometry
|
| 9 |
+
- spectral-methods
|
| 10 |
+
- computer-vision
|
| 11 |
+
metrics:
|
| 12 |
+
- accuracy
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# Anastrophic Regularization CNN (Split-MNIST)
|
| 16 |
+
|
| 17 |
+
This model card hosts the weights for a CNN trained using **Anastrophic Regularization ($\mathcal{R}_{ana}$)**, a novel approach to mitigate catastrophic forgetting in sequential learning tasks.
|
| 18 |
+
|
| 19 |
+
## Model Description
|
| 20 |
+
Anastrophic Regularization is derived from **Anastrophic Theory**, a mathematical framework for analyzing discrete periodic systems. Unlike standard $L_2$ decay or EWC, this method preserves the structural "Harmonic Memory" of the network by guiding weight evolution along Fisher-Rao geodetic paths.
|
| 21 |
+
|
| 22 |
+
### Key Advantages
|
| 23 |
+
* **Maximum Plasticity**: Weights adapt to new tasks while maintaining the global periodic functional invariants of previous ones.
|
| 24 |
+
* **100% Data-Free**: Operates strictly in the spectral domain via Fast Fourier Transforms (FFT). No access to previous training data is required.
|
| 25 |
+
* **Privacy Preserving**: Ideal for environments with data-retention constraints where EWC or Replay buffers are not feasible.
|
| 26 |
+
|
| 27 |
+
## Intended Use
|
| 28 |
+
This specific model serves as a benchmark for **Continual Learning**. It was trained on the Split-MNIST dataset:
|
| 29 |
+
1. **Task A**: Digits 0-4
|
| 30 |
+
2. **Task B**: Digits 5-9 (Trained using $\mathcal{R}_{ana}$ to prevent forgetting Task A).
|
| 31 |
+
|
| 32 |
+
## Evaluation Results
|
| 33 |
+
The model achieves the following performance:
|
| 34 |
+
* **Task B (Current) Accuracy**: ~86.69%
|
| 35 |
+
* **Task A (Retained) Accuracy**: ~71.16%
|
| 36 |
+
|
| 37 |
+
## Mathematical Formulation
|
| 38 |
+
The weights were optimized using the following objective:
|
| 39 |
+
|
| 40 |
+
$$\mathcal{R}_{ana}(W) = \lambda(1 - \Phi(Spec(W))) + \eta BB(W, W_{prev})$$
|
| 41 |
+
|
| 42 |
+
## Citation and Full Paper
|
| 43 |
+
For the complete theoretical framework, proof of the Fisher-Rao geodetic paths, and the original publication, please refer to:
|
| 44 |
+
|
| 45 |
+
**Zenodo Repository:** [https://zenodo.org/records/18699347]
|
| 46 |
+
|
| 47 |
+
**GitHub Implementation:** [https://github.com/MituMath/Anastrophic-Regularization-PyTorch]
|