Clean README for GitHub, use deploy script for HF metadata
Browse files
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
title: Modular Addition Feature Learning
|
| 3 |
-
emoji: π’
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: yellow
|
| 6 |
sdk: gradio
|
|
@@ -28,14 +28,7 @@ This repository provides the code for studying how a two-layer neural network le
|
|
| 28 |
2. **Lottery Ticket Dynamics** β Random initialization determines which frequency each neuron will specialize in: the frequency with the best initial phase alignment wins a winner-take-all competition.
|
| 29 |
3. **Grokking** β Under partial data with weight decay, the network first memorizes, then suddenly generalizes through a three-stage process: memorization β sparsification β cleanup.
|
| 30 |
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
An interactive Gradio app visualizes all results with math explanations and interactive Plotly charts:
|
| 34 |
-
|
| 35 |
-
- **9 analysis tabs** covering mechanism, dynamics, grokking, and analytical simulations
|
| 36 |
-
- **Interactive features**: neuron frequency inspector, logit explorer, grokking epoch slider
|
| 37 |
-
- **On-demand training**: generate results for any odd $p \geq 3$ directly from the app
|
| 38 |
-
- **Pre-computed examples** included for $p = 15, 23, 29, 31$
|
| 39 |
|
| 40 |
### Launch Locally
|
| 41 |
|
|
@@ -63,36 +56,14 @@ huggingface-cli login
|
|
| 63 |
|
| 64 |
**Upload to the Space:**
|
| 65 |
|
| 66 |
-
```python
|
| 67 |
-
from huggingface_hub import HfApi
|
| 68 |
-
api = HfApi()
|
| 69 |
-
api.upload_folder(
|
| 70 |
-
folder_path=".",
|
| 71 |
-
repo_id="y-agent/modular-addition-feature-learning",
|
| 72 |
-
repo_type="space",
|
| 73 |
-
ignore_patterns=[
|
| 74 |
-
"trained_models/*", "saved_models/*", "src/saved_models/*",
|
| 75 |
-
".git/*", ".claude/*", ".DS_Store", "tmp/*",
|
| 76 |
-
"notebooks/*", "figures/*", "__pycache__/*", "src/wandb/*",
|
| 77 |
-
],
|
| 78 |
-
commit_message="Update app",
|
| 79 |
-
)
|
| 80 |
-
```
|
| 81 |
-
|
| 82 |
-
Or as a one-liner from the project root:
|
| 83 |
-
|
| 84 |
```bash
|
| 85 |
-
python
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
repo_type='space', ignore_patterns=[
|
| 89 |
-
'trained_models/*','saved_models/*','src/saved_models/*',
|
| 90 |
-
'.git/*','.claude/*','.DS_Store','tmp/*',
|
| 91 |
-
'notebooks/*','figures/*','__pycache__/*','src/wandb/*'],
|
| 92 |
-
commit_message='Update app')
|
| 93 |
-
"
|
| 94 |
```
|
| 95 |
|
|
|
|
|
|
|
| 96 |
**What gets uploaded:** Only the files the app needs β `hf_app/`, `precompute/`, `precomputed_results/`, `src/`, `requirements.txt`, `README.md`. Model checkpoints, notebooks, and figures are excluded.
|
| 97 |
|
| 98 |
**On-demand training:** Users can generate results for new $p$ values directly from the app's "Generate" button. Streaming logs show real-time training progress. New results are auto-committed back to the Space repo so they persist across restarts.
|
|
|
|
| 1 |
---
|
| 2 |
title: Modular Addition Feature Learning
|
| 3 |
+
emoji: "π’"
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: yellow
|
| 6 |
sdk: gradio
|
|
|
|
| 28 |
2. **Lottery Ticket Dynamics** β Random initialization determines which frequency each neuron will specialize in: the frequency with the best initial phase alignment wins a winner-take-all competition.
|
| 29 |
3. **Grokking** β Under partial data with weight decay, the network first memorizes, then suddenly generalizes through a three-stage process: memorization β sparsification β cleanup.
|
| 30 |
|
| 31 |
+
An [**Interactive Demo**](https://huggingface.co/spaces/y-agent/modular-addition-feature-learning) on Hugging Face Spaces visualizes all results with 9 analysis tabs, interactive Plotly charts, and on-demand training for any odd $p \geq 3$. Pre-computed examples are included for $p = 15, 23, 29, 31$.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
### Launch Locally
|
| 34 |
|
|
|
|
| 56 |
|
| 57 |
**Upload to the Space:**
|
| 58 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
```bash
|
| 60 |
+
python deploy_to_hf.py
|
| 61 |
+
# or with a custom commit message:
|
| 62 |
+
python deploy_to_hf.py --message "Update app"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
```
|
| 64 |
|
| 65 |
+
The deploy script prepends the required HuggingFace Space metadata (SDK config, app path, etc.) to `README.md` before uploading, so the GitHub README stays clean.
|
| 66 |
+
|
| 67 |
**What gets uploaded:** Only the files the app needs β `hf_app/`, `precompute/`, `precomputed_results/`, `src/`, `requirements.txt`, `README.md`. Model checkpoints, notebooks, and figures are excluded.
|
| 68 |
|
| 69 |
**On-demand training:** Users can generate results for new $p$ values directly from the app's "Generate" button. Streaming logs show real-time training progress. New results are auto-committed back to the Space repo so they persist across restarts.
|