update readme
Browse files
README.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
| 1 |
# jaxgmg2_shared_init
|
| 2 |
|
| 3 |
-
20 RL agent checkpoints studying the effect of shared initialization. Two base models
|
| 4 |
-
|
| 5 |
-
|
|
|
|
|
|
|
| 6 |
|
| 7 |
**WandB:** https://wandb.ai/devinterp/jaxgmg2_shared_init
|
| 8 |
|
|
@@ -48,7 +50,7 @@ Checkpoints are named `al_1.0_g_0.98_id_{run_id}_shared_init_seed_{seed}`.
|
|
| 48 |
See `train.yaml` in this repository. Run with:
|
| 49 |
|
| 50 |
```bash
|
| 51 |
-
|
| 52 |
```
|
| 53 |
|
| 54 |
from the [timaeus monorepo](https://github.com/timaeus-research/timaeus).
|
|
|
|
| 1 |
# jaxgmg2_shared_init
|
| 2 |
|
| 3 |
+
20 RL agent checkpoints studying the effect of shared initialization. Two base models
|
| 4 |
+
(`al_1.0_g_0.98_id_19_seed_981019` and `al_1.0_g_0.98_id_27_seed_981027` from
|
| 5 |
+
[jaxgmg2_3phase_optim_state](https://huggingface.co/timaeus/jaxgmg2_3phase_optim_state))
|
| 6 |
+
are each used as a shared starting point, then independently continued from checkpoint 0
|
| 7 |
+
(fresh optimizer state) with alpha=1.0 across 10 different random seeds each.
|
| 8 |
|
| 9 |
**WandB:** https://wandb.ai/devinterp/jaxgmg2_shared_init
|
| 10 |
|
|
|
|
| 50 |
See `train.yaml` in this repository. Run with:
|
| 51 |
|
| 52 |
```bash
|
| 53 |
+
timaeus run train.yaml
|
| 54 |
```
|
| 55 |
|
| 56 |
from the [timaeus monorepo](https://github.com/timaeus-research/timaeus).
|