Aether258 commited on
Commit
1d10707
·
verified ·
1 Parent(s): b121006

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -70,7 +70,8 @@ only.
70
  | batch size | 128 |
71
  | this checkpoint | step 8000 (~1.24 epoch; 1 epoch = 6,444 steps) |
72
  | planned length | 20,000 steps |
73
- | lr | peak 2e-4, 1,000 warmup steps, cosine decay over 100,000 |
 
74
  | LoRA | rank 16 on the LLM, rank 32 on the action expert |
75
  | vision tower | **fully fine-tuned** -- the freeze filter matches only `.*llm.*` |
76
 
 
70
  | batch size | 128 |
71
  | this checkpoint | step 8000 (~1.24 epoch; 1 epoch = 6,444 steps) |
72
  | planned length | 20,000 steps |
73
+ | lr | cosine decay, 1,000 warmup steps: peak 2.5e-5 -> 2.5e-6 over 30,000 steps |
74
+ | | (`CosineDecaySchedule` defaults -- `pi05_bi` does not override `lr_schedule`; the `peak_lr=2e-4 / decay_steps=100000` block in `config.py` is referenced only by `pi05_single*`) |
75
  | LoRA | rank 16 on the LLM, rank 32 on the action expert |
76
  | vision tower | **fully fine-tuned** -- the freeze filter matches only `.*llm.*` |
77