| # Vanilla GPT Training Log |
|
|
| **Started:** 2026-01-01 17:39:37 |
|
|
| **Output Directory:** `out_ppiGPLM_Med4-solo_hope.v3-8ki_4k_768_vanilla` |
|
|
| **Mode:** Vanilla (standard GPT) |
|
|
| --- |
|
|
| ## Hyperparameters |
|
|
| ### Model Architecture |
|
|
| | Parameter | Value | |
| |-----------|-------| |
| | n_layer | 12 | |
| | n_head | 12 | |
| | n_embd | 768 | |
| | block_size | 4096 | |
| | vocab_size | 29 | |
| | dropout | 0.2 | |
| | bias | True | |
| |
| ### Training Configuration |
| |
| | Parameter | Value | |
| |-----------|-------| |
| | dataset | shakespeare_char | |
| | batch_size | 12 | |
| | gradient_accumulation_steps | 1 | |
| | tokens_per_iter | 98,304 | |
| | learning_rate | 0.0005 | |
| | min_lr | 1e-05 | |
| | max_iters | 8001 | |
| | warmup_iters | 200 | |
| | lr_decay_iters | 8000 | |
| | weight_decay | 0.1 | |
| | beta1 | 0.9 | |
| | beta2 | 0.99 | |
| | grad_clip | 1.0 | |
| | eval_interval | 250 | |
| | eval_iters | 100 | |
| |
| ### System Configuration |
| |
| | Parameter | Value | |
| |-----------|-------| |
| | device | cuda:0 | |
| | dtype | bfloat16 | |
| | compile | False | |
| | Total Parameters | 85,078,272 | |
| |
| --- |
| |
| ## Training Progress |
| |
| | Iteration | Train Loss | Val Loss | Checkpoint | Timestamp | |
| |-----------|------------|----------|------------|----------| |
| | 250 | 2.7533 | 2.7560 | `ckpt_250.pt` | 2026-01-01 17:45:05 | |
| | 500 | 2.6923 | 2.6987 | `ckpt_500.pt` | 2026-01-01 17:49:45 | |
| | 750 | 2.5995 | 2.6146 | `ckpt_750.pt` | 2026-01-01 17:54:25 | |
| | 1000 | 2.3717 | 2.3937 | `ckpt_1000.pt` | 2026-01-01 17:59:05 | |
| | 1250 | 2.0253 | 2.0677 | `ckpt_1250.pt` | 2026-01-01 18:03:45 | |
| | 1500 | 1.2687 | 1.3372 | `ckpt_1500.pt` | 2026-01-01 18:08:25 | |
| | 1750 | 0.6444 | 0.7193 | `ckpt_1750.pt` | 2026-01-01 18:13:06 | |
| | 2000 | 0.3304 | 0.4024 | `ckpt_2000.pt` | 2026-01-01 18:17:47 | |
| | 2250 | 0.1911 | 0.2712 | `ckpt_2250.pt` | 2026-01-01 18:22:28 | |
| | 2500 | 0.1344 | 0.1991 | `ckpt_2500.pt` | 2026-01-01 18:27:08 | |
| | 2750 | 0.0992 | 0.1633 | `ckpt_2750.pt` | 2026-01-01 18:31:49 | |
| | 3000 | 0.0761 | 0.1379 | `ckpt_3000.pt` | 2026-01-01 18:36:29 | |
| | 3250 | 0.0658 | 0.1258 | `ckpt_3250.pt` | 2026-01-01 18:41:10 | |
| | 3500 | 0.0553 | 0.1021 | `ckpt_3500.pt` | 2026-01-01 18:45:50 | |
| | 3750 | 0.0519 | 0.0874 | `ckpt_3750.pt` | 2026-01-01 18:50:31 | |
| | 4000 | 0.0462 | 0.0907 | `ckpt_4000.pt` | 2026-01-01 18:55:11 | |
| | 4250 | 0.0417 | 0.0802 | `ckpt_4250.pt` | 2026-01-01 18:59:51 | |
| | 4500 | 0.0389 | 0.0802 | `ckpt_4500.pt` | 2026-01-01 19:04:32 | |
| | 4750 | 0.0368 | 0.0802 | `ckpt_4750.pt` | 2026-01-01 19:09:12 | |
| | 5000 | 0.0334 | 0.0859 | `ckpt_5000.pt` | 2026-01-01 19:13:52 | |
| | 5250 | 0.0318 | 0.0872 | `ckpt_5250.pt` | 2026-01-01 19:18:32 | |
| | 5500 | 0.0287 | 0.0824 | `ckpt_5500.pt` | 2026-01-01 19:23:13 | |
| | 5750 | 0.0271 | 0.0768 | `ckpt_5750.pt` | 2026-01-01 19:27:53 | |
| | 6000 | 0.0256 | 0.0927 | `ckpt_6000.pt` | 2026-01-01 19:32:33 | |
| | 6250 | 0.0249 | 0.0833 | `ckpt_6250.pt` | 2026-01-01 19:37:13 | |
| | 6500 | 0.0232 | 0.0919 | `ckpt_6500.pt` | 2026-01-01 19:41:53 | |
| | 6750 | 0.0224 | 0.0897 | `ckpt_6750.pt` | 2026-01-01 19:46:33 | |
| | 7000 | 0.0219 | 0.0891 | `ckpt_7000.pt` | 2026-01-01 19:51:17 | |
| | 7250 | 0.0217 | 0.0880 | `ckpt_7250.pt` | 2026-01-01 19:56:00 | |
| | 7500 | 0.0211 | 0.0914 | `ckpt_7500.pt` | 2026-01-01 20:00:44 | |
| | 7750 | 0.0209 | 0.0872 | `ckpt_7750.pt` | 2026-01-01 20:05:26 | |
| | 8000 | 0.0202 | 0.0821 | `ckpt_8000.pt` | 2026-01-01 20:10:10 | |
|
|
