ROC-Checkpoints/training_log.md

Vanilla GPT Training Log

Started: 2026-01-01 17:39:37

Output Directory: out_ppiGPLM_Med4-solo_hope.v3-8ki_4k_768_vanilla

Mode: Vanilla (standard GPT)

---

Hyperparameters

Model Architecture

Parameter	Value
n_layer	12
n_head	12
n_embd	768
block_size	4096
vocab_size	29
dropout	0.2
bias	True

Training Configuration

Parameter	Value
dataset	shakespeare_char
batch_size	12
gradient_accumulation_steps	1
tokens_per_iter	98,304
learning_rate	0.0005
min_lr	1e-05
max_iters	8001
warmup_iters	200
lr_decay_iters	8000
weight_decay	0.1
beta1	0.9
beta2	0.99
grad_clip	1.0
eval_interval	250
eval_iters	100

System Configuration

Parameter	Value
device	cuda:0
dtype	bfloat16
compile	False
Total Parameters	85,078,272

---

Training Progress

Iteration	Train Loss	Val Loss	Checkpoint	Timestamp
250	2.7533	2.7560	ckpt_250.pt	2026-01-01 17:45:05
500	2.6923	2.6987	ckpt_500.pt	2026-01-01 17:49:45
750	2.5995	2.6146	ckpt_750.pt	2026-01-01 17:54:25
1000	2.3717	2.3937	ckpt_1000.pt	2026-01-01 17:59:05
1250	2.0253	2.0677	ckpt_1250.pt	2026-01-01 18:03:45
1500	1.2687	1.3372	ckpt_1500.pt	2026-01-01 18:08:25
1750	0.6444	0.7193	ckpt_1750.pt	2026-01-01 18:13:06
2000	0.3304	0.4024	ckpt_2000.pt	2026-01-01 18:17:47
2250	0.1911	0.2712	ckpt_2250.pt	2026-01-01 18:22:28
2500	0.1344	0.1991	ckpt_2500.pt	2026-01-01 18:27:08
2750	0.0992	0.1633	ckpt_2750.pt	2026-01-01 18:31:49
3000	0.0761	0.1379	ckpt_3000.pt	2026-01-01 18:36:29
3250	0.0658	0.1258	ckpt_3250.pt	2026-01-01 18:41:10
3500	0.0553	0.1021	ckpt_3500.pt	2026-01-01 18:45:50
3750	0.0519	0.0874	ckpt_3750.pt	2026-01-01 18:50:31
4000	0.0462	0.0907	ckpt_4000.pt	2026-01-01 18:55:11
4250	0.0417	0.0802	ckpt_4250.pt	2026-01-01 18:59:51
4500	0.0389	0.0802	ckpt_4500.pt	2026-01-01 19:04:32
4750	0.0368	0.0802	ckpt_4750.pt	2026-01-01 19:09:12
5000	0.0334	0.0859	ckpt_5000.pt	2026-01-01 19:13:52
5250	0.0318	0.0872	ckpt_5250.pt	2026-01-01 19:18:32
5500	0.0287	0.0824	ckpt_5500.pt	2026-01-01 19:23:13
5750	0.0271	0.0768	ckpt_5750.pt	2026-01-01 19:27:53
6000	0.0256	0.0927	ckpt_6000.pt	2026-01-01 19:32:33
6250	0.0249	0.0833	ckpt_6250.pt	2026-01-01 19:37:13
6500	0.0232	0.0919	ckpt_6500.pt	2026-01-01 19:41:53
6750	0.0224	0.0897	ckpt_6750.pt	2026-01-01 19:46:33
7000	0.0219	0.0891	ckpt_7000.pt	2026-01-01 19:51:17
7250	0.0217	0.0880	ckpt_7250.pt	2026-01-01 19:56:00
7500	0.0211	0.0914	ckpt_7500.pt	2026-01-01 20:00:44
7750	0.0209	0.0872	ckpt_7750.pt	2026-01-01 20:05:26
8000	0.0202	0.0821	ckpt_8000.pt	2026-01-01 20:10:10