hazarsozer/hallm-wikitext103

HaLLM — trained checkpoints

Six GPT-style LMs (d=512, GPT-2 BPE, ctx=512) trained from scratch on WikiText-103 under a matched budget (50k steps ≈ 614M tokens, seed 1337) with different weight-sharing schemes. Code + full results: https://github.com/hazarsozer/hallm (ITU graduation thesis; extends HaLViT's W+Wᵀ sharing to language models).

folder	sharing	depth	non-emb params	test PPL
A0	none	8	25.2M	26.06
A1	ALBERT cross-layer	8	3.1M	35.63
A2	HaLViT W+Wᵀ	8	12.6M	29.68
A3	both	8	1.6M	43.30
A2-iso	W+Wᵀ, iso-storage vs A0	16	25.2M	27.01
A0-deep	none (control)	16	50.3M	23.98

Each folder: model.pt (state_dict + model_config, torch.load(..., weights_only=True)) and the exact training YAML. Load with hallm.model.GPT from the GitHub repo.