B4K2xx commited on
Commit
b644a66
·
verified ·
1 Parent(s): e027f8b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -21,6 +21,13 @@ architectural axis from a shared baseline, trained on 1B tokens of
21
  (`sample-10BT`, GPT-2 tokenizer). Full paper, figures, and code:
22
  https://github.com/<your-username>/<your-repo> (GitHub repo URL — fill in once pushed).
23
 
 
 
 
 
 
 
 
24
  Every subfolder (`m01/` ... `m19/`) is one model: `model.safetensors` (weights,
25
  optimizer state stripped), `config.json` (architecture + training hyperparameters),
26
  `eval_results.json` (perplexity / FLOPs / throughput / VRAM), and
 
21
  (`sample-10BT`, GPT-2 tokenizer). Full paper, figures, and code:
22
  https://github.com/<your-username>/<your-repo> (GitHub repo URL — fill in once pushed).
23
 
24
+ ![Training and validation loss curves for all 19 models](images/loss_curves_all.png)
25
+
26
+ <table><tr>
27
+ <td><img src="images/bench_vs_ppl.png" alt="Zero-shot capability vs. validation perplexity"></td>
28
+ <td><img src="images/pareto.png" alt="Perplexity vs. KV-cached decode speed Pareto trade-off"></td>
29
+ </tr></table>
30
+
31
  Every subfolder (`m01/` ... `m19/`) is one model: `model.safetensors` (weights,
32
  optimizer state stripped), `config.json` (architecture + training hyperparameters),
33
  `eval_results.json` (perplexity / FLOPs / throughput / VRAM), and