Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"/> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"/> | |
| <title>LUNA-300M Program</title> | |
| <style> | |
| :root{--bg:#0b1020;--panel:#131a30;--panel2:#182139;--line:#26304d;--text:#e8ecf8;--muted:#9aa7c7;--accent:#8b97ad;--accent2:#c3cbd9;--green:#34d399;} | |
| *{box-sizing:border-box;margin:0;padding:0;} | |
| body{background:var(--bg);color:var(--text);font-family:system-ui,-apple-system,'Segoe UI',Roboto,sans-serif;line-height:1.55;} | |
| a{color:var(--accent2);text-decoration:none;} a:hover{text-decoration:underline;} | |
| .wrap{max-width:1080px;margin:0 auto;padding:32px 20px 64px;} | |
| header.hero{padding:40px 0 28px;border-bottom:1px solid var(--line);} | |
| .tag{display:inline-block;font-size:12px;letter-spacing:1.5px;text-transform:uppercase;color:var(--accent2);font-weight:700;} | |
| h1{font-size:34px;margin:8px 0 6px;} h1 span{color:var(--accent);} | |
| .sub{color:var(--muted);max-width:760px;} | |
| .links{margin-top:14px;} .links a{margin-right:14px;font-weight:600;} | |
| .cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(170px,1fr));gap:14px;margin:26px 0 8px;} | |
| .card{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:16px 18px;} | |
| .card .v{font-size:22px;font-weight:800;color:var(--accent);} .card .l{font-size:12px;color:var(--muted);margin-top:4px;text-transform:uppercase;letter-spacing:.5px;} | |
| .tabs{display:flex;flex-wrap:wrap;gap:8px;margin:28px 0 18px;} | |
| .tab{background:var(--panel);border:1px solid var(--line);color:var(--muted);padding:10px 18px;border-radius:999px;cursor:pointer;font-weight:600;font-size:14px;} | |
| .tab.active{background:var(--accent);color:#fff;border-color:var(--accent);} | |
| .panel{display:none;} .panel.active{display:block;} | |
| .grid2{display:grid;grid-template-columns:1fr 1fr;gap:16px;} | |
| @media(max-width:760px){.grid2{grid-template-columns:1fr;}} | |
| table{width:100%;border-collapse:collapse;background:var(--panel);border-radius:12px;overflow:hidden;font-size:14px;} | |
| th,td{text-align:left;padding:10px 14px;border-bottom:1px solid var(--line);} | |
| th{background:var(--panel2);color:var(--muted);font-size:12px;text-transform:uppercase;letter-spacing:.6px;} | |
| tr:last-child td{border-bottom:none;} | |
| .h2{font-size:20px;margin:22px 0 12px;} | |
| .note{color:var(--muted);font-size:13px;margin:6px 0 16px;} | |
| .filters{display:flex;gap:10px;margin:14px 0;flex-wrap:wrap;} | |
| select{background:var(--panel2);color:var(--text);border:1px solid var(--line);border-radius:8px;padding:9px 12px;font-size:14px;} | |
| .sample{background:var(--panel);border:1px solid var(--line);border-radius:12px;padding:16px 18px;margin-bottom:12px;} | |
| .sample h4{font-size:15px;margin-bottom:6px;} | |
| .sample .i{color:var(--muted);font-size:13px;margin-bottom:8px;white-space:pre-wrap;} | |
| .sample .o{font-size:14px;white-space:pre-wrap;} | |
| .meta{display:inline-block;background:var(--panel2);border:1px solid var(--line);color:var(--accent2);font-size:11px;border-radius:6px;padding:2px 8px;margin:2px 6px 2px 0;} | |
| .pill{display:inline-block;background:rgba(139,151,173,.12);color:var(--accent);border:1px solid rgba(139,151,173,.3);font-size:11px;border-radius:6px;padding:2px 8px;margin:2px 6px 2px 0;} | |
| .ok{color:var(--green);font-weight:700;} | |
| code{background:var(--panel2);padding:1px 5px;border-radius:5px;font-size:13px;} | |
| footer{margin-top:44px;padding-top:20px;border-top:1px solid var(--line);color:var(--muted);font-size:13px;} | |
| </style> | |
| </head> | |
| <body> | |
| <div class="wrap"> | |
| <header class="hero"> | |
| <div class="tag">ASTERIZER · LUNA Model Family</div> | |
| <h1>LUNA-300M <span>Program</span></h1> | |
| <p class="sub">A ~303M-parameter, English-first causal language model — pretrained <b>from scratch</b> on the 4.5B-token <b>LUNA_PreTrain</b> corpus. The scaled-up sibling of <a href="https://huggingface.co/ASTERIZER/LUNA-100M">LUNA-100M</a>.</p> | |
| <div class="links"> | |
| <a href="https://huggingface.co/ASTERIZER/LUNA-300M">Model · LUNA-300M</a> | |
| <a href="https://huggingface.co/ASTERIZER/LUNA-100M">Model · LUNA-100M</a> | |
| <a href="https://huggingface.co/datasets/ASTERIZER/LUNA_PreTrain">Dataset · LUNA_PreTrain</a> | |
| <a href="https://huggingface.co/spaces/ASTERIZER/LUNA">Workspace</a> | |
| <a href="https://huggingface.co/collections/ASTERIZER/luna-300m-program-6a91195ab75671026f83ac94">Collection</a> | |
| </div> | |
| </header> | |
| <div class="cards"> | |
| <div class="card"><div class="v">≈303M</div><div class="l">Parameters</div></div> | |
| <div class="card"><div class="v">31,052</div><div class="l">Pretrain steps</div></div> | |
| <div class="card"><div class="v">4.5B</div><div class="l">Tokens trained</div></div> | |
| <div class="card"><div class="v">1,024</div><div class="l">Context window</div></div> | |
| </div> | |
| <h2>Status</h2> | |
| <p class="note"><span class="ok">✓ TRAINED (pretraining complete)</span> — final fp32 weights available at <code>pretrained/final/lit_model.pth</code>. Instruction tuning (RAG + MCP SFT, mirroring LUNA-100M) is planned on top of this checkpoint.</p> | |
| <div class="grid2"> | |
| <div> | |
| <h2>Architecture</h2> | |
| <table><thead><tr><th>Property</th><th>Value</th></tr></thead><tbody> | |
| <tr><td>Layers</td><td>20</td></tr> | |
| <tr><td>Hidden size</td><td>1024</td></tr> | |
| <tr><td>Heads</td><td>16</td></tr> | |
| <tr><td>Context</td><td>1,024 tokens</td></tr> | |
| <tr><td>Vocab</td><td>50,304 (Pythia-160m tokenizer)</td></tr> | |
| <tr><td>Precision</td><td>bf16 / fp16</td></tr> | |
| </tbody></table> | |
| </div> | |
| <div> | |
| <h2>Training</h2> | |
| <table><thead><tr><th>Property</th><th>Value</th></tr></thead><tbody> | |
| <tr><td>Corpus</td><td>LUNA_PreTrain (4,515,286,950 tokens)</td></tr> | |
| <tr><td>Optimizer</td><td>AdamW (wd 0.1, clip 1.0)</td></tr> | |
| <tr><td>LR</td><td>3e-4 → 3e-5 cosine</td></tr> | |
| <tr><td>Global batch</td><td>120</td></tr> | |
| <tr><td>Steps</td><td>31,052</td></tr> | |
| </tbody></table> | |
| </div> | |
| </div> | |
| <h2>Checkpoints</h2> | |
| <table><thead><tr><th>File</th><th>Description</th></tr></thead><tbody> | |
| <tr><td><code>pretrained/final/lit_model.pth</code></td><td>Final fp32 weights (1.2 GB) — recommended</td></tr> | |
| <tr><td><code>pretrained/step-00031052/lit_model.pth</code></td><td>Final logged step</td></tr> | |
| <tr><td><code>pretrained/step-00031000/lit_model.pth</code></td><td>Milestone checkpoint</td></tr> | |
| <tr><td><code>pretrained/latest.pt</code></td><td>Weights + optimizer state (3.6 GB)</td></tr> | |
| </tbody></table> | |
| <footer> | |
| Data & models by <a href="https://huggingface.co/ASTERIZER">ASTERIZER</a> · Static program page · No analytics | |
| </footer> | |
| </div> | |
| </body> | |
| </html> | |