Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>One training step of a 1,024-GPU MoE</title> | |
| <meta name="description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live in your browser."> | |
| <meta name="twitter:card" content="player"> | |
| <meta name="twitter:title" content="One training step of a 1,024-GPU MoE"> | |
| <meta name="twitter:description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live. Hover a GPU to see what it is doing."> | |
| <meta name="twitter:player" content="https://thomwolf-moe-training-step.static.hf.space/player.html?embed=1&theme=dark"> | |
| <meta name="twitter:player:width" content="640"> | |
| <meta name="twitter:player:height" content="480"> | |
| <meta name="twitter:image" content="https://thomwolf-moe-training-step.static.hf.space/card.png"> | |
| <meta name="twitter:image:alt" content="An isometric city of 1,024 GPUs in four pipeline stages, with forward passes in blue and backward passes in orange."> | |
| <meta property="og:type" content="website"> | |
| <meta property="og:title" content="One training step of a 1,024-GPU MoE"> | |
| <meta property="og:description" content="A 45-layer mixture-of-experts model running one training step on 1,024 GPUs, live in your browser."> | |
| <meta property="og:image" content="https://thomwolf-moe-training-step.static.hf.space/card.png"> | |
| <meta property="og:url" content="https://thomwolf-moe-training-step.static.hf.space/share.html"> | |
| <style> | |
| html, body { margin: 0; height: 100%; background: #111215; color: #e6e8eb; font: 400 14px/1.4 "IBM Plex Sans", -apple-system, "Helvetica Neue", Arial, sans-serif; } | |
| body { display: flex; flex-direction: column; } | |
| iframe { flex: 1; width: 100%; border: 0; display: block; } | |
| a.post { display: block; padding: 12px 16px; text-align: center; color: inherit; text-decoration: none; border-top: 1px solid #2a2d33; } | |
| a.post:hover { text-decoration: underline; } | |
| </style> | |
| </head> | |
| <body> | |
| <iframe src="player.html?embed=1&theme=dark" title="One training step of a 1,024-GPU MoE"></iframe> | |
| <a class="post" href="https://huggingface.co/spaces/huggingface/moe-blogpost">Read the post: how to train a mixture-of-experts model, knob by knob →</a> | |
| </body> | |
| </html> | |