Mobile Testing
Collection
2 items
ā¢
Updated
Manohar Allu
manohar03
AI & ML interests
None yet
Recent Activity
updated
a collection
about 9 hours ago
Mobile Testing
liked
a model
about 9 hours ago
LiquidAI/LFM2-2.6B-Exp-GGUF
liked
a model
9 days ago
nvidia/llama-nemotron-rerank-1b-v2
Organizations
updated
a
collection
about 9 hours ago
liked
a
model
about 9 hours ago
liked
a
model
9 days ago
liked
a
model
10 days ago
liked
a
Space
10 days ago
updated
a
collection
12 days ago
liked
a
model
12 days ago
liked
a
model
18 days ago
updated
a
collection
19 days ago
updated
a
collection
24 days ago
liked
a
model
24 days ago
updated
a
collection
27 days ago
liked
a
model
27 days ago
liked
a
Space
30 days ago
updated
a
collection
about 1 month ago
liked
2
models
about 1 month ago
updated
a
collection
about 2 months ago
liked
a
model
about 2 months ago
reacted to
nouamanetazi's
post with ā¤ļø
about 2 months ago
Post
4119
After training šš¦šØš„ššš on ššš ššššš¬ for nearly a month, I've come to realize something most people overlook: š¢š§šš«šš¬šš«š®ššš®š«š š¢š¬ šš”š š¦šš¤š-šØš«-šš«ššš¤ šššššØš« š¢š§ ššš šš«šš¢š§š¢š§š . š„
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious šššš šš«š«šØš«š¬, or when your expensive GPU cluster is running at šš% šššš¢šš¢šš§šš², the problem isn't your model. It's most probably a š¦š¢š¬š®š¬š šØš šš”š š”šš«šš°šš«š. š ļø
Questions that seemed simple but had no clear answers: Why is ššØš šš«šš¢š§š¢š§š š¬š„šØš°šš« šš”šš§ ššš§š¬š š¦šØššš„š¬? Which šššš šš„šš š¬ should we actually set? How often should we checkpoint without killing throughput?
That's why we built šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤ š: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the š¢š§šš«šš¬šš«š®ššš®š«š š„šš²šš« that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: šššš š”š¢ššš¢š§š š šš/š¬, šššš¢š§š¤ š.š š«šššš”š¢š§š ššš šš/š¬, šššš ššš§š šš šš.š šš/š¬. Then we ran collective operations across ššš šššš¬ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ššš šš/š¬ on a single node to ššš-ššš šš/š¬ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤: https://lnkd.in/e5MKXUHS
Shared with ā¤ļø by the HuggingFace team
Everyone talks about model architecture and data quality. And yes, those matter immensely. But here's what nobody tells you: when your training run fails at 2 AM because of mysterious šššš šš«š«šØš«š¬, or when your expensive GPU cluster is running at šš% šššš¢šš¢šš§šš², the problem isn't your model. It's most probably a š¦š¢š¬š®š¬š šØš šš”š š”šš«šš°šš«š. š ļø
Questions that seemed simple but had no clear answers: Why is ššØš šš«šš¢š§š¢š§š š¬š„šØš°šš« šš”šš§ ššš§š¬š š¦šØššš„š¬? Which šššš šš„šš š¬ should we actually set? How often should we checkpoint without killing throughput?
That's why we built šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤ š: a complete guide covering everything from model architecture and data curation to the SmolLM3 training marathon, post-training techniques, and crucially, the š¢š§šš«šš¬šš«š®ššš®š«š š„šš²šš« that most teams get wrong.
We validated real vs theoretical bandwidth across the entire stack: šššš š”š¢ššš¢š§š š šš/š¬, šššš¢š§š¤ š.š š«šššš”š¢š§š ššš šš/š¬, šššš ššš§š šš šš.š šš/š¬. Then we ran collective operations across ššš šššš¬ (16 nodes, 8xH100s each) and measured how performance degrades at scale: all-reduce drops from ššš šš/š¬ on a single node to ššš-ššš šš/š¬ across 16 nodes.
If you've ever wondered why your training runs are slower than they should be, or you're planning to scale up and want to avoid expensive mistakes, this guide might save you weeks of debugging.
šš”š šš¦šØš„ šš«šš¢š§š¢š§š šš„šš²ššØšØš¤: https://lnkd.in/e5MKXUHS
Shared with ā¤ļø by the HuggingFace team