view article Article The Most Intuitive Way to Understand Score Function Centering: Understand Trainer-Generator Mismatch in Async RL from the Score Function NormalUhr • 22 days ago • 16
view article Article Unlocking Agentic RL Training for GPT-OSS: A Practical Retrospective LinkedIn • Jan 27 • 82
Running on CPU Upgrade Featured 3.35k The Smol Training Playbook 📚 3.35k The secrets to building world-class LLMs
view article Article SmolLM3: smol, multilingual, long-context reasoner +21 eliebak, cmpatino, anton-l, edbeeching, m-ric, nouamanetazi, akseljoonas, guipenedo, hynky, clefourrier, SaylorTwift, kashif, qgallouedec, hlarcher, glutamatt, Xenova, reach-vb, ngxson, craffel, lewtun, loubnabnl, lvwerra, thomwolf • Jul 8, 2025 • 805
Running Featured 1.46k FineWeb: decanting the web for the finest text data at scale 🍷 1.46k Explore the FineWeb dataset and its creation process
Running 4.12k The Ultra-Scale Playbook 🌌 4.12k The ultimate guide to training LLM on large GPU Clusters
view article Article DualPipe Explained: A Comprehensive Guide to DualPipe That Anyone Can Understand—Even Without a Distributed Training Background NormalUhr • Feb 28, 2025 • 22
Zephyr ORPO Collection Models and datasets to align LLMs with Odds Ratio Preference Optimisation (ORPO). Recipes here: https://github.com/huggingface/alignment-handbook • 3 items • Updated Aug 3 • 19