The Ultimate MoE Training Handbook
🧩
103
A hands-on guide to training Mixture-of-Experts models
Ridha Chahed
ridha
AI & ML interests
ML
Recent Activity
liked a Space about 10 hours ago
huggingface/moe-blogpost upvoted an article 3 days ago
MOPD: how frontier labs merge RL experts into one model and how to do it using TRL upvoted a paper 6 days ago
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via
Reinforcement LearningOrganizations
liked a
Space about 10 hours ago
upvoted an article 3 days ago
Article
MOPD: how frontier labs merge RL experts into one model and how to do it using TRL
sergiopaniego
• • 21 upvoted a paper 6 days ago
upvoted 2
articles 6 days ago
Article
A brief history of distillation in AI
sergiopaniego
• • 5 Article
Distillation in 2026 (so far): which frontier models use it and how
sergiopaniego
• • 25 upvoted an article 8 days ago
Article
A curriculum for post-training agents
huggingface
• • 104 liked a
Space 14 days ago
liked a
Space 24 days ago
upvoted a paper 2 months ago
upvoted an article 2 months ago
Article
How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code
nielsr
• • 34 liked a
Space 6 months ago
upvoted a collection 7 months ago
upvoted 2
articles 8 months ago
Article
PRX Part 3 — Training a Text-to-Image Model in 24h!
Photoroom
• • 69 Article
Mixture of Experts (MoEs) in Transformers
- +5
ariG23498, pcuenq, merve, IlyasMoutawwakil, ArthurZ, sergiopaniego, Molbap
• • 182 upvoted an article 9 months ago
Article
Did GPT 5.2 make a breakthrough discovery in theoretical physics?
dlouapre
• • 63 upvoted an article 12 months ago
Article
Diffusers welcomes FLUX-2
- +6
YiYiXu, dg845, sayakpaul, OzzyGT, dn6, ariG23498, linoyts, multimodalart
• • 192 upvoted an article about 1 year ago
Article
Scaling Test-Time Compute to Achieve Gold Medal at IOI 2025 with Open-Weight Models
nvidia
• • 19 reacted to Kseniase's post with 👍 about 1 year ago
Post
4,167
5 Lectures and keynotes defining AI right now
If you want to understand the multifaceted AI landscape in 2025 and see where the field is heading – start with (or revisit) these legendary talks. They can help you capture what’s happening in AI from multiple angles:
1. Andrej Karpathy: Software Is Changing (Again) → https://www.youtube.com/watch?v=LCEmiRjPEtQ
Unveils Software 3.0 – a paradigm where LLMs are the new computers, programmed with prompts instead of code. The key: developers must now master coding, training, and prompting as AI becomes the heart of software building
2. Richard Sutton, The OaK Architecture: A Vision of SuperIntelligence from Experience → https://www.youtube.com/watch?v=gEbbGyNkR2U
Unveils the OaK (Options and Knowledge) architecture – a model-based RL framework for continual intelligence, where every component learns, meta-learns & builds hierarchical abstractions
3. GTC March 2025 Keynote with NVIDIA CEO Jensen Huang → https://www.youtube.com/watch?v=_waPvOwL9Z8
Dives into the accelerated computing and the importance of Physical AI. From the Blackwell GPU architecture & AI factories to breakthroughs in agentic AI & robotics, Jensen Huang explains how NVIDIA aims to power every layer of the AI ecosystem
4. Yann LeCun "Mathematical Obstacles on the Way to Human-Level AI" → https://www.youtube.com/watch?v=ETZfkkv6V7
Yann LeCun always argues we need a new path to machines that reason about the world – not LLMs or RL. So this lecture is about self-supervised systems with world models, planning, memory and energy-based learning
5. Andrew Ng: State of AI Agents → https://www.youtube.com/watch?v=4pYzYmSdSH4
Highlights one of the most pressing topics of 2025 – agents, explaining why most effective AI agents rely on simple, linear workflows built from modular “Lego-brick” tasks + what predicts AI startup success in the new agent era
Subscribe to the Turing Post: https://www.turingpost.com/subscribe –your shortcut to deep, clear AI analysis
If you want to understand the multifaceted AI landscape in 2025 and see where the field is heading – start with (or revisit) these legendary talks. They can help you capture what’s happening in AI from multiple angles:
1. Andrej Karpathy: Software Is Changing (Again) → https://www.youtube.com/watch?v=LCEmiRjPEtQ
Unveils Software 3.0 – a paradigm where LLMs are the new computers, programmed with prompts instead of code. The key: developers must now master coding, training, and prompting as AI becomes the heart of software building
2. Richard Sutton, The OaK Architecture: A Vision of SuperIntelligence from Experience → https://www.youtube.com/watch?v=gEbbGyNkR2U
Unveils the OaK (Options and Knowledge) architecture – a model-based RL framework for continual intelligence, where every component learns, meta-learns & builds hierarchical abstractions
3. GTC March 2025 Keynote with NVIDIA CEO Jensen Huang → https://www.youtube.com/watch?v=_waPvOwL9Z8
Dives into the accelerated computing and the importance of Physical AI. From the Blackwell GPU architecture & AI factories to breakthroughs in agentic AI & robotics, Jensen Huang explains how NVIDIA aims to power every layer of the AI ecosystem
4. Yann LeCun "Mathematical Obstacles on the Way to Human-Level AI" → https://www.youtube.com/watch?v=ETZfkkv6V7
Yann LeCun always argues we need a new path to machines that reason about the world – not LLMs or RL. So this lecture is about self-supervised systems with world models, planning, memory and energy-based learning
5. Andrew Ng: State of AI Agents → https://www.youtube.com/watch?v=4pYzYmSdSH4
Highlights one of the most pressing topics of 2025 – agents, explaining why most effective AI agents rely on simple, linear workflows built from modular “Lego-brick” tasks + what predicts AI startup success in the new agent era
Subscribe to the Turing Post: https://www.turingpost.com/subscribe –your shortcut to deep, clear AI analysis
upvoted 2
articles about 1 year ago
Article
SmolLM3: smol, multilingual, long-context reasoner
- +21
eliebak, cmpatino, anton-l, edbeeching, m-ric, nouamanetazi, akseljoonas, guipenedo, hynky, clefourrier, SaylorTwift, kashif, qgallouedec, hlarcher, glutamatt, Xenova, reach-vb, ngxson, craffel, lewtun, loubnabnl, lvwerra, thomwolf
• • 805 Article
Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers
- +5
ariG23498, sergiopaniego, reach-vb, pcuenq, ArthurZ, SaylorTwift, cyrilvallez
• • 192