H Company's new Holo2 model takes the lead in UI Localization
•
5
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
sergiopaniego
posted
an
update
about 22 hours ago
Post
187
if you're looking for a good first issue to get your open-source journey started, you could contribute to this TRL issue by documenting one impactful paper in the docs
we have a broad list to cover!! 🧐
https://github.com/huggingface/trl/issues/4407
we have a broad list to cover!! 🧐
https://github.com/huggingface/trl/issues/4407
ramzidecoster
published
an
article
7 days ago
Article
H Company's new Holo2 model takes the lead in UI Localization
•
5
hamza-hcompany
updated
a
model
7 days ago
hamza-hcompany
published
a
model
7 days ago
plcedoz38
updated
a
collection
8 days ago
sergiopaniego
posted
an
update
12 days ago
Post
423
Meet the Post-Training Toolkit (PTT), which easily integrates with TRL via a single callback, by Aditya Challapally (
@microsoft
):
🔍 Detects training issues early
🛠 Lets you intervene safely
📊 Keeps long training runs stable, auditable & efficient
Microsoft blog: https://devblogs.microsoft.com/engineering-at-microsoft/diagnosing-instability-in-production-scale-agent-rl/
Integration guide: https://huggingface.co/docs/trl/main/en/ptt_integration
Code: https://github.com/microsoft/post-training-toolkit
🔍 Detects training issues early
🛠 Lets you intervene safely
📊 Keeps long training runs stable, auditable & efficient
Microsoft blog: https://devblogs.microsoft.com/engineering-at-microsoft/diagnosing-instability-in-production-scale-agent-rl/
Integration guide: https://huggingface.co/docs/trl/main/en/ptt_integration
Code: https://github.com/microsoft/post-training-toolkit
sergiopaniego
posted
an
update
13 days ago
Post
2516
New TRL + OpenEnv example! 💥
Fine tune an LLM for playing Sudoku using an RL env via OpenEnv
Includes a script that runs on 1 or multiple GPUs with vLLM, plus a Colab-ready notebook.
Enjoy!
Notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/openenv_sudoku_grpo.ipynb
Script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/sudoku.py
Fine tune an LLM for playing Sudoku using an RL env via OpenEnv
Includes a script that runs on 1 or multiple GPUs with vLLM, plus a Colab-ready notebook.
Enjoy!
Notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/openenv_sudoku_grpo.ipynb
Script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/sudoku.py
- 1 reply
sergiopaniego
posted
an
update
15 days ago
Post
2133
Date idea: read the entire Transformers v5.0.0 release notes
Officially stable now: https://github.com/huggingface/transformers/releases/tag/v5.0.0
Officially stable now: https://github.com/huggingface/transformers/releases/tag/v5.0.0
- 1 reply
Update README.md
#2 opened 19 days ago
by
mandreux-hai
sergiopaniego
posted
an
update
22 days ago
Post
1607
FunctionGemma Tuning Lab is a new no-code tool by
@google
that lets you fine-tune a model directly from the browser, with no coding knowledge required, using TRL behind the scenes.
blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/
try it out: google/functiongemma-tuning-lab
This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma
blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/
try it out: google/functiongemma-tuning-lab
This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma
- 1 reply
sergiopaniego
posted
an
update
25 days ago
Post
811
TRL v0.27.0 is out!! 🥳
It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.
Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)
Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0
It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.
Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)
Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0
antonioloison
authored
a
paper
26 days ago
sergiopaniego
posted
an
update
28 days ago
Post
3009
New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.
Ready for inference & post-training using trajectories. Handles long contexts:
> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result
Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
Used in the Recursive Language Models (RLM) paper by Alex Zhang.
Ready for inference & post-training using trajectories. Handles long contexts:
> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result
Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py
sergiopaniego
posted
an
update
29 days ago
Post
506
Recursive Language Models (RLM) is a new interface for LLMs with cool ideas by Alex Zhang!
⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)
We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282
More resources:
> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm
⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)
We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282
More resources:
> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm
- 2 replies
sergiopaniego
posted
an
update
about 1 month ago
Post
2271
New GRPO + TRL free Colab notebook out! 🔥
Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO
7B model uses only 9.2 GB VRAM (~7× reduction) 🤯
Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb
Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO
7B model uses only 9.2 GB VRAM (~7× reduction) 🤯
Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb
Post
3265
👉 What happened in AI in 2025? 👈
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯
Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
🫡 @reach-vb for the original idea, design and recipe
🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition
🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯
Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
🫡 @reach-vb for the original idea, design and recipe
🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition
🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂
- 2 replies
sergiopaniego
posted
an
update
about 1 month ago
Post
2608
The list of hands-on notebooks (some beginner-friendly!) to get started with fine-tuning using TRL keeps growing!!
• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!
https://github.com/huggingface/trl/tree/main/examples/notebooks
• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!
https://github.com/huggingface/trl/tree/main/examples/notebooks
sergiopaniego
posted
an
update
about 1 month ago
Post
500
As the year comes to an end, it’s a good moment to catch up on some of the best long-form pieces published by the
@huggingface
team.
I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts
I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts
sergiopaniego
posted
an
update
about 1 month ago
Post
2323
This super detailed tutorial by
@Paulescu
is pure gold 🪙 "Fine-tuning a Small Language Model for browser control with GRPO and OpenEnv"
LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝
https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser
LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝
https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser
sergiopaniego
posted
an
update
about 2 months ago
Post
503
if you’re on holidays 🎄 and want some reading, here are blogs I contributed to this year:
🦄 VLMs in TRL: https://huggingface.co/blog/trl-vlm-alignment
🦖 VLMs in 2025: https://huggingface.co/blog/vlms-2025
👾 tokenization in transformers v5: https://huggingface.co/blog/tokenizers
🛸 faster transformers: https://huggingface.co/blog/faster-transformers
🦄 VLMs in TRL: https://huggingface.co/blog/trl-vlm-alignment
🦖 VLMs in 2025: https://huggingface.co/blog/vlms-2025
👾 tokenization in transformers v5: https://huggingface.co/blog/tokenizers
🛸 faster transformers: https://huggingface.co/blog/faster-transformers