Prime Agent: A Self-Improving RLM Harness
Paper โข 2608.23552 โข Published โข 58
AI & ML interests
None defined yet.
eliebakย
authored 2
papers about 2 months ago
alvarobarttย
posted an update 5 months ago
Post
2191
Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!
> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio
https://alvarobartt.com/agents-on-aws-sagemaker
> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio
https://alvarobartt.com/agents-on-aws-sagemaker
alvarobarttย
posted an update 5 months ago
Post
4693
Latest
TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.
๐ง
๐๏ธ Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
โก Active params isn't the same as memory footprint, especially for sparse architectures
๐ฆ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
๐ KV cache can still dominate depending on context length, batch size, and concurrency
๐ Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
๐ Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving
Check the repository at https://github.com/alvarobartt/hf-mem
hf-mem release added a breakdown of Mixture-of-Experts (MoE) memory usage!TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.
๐ง
hf-mem now splits MoE memory into base model weights, routed experts, and KV cache๐๏ธ Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
โก Active params isn't the same as memory footprint, especially for sparse architectures
๐ฆ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
๐ KV cache can still dominate depending on context length, batch size, and concurrency
๐ Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
๐ Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving
Check the repository at https://github.com/alvarobartt/hf-mem
alvarobarttย
posted an update 7 months ago
Post
4610
Learn how to deploy Microsoft Research VibeVoice ASR on Microsoft Azure Foundry with Hugging Face to generate rich audio transcriptions with Who, When, and What! ๐ฅ
> ๐ 60-minute single-pass processing, no chunking or stitching
> ๐ค Customized hotwords to guide recognition on domain-specific content
> ๐ Rich transcription: joint ASR + diarization + timestamping in one pass
> ๐ 50+ languages with automatic detection and code-switching support
> ๐ค Deployed on Microsoft Foundry via an OpenAI-compatible Chat Completions API
https://huggingface.co/docs/microsoft-azure/foundry/examples/deploy-vibevoice-asr
> ๐ 60-minute single-pass processing, no chunking or stitching
> ๐ค Customized hotwords to guide recognition on domain-specific content
> ๐ Rich transcription: joint ASR + diarization + timestamping in one pass
> ๐ 50+ languages with automatic detection and code-switching support
> ๐ค Deployed on Microsoft Foundry via an OpenAI-compatible Chat Completions API
https://huggingface.co/docs/microsoft-azure/foundry/examples/deploy-vibevoice-asr
mryabย
submitted a
paper to Daily Papers 8 months ago
lewtunย
submitted 2
papers to Daily Papers 8 months ago
alvarobarttย
posted an update 9 months ago
Post
3847
๐ฅ
๐ก Alternatively, you can also set the
hf-mem v0.4.1 now also estimates KV cache memory requirements for any context length and batch size with the --experimental flag!uvx hf-mem --model-id ... --experimental will automatically pull the required information from the Hugging Face Hub to include the KV cache estimation, when applicable.๐ก Alternatively, you can also set the
--max-model-len, --batch-size and --kv-cache-dtype arguments (ร la vLLM) manually if preferred. - 1 reply
Post
7851
๐ What happened in AI in 2025? ๐
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1๏ธโฃ Q1 โ Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2๏ธโฃ Q2 โ Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3๏ธโฃ Q3 โ "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4๏ธโฃ Q4 โ Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 ๐คฏ
Credits
๐ NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
๐ซก @reach-vb for the original idea, design and recipe
๐ @ariG23498 and yours truly for compiling and verifying the 2025 edition
๐ฅณ Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! ๐ฅ
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1๏ธโฃ Q1 โ Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2๏ธโฃ Q2 โ Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3๏ธโฃ Q3 โ "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4๏ธโฃ Q4 โ Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 ๐คฏ
Credits
๐ NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
๐ซก @reach-vb for the original idea, design and recipe
๐ @ariG23498 and yours truly for compiling and verifying the 2025 edition
๐ฅณ Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! ๐ฅ
- 3 replies
eliebakย
submitted a
paper to Daily Papers 10 months ago
alozowskiย
authored a
paper 10 months ago
Post
4738
Super excited to announce that our research team at Hugging Face will be doing an AMA on reddit r/LocalLLaMA.
Come ask any questions to the team behind SmolLM, FineWeb and more! And who knows, maybe thereโll be a shiny new release to talk about?
Thursday 4th September, 8AM-11AM PST ๐ค
science
Come ask any questions to the team behind SmolLM, FineWeb and more! And who knows, maybe thereโll be a shiny new release to talk about?
Thursday 4th September, 8AM-11AM PST ๐ค
Post
799
Motif 2.6B tech report is pretty insane, first time i see a model with differential attention and polynorm trained at scale!
> It's trained on 2.5T of token, with a "data mixture schedule" to continuously adjust the mixture over training.
> They use WSD with a "Simple moving average" averaging the last 6 ckpt every 8B token.
> They trained on Finemath, Fineweb2, DCLM, TxT360.
> Lot of details in the finetuning data they used, for instance they used EvolKit and did some "dataset fusion" to have more compressed knowledge into the data.
> They mention they also tried Normalized GPT, QK-Norm and Cross Layer Attention.
Motif-Technologies/Motif-2.6B
> It's trained on 2.5T of token, with a "data mixture schedule" to continuously adjust the mixture over training.
> They use WSD with a "Simple moving average" averaging the last 6 ckpt every 8B token.
> They trained on Finemath, Fineweb2, DCLM, TxT360.
> Lot of details in the finetuning data they used, for instance they used EvolKit and did some "dataset fusion" to have more compressed knowledge into the data.
> They mention they also tried Normalized GPT, QK-Norm and Cross Layer Attention.
Motif-Technologies/Motif-2.6B
Post
31717
Okay this is insane... WebGPU-accelerated semantic video tracking, powered by DINOv3 and Transformers.js! ๐คฏ
Demo (+ source code): webml-community/DINOv3-video-tracking
This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! ๐
How does it work? ๐ค
1๏ธโฃ Generate and cache image features for each frame
2๏ธโฃ Create a list of embeddings for selected patch(es)
3๏ธโฃ Compute cosine similarity between each patch and the selected patch(es)
4๏ธโฃ Highlight those whose score is above some threshold
... et voilร ! ๐ฅณ
You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.
Excited to see what the community builds with it!
Demo (+ source code): webml-community/DINOv3-video-tracking
This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! ๐
How does it work? ๐ค
1๏ธโฃ Generate and cache image features for each frame
2๏ธโฃ Create a list of embeddings for selected patch(es)
3๏ธโฃ Compute cosine similarity between each patch and the selected patch(es)
4๏ธโฃ Highlight those whose score is above some threshold
... et voilร ! ๐ฅณ
You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.
Excited to see what the community builds with it!
- 4 replies
Post
5258
The next generation of AI-powered websites is going to be WILD! ๐คฏ
In-browser tool calling & MCP is finally here, allowing LLMs to interact with websites programmatically.
To show what's possible, I built a demo using Liquid AI's new LFM2 model, powered by ๐ค Transformers.js: LiquidAI/LFM2-WebGPU
As always, the demo is open source (which you can find under the "Files" tab), so I'm excited to see how the community builds upon this! ๐
In-browser tool calling & MCP is finally here, allowing LLMs to interact with websites programmatically.
To show what's possible, I built a demo using Liquid AI's new LFM2 model, powered by ๐ค Transformers.js: LiquidAI/LFM2-WebGPU
As always, the demo is open source (which you can find under the "Files" tab), so I'm excited to see how the community builds upon this! ๐
- 2 replies
Post
3967
Introducing Voxtral WebGPU: State-of-the-art audio transcription directly in your browser! ๐คฏ
๐ฃ๏ธ Transcribe videos, meeting notes, songs and more
๐ Runs on-device, meaning no data is sent to a server
๐ Multilingual (8 languages)
๐ค Completely free (forever) & open source
That's right, we're running Mistral's new Voxtral-Mini-3B model 100% locally in-browser on WebGPU, powered by Transformers.js and ONNX Runtime Web! ๐ฅ
Try it out yourself! ๐
webml-community/Voxtral-WebGPU
๐ฃ๏ธ Transcribe videos, meeting notes, songs and more
๐ Runs on-device, meaning no data is sent to a server
๐ Multilingual (8 languages)
๐ค Completely free (forever) & open source
That's right, we're running Mistral's new Voxtral-Mini-3B model 100% locally in-browser on WebGPU, powered by Transformers.js and ONNX Runtime Web! ๐ฅ
Try it out yourself! ๐
webml-community/Voxtral-WebGPU
Post
4867
Kimi K2 tech report is full of gems as always. Here are my notes on it:
> MuonClip: Pretty crazy how after 70k the training stabilizes and the QK-clip is basically inactive. There is also no loss in perf with QK-clip which is not trivial at all (at small scale but with aggressive threshold). Also a cool explanation of why muon makes the logit explode in appendix E (tl;dr is that muon makes the singular value of the update matrix higher)
> Sparsity scaling laws to justify their ratio, they have a very solid training infra that allows the model to be trained at this sparsity level, they could have increased even more but as sparsity increases the training becomes less efficient.
> They diminish the number of attention heads to make it more efficient for long context since attention heads are a big bottleneck for long context. They also remove 2 of the 3 "first dense" layers in the dsv3 arch.
With the sparsity and attention heads (divided by 2) they achieve 83% increased flops compared to deepseek v3 arch at 128k.
> Data: Rephrasing is KEY. They do a lot more synthetic data generation and rephrase their corpus to have different styles, for longer documents they do it by chunk. I'm (half) surprised by the fact that ONLY 1 epoch (assuming same number of training tokens I think?) of data rephrased 10 times has better accuracy than 10 epochs of the same data rephrased once.
> They do rewriting for Math and Knowledge, for Math they apply the ShallowMath recipe and instruct the model to rephrase in a "learning note" style
> They talk about diversity and probably have some internal stuff/eval to test that, as always still a bit unclear for me how to properly measure that.
The infra is also very nice, quick summary:
> PP=16 (1F1B schedule, a bit custom), EP=16, zero1
> No FP8 computation but for storage of specific layers, selective recomputation for inexpensive block, activation offloading to CPU
> MuonClip: Pretty crazy how after 70k the training stabilizes and the QK-clip is basically inactive. There is also no loss in perf with QK-clip which is not trivial at all (at small scale but with aggressive threshold). Also a cool explanation of why muon makes the logit explode in appendix E (tl;dr is that muon makes the singular value of the update matrix higher)
> Sparsity scaling laws to justify their ratio, they have a very solid training infra that allows the model to be trained at this sparsity level, they could have increased even more but as sparsity increases the training becomes less efficient.
> They diminish the number of attention heads to make it more efficient for long context since attention heads are a big bottleneck for long context. They also remove 2 of the 3 "first dense" layers in the dsv3 arch.
With the sparsity and attention heads (divided by 2) they achieve 83% increased flops compared to deepseek v3 arch at 128k.
> Data: Rephrasing is KEY. They do a lot more synthetic data generation and rephrase their corpus to have different styles, for longer documents they do it by chunk. I'm (half) surprised by the fact that ONLY 1 epoch (assuming same number of training tokens I think?) of data rephrased 10 times has better accuracy than 10 epochs of the same data rephrased once.
> They do rewriting for Math and Knowledge, for Math they apply the ShallowMath recipe and instruct the model to rephrase in a "learning note" style
> They talk about diversity and probably have some internal stuff/eval to test that, as always still a bit unclear for me how to properly measure that.
The infra is also very nice, quick summary:
> PP=16 (1F1B schedule, a bit custom), EP=16, zero1
> No FP8 computation but for storage of specific layers, selective recomputation for inexpensive block, activation offloading to CPU
eliebakย
authored a
paper over 1 year ago