Models That Know How Evaluations Are Designed Score Safer
Paper • 2605.28591 • Published • 8
AI & ML interests
Hackathon de PLN en español open-source y enfocado a los Objetivos de Desarrollo Sostenible de la ONU. Organizado por Somos NLP y patrocinado por Platzi, Paperspace y Hugging Face.
haritzpuerto
authored a
paper 9 days ago
haritzpuerto
submitted a
paper to Daily Papers 10 days ago
espejelomar
posted an update 19 days ago
Post
4720
Sharing WorldForge with @abdelstark
It's an open-source Python project for evaluating and replaying robotics and world-model workflows.
The useful part is not only calling a model. WorldForge records the run, validates action shapes, translates outputs into actions, and keeps replay artifacts you can inspect later.
The current demo uses LeRobot + LeWorldModel on PushT through the official loader:
The harness also has replay-only paths for Cosmos-Policy and GR00T-style outputs, so you can inspect the provider contract from saved artifacts without keeping a GPU server online.
Try it:
Repo: https://github.com/AbdelStark/worldforge
Docs: https://abdelstark.github.io/worldforge/
Pre-1.0, MIT, and actively looking for contributors. Good areas:
- robotics provider adapters
- replay artifacts
- eval flows
- docs & first-run demos
Good first issues: https://github.com/AbdelStark/worldforge/contribute
If you're building robot policy evals or model adapters, would love a PR — or an issue describing what's missing.
It's an open-source Python project for evaluating and replaying robotics and world-model workflows.
The useful part is not only calling a model. WorldForge records the run, validates action shapes, translates outputs into actions, and keeps replay artifacts you can inspect later.
The current demo uses LeRobot + LeWorldModel on PushT through the official loader:
stable_worldmodel.policy.AutoCostModel("pusht/lewm")The harness also has replay-only paths for Cosmos-Policy and GR00T-style outputs, so you can inspect the provider contract from saved artifacts without keeping a GPU server online.
Try it:
pip install worldforge-aiuv run --extra harness worldforge-harness --flow robotics-compareRepo: https://github.com/AbdelStark/worldforge
Docs: https://abdelstark.github.io/worldforge/
Pre-1.0, MIT, and actively looking for contributors. Good areas:
- robotics provider adapters
- replay artifacts
- eval flows
- docs & first-run demos
Good first issues: https://github.com/AbdelStark/worldforge/contribute
If you're building robot policy evals or model adapters, would love a PR — or an issue describing what's missing.
haritzpuerto
authored a
paper 3 months ago
albertvillanova
posted an update 3 months ago
Post
2864
🚀 TRL v0.29.0 introduces trl-training: an agent-native training skill.
This makes the TRL CLI a structured, agent-readable capability, allowing AI agents to reliably execute training workflows such as:
- Supervised Fine-Tuning (SFT)
- Direct Preference Optimization (DPO)
- Group Relative Policy Optimization (GRPO)
We’re excited to see what the community builds on top of this.
If you’re working on AI agents, alignment research, or scalable RL training infrastructure: give TRL v0.29.0 a try! 🤗
The future of ML tooling is agent-native.
🔗 https://github.com/huggingface/trl/releases/tag/v0.29.0
This makes the TRL CLI a structured, agent-readable capability, allowing AI agents to reliably execute training workflows such as:
- Supervised Fine-Tuning (SFT)
- Direct Preference Optimization (DPO)
- Group Relative Policy Optimization (GRPO)
We’re excited to see what the community builds on top of this.
If you’re working on AI agents, alignment research, or scalable RL training infrastructure: give TRL v0.29.0 a try! 🤗
The future of ML tooling is agent-native.
🔗 https://github.com/huggingface/trl/releases/tag/v0.29.0
mariagrandury
authored 2
papers 4 months ago
albertvillanova
posted an update 4 months ago
Post
1997
5 years already working in democratizing AI 🤗
Grateful to be part of such an awesome team making it happen every day.
Grateful to be part of such an awesome team making it happen every day.
Post
5123
👉 What happened in AI in 2025? 👈
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯
Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
🫡 @reach-vb for the original idea, design and recipe
🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition
🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂
We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!
Play with it here:
2025-ai-timeline/2025-ai-timeline
Here's my personal quarterly TL;DR:
1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.
Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)
2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.
Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4
3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.
Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5
4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!
Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯
Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline
🫡 @reach-vb for the original idea, design and recipe
🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition
🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂
- 3 replies
osanseviero
authored a
paper 8 months ago
mariagrandury
authored 4
papers 9 months ago
Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
Paper • 2509.14405 • Published • 2
Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans
Paper • 2506.22439 • Published • 3
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Paper • 2509.14233 • Published • 20
La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America
Paper • 2507.00999 • Published • 1
albertvillanova
posted an update 10 months ago
Post
4629
Latest smolagents release supports GPT-5: build agents that think, plan, and act.
⚡ Upgrade now and put GPT-5 to work!
⚡ Upgrade now and put GPT-5 to work!
- 1 reply
albertvillanova
posted an update 10 months ago
Post
722
🚀 smolagents v1.21.0 is here!
Now with improved safety in the local Python executor: dunder calls are blocked!
⚠️ Still, not fully isolated: for untrusted code, use a remote executor instead: Docker, E2B, Wasm.
✨ Many bug fixes: more reliable code.
👉 https://github.com/huggingface/smolagents/releases/tag/v1.21.0
Now with improved safety in the local Python executor: dunder calls are blocked!
⚠️ Still, not fully isolated: for untrusted code, use a remote executor instead: Docker, E2B, Wasm.
✨ Many bug fixes: more reliable code.
👉 https://github.com/huggingface/smolagents/releases/tag/v1.21.0
haritzpuerto
authored 2
papers 11 months ago
albertvillanova
posted an update 11 months ago
Post
843
🚀 New in smolagents v1.20.0: Remote Python Execution via WebAssembly (Wasm)
We've just merged a major new capability into the smolagents framework: the CodeAgent can now execute Python code remotely in a secure, sandboxed WebAssembly environment!
🔧 Powered by Pyodide and Deno, this new WasmExecutor lets your agent-generated Python code run safely: without relying on Docker or local execution.
Why this matters:
✅ Isolated execution = no host access
✅ No need for Python on the user's machine
✅ Safer evaluation of arbitrary code
✅ Compatible with serverless / edge agent workloads
✅ Ideal for constrained or untrusted environments
This is just the beginning: a focused initial implementation with known limitations. A solid MVP designed for secure, sandboxed use cases. 💡
💡 We're inviting the open-source community to help evolve this executor:
• Tackle more advanced Python features
• Expand compatibility
• Add test coverage
• Shape the next-gen secure agent runtime
🔗 Check out the PR: https://github.com/huggingface/smolagents/pull/1261
Let's reimagine what agent-driven Python execution can look like: remote-first, wasm-secure, and community-built.
This feature is live in smolagents v1.20.0!
Try it out.
Break things. Extend it. Give us feedback.
Let's build safer, smarter agents; together 🧠⚙️
👉 https://github.com/huggingface/smolagents/releases/tag/v1.20.0
#smolagents #WebAssembly #Python #AIagents #Pyodide #Deno #OpenSource #HuggingFace #AgenticAI
We've just merged a major new capability into the smolagents framework: the CodeAgent can now execute Python code remotely in a secure, sandboxed WebAssembly environment!
🔧 Powered by Pyodide and Deno, this new WasmExecutor lets your agent-generated Python code run safely: without relying on Docker or local execution.
Why this matters:
✅ Isolated execution = no host access
✅ No need for Python on the user's machine
✅ Safer evaluation of arbitrary code
✅ Compatible with serverless / edge agent workloads
✅ Ideal for constrained or untrusted environments
This is just the beginning: a focused initial implementation with known limitations. A solid MVP designed for secure, sandboxed use cases. 💡
💡 We're inviting the open-source community to help evolve this executor:
• Tackle more advanced Python features
• Expand compatibility
• Add test coverage
• Shape the next-gen secure agent runtime
🔗 Check out the PR: https://github.com/huggingface/smolagents/pull/1261
Let's reimagine what agent-driven Python execution can look like: remote-first, wasm-secure, and community-built.
This feature is live in smolagents v1.20.0!
Try it out.
Break things. Extend it. Give us feedback.
Let's build safer, smarter agents; together 🧠⚙️
👉 https://github.com/huggingface/smolagents/releases/tag/v1.20.0
#smolagents #WebAssembly #Python #AIagents #Pyodide #Deno #OpenSource #HuggingFace #AgenticAI
not working
1
#1 opened 11 months ago
by 
andresadmim