README.md

metadata

title: SakThai Agentic Eval
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.37.0
short_description: Agentic and tool-use eval for SakThai models.
app_file: app.py

SakThai Agentic Eval

A Gradio Space for evaluating agentic and tool-use capabilities of SakThai models.

Features

• Real-time tool-use evaluation on SakThai models
• Multi-task assessment: dialogue, planning, tool-calling
• Integration with HermesToolEnvironment for standardized evaluation
• ZeroGPU support for cost-free inference

Evaluated Assets

• Models: SakThai Context Family (0.5B, 1.5B, 7B)
• Datasets: SakThai Tool-use and Planning tasks
• Environment: HermesToolEnvironment with 12-step action loops

How to Use

1. Select a task from the dropdown
2. Choose a SakThai model variant
3. Click "Evaluate" to run the agentic loop
4. Review the action trace and performance metrics

Technical Details

Built with Gradio + Transformers + HermesToolEnvironment. Runs on ZeroGPU for zero-cost inference on free tier.