File size: 2,323 Bytes
eedd242 729c21c eedd242 729c21c eedd242 729c21c 313111d 729c21c 313111d 729c21c 313111d 729c21c 9b1341d 3c01f42 313111d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | ---
title: README
emoji: 📊
colorFrom: blue
colorTo: indigo
sdk: static
pinned: false
---
# Quantiles
**Open-source, local-first evaluation infrastructure for applied AI systems, designed for developer and coding-agent workflows.**
Quantiles gives developers a CLI and Python SDK for creating, running, analyzing, and comparing AI evaluations from their existing development environment. It records metrics, sample-level outputs, execution traces, and run history locally, making it easier to measure system behavior, detect regressions, validate changes, and ship higher-quality, more reliable AI systems.
With Quantiles, developers can:
- Run built-in benchmarks or create custom evaluations from configuration or Python.
- Debug individual samples with step-level inputs, outputs, metrics, and errors.
- Resume interrupted runs and reuse completed steps.
- Compare models, prompts, datasets, and configurations from the CLI.
- Run evaluation workflows with coding agents using the Quantiles agent skill.
## Quickstart
Install the CLI and run SimpleQA Verified:
```bash
curl -fsSL https://cli.quantiles.io/install.sh | bash
qt run simpleqa-verified
qt show 1 --json
```
This first run uses the built-in demo model, so it requires no model API key and incurs no inference cost. Use it to verify the workflow, then configure a provider-backed model for a real evaluation.
## Documentation & Resources
- [Documentation](https://quantiles.io/documentation) — Everything needed to use Quantiles.
- [Quickstart](https://quantiles.io/documentation/quickstart) — Install Quantiles and run your first evaluation.
- [GitHub](https://github.com/quantiles-evals/quantiles) — Explore the source code and contribute.
- [Agent skill](https://github.com/quantiles-evals/skill) — Run and analyze AI evaluations with a coding agent.
### Articles
- **Aug 4, 2026** | [Build an AI Evaluation from a Hugging Face Dataset Without Writing Python](https://huggingface.co/blog/phranzia/build-an-ai-evaluation-from-a-hugging-face-dataset)
- **Jul 27, 2026** | [Run and Compare AI Evaluations with a CLI for Developers and Coding Agents](https://huggingface.co/blog/phranzia/quantiles-local-ai-evaluation)
Quantiles is available under the [Apache License 2.0](https://github.com/quantiles-evals/quantiles/blob/main/LICENSE). |