File size: 2,761 Bytes
b420f53 89dfdc3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | ---
title: README
emoji: 📈
colorFrom: red
colorTo: pink
sdk: static
pinned: false
---
# SimReal
**Make the world's work checkable.**
AI agents are getting good at passing tests. We build the places where they have to do the work instead: realistic environments where an agent acts, the world responds, and real outcomes decide the score.
Every benchmark we release follows the same rules. The verifier is a program, not a judge model. The scored data is never shown to the agent. And wherever possible, the bar is a real human result, not an estimate.
## Benchmarks
| | What the agent does | How it's scored |
|---|---|---|
| **[Xitadel-QuantBench](https://huggingface.co/spaces/SimrealAI/xitadel-quantbench-leaderboard)** · [GitHub](https://github.com/Simreal-AI/Xitadel-QuantBench) | Researches historical order books, writes a Python trading strategy and ships it | Replayed on an unseen market day against the best human competition strategy that day: P&L, drawdown, Sharpe |
| **[Hard Puzzle Benchmark](https://huggingface.co/datasets/SimrealAI/hard-puzzle-benchmark)** · [GitHub](https://github.com/Simreal-AI/hard-puzzle-benchmark) | Solves 749 hard, human-written puzzles across 15 topics, many with images | Exact answers on a 707-item core, plus a separate proof-certificate track |
| **MathmoBench** · [GitHub](https://github.com/Simreal-AI/MathmoBench) | Solves families of near-identical combinatorics problems where one small edit flips the answer | Machine-checkable certificates: a witness if feasible, an impossibility proof if not |
| **ML Benchmark** · [GitHub](https://github.com/Simreal-AI/Simreal-MLBench) | Runs a full ML research loop across 60 tasks, from tabular data to vision and audio | Test labels stay with the competition platform, so the scorer is not us |
## Latest result
On Xitadel-QuantBench, **no model has cleared the human bar overall**. The best model scores 77.28 against a human reference of 80, under a shortened pilot research budget. [See the leaderboard →](https://huggingface.co/spaces/SimrealAI/xitadel-quantbench-leaderboard)
## Work with us
- **Evaluate your agent.** Research teams can request an official evaluation. Results go to you first and are published only with your consent. → [business@simreal.co](mailto:business@simreal.co?subject=Evaluate%20my%20agent)
- **Partner with us.** Labs and teams can get full access to our environments. → [business@simreal.co](mailto:business@simreal.co)
- **Join us.** We work with researchers, engineers and domain experts. → [simreal.co/experts](https://simreal.co/experts)
[Website](https://simreal.co) · [GitHub](https://github.com/Simreal-AI) · [X](https://x.com/simrealhq) · [LinkedIn](https://www.linkedin.com/company/simreal-ai)
|