pinned
Running
Xitadel-QuantBench Leaderboard
📈
Trading benchmark vs. best human competition strategy
AI & ML interests
RL environments, verifiers, agent evaluation, benchmarks, reinforcement learning, LLM agents, quantitative finance
Recent Activity
Organization Card
SimReal
Make the world's work checkable.
AI agents are getting good at passing tests. We build the places where they have to do the work instead: realistic environments where an agent acts, the world responds, and real outcomes decide the score.
Every benchmark we release follows the same rules. The verifier is a program, not a judge model. The scored data is never shown to the agent. And wherever possible, the bar is a real human result, not an estimate.
Benchmarks
| What the agent does | How it's scored | |
|---|---|---|
| Xitadel-QuantBench · GitHub | Researches historical order books, writes a Python trading strategy and ships it | Replayed on an unseen market day against the best human competition strategy that day: P&L, drawdown, Sharpe |
| Hard Puzzle Benchmark · GitHub | Solves 749 hard, human-written puzzles across 15 topics, many with images | Exact answers on a 707-item core, plus a separate proof-certificate track |
| MathmoBench · GitHub | Solves families of near-identical combinatorics problems where one small edit flips the answer | Machine-checkable certificates: a witness if feasible, an impossibility proof if not |
| ML Benchmark · GitHub | Runs a full ML research loop across 60 tasks, from tabular data to vision and audio | Test labels stay with the competition platform, so the scorer is not us |
Latest result
On Xitadel-QuantBench, no model has cleared the human bar overall. The best model scores 77.28 against a human reference of 80, under a shortened pilot research budget. See the leaderboard →
Work with us
- Evaluate your agent. Research teams can request an official evaluation. Results go to you first and are published only with your consent. → business@simreal.co
- Partner with us. Labs and teams can get full access to our environments. → business@simreal.co
- Join us. We work with researchers, engineers and domain experts. → simreal.co/experts
models 0
None public yet