yzhuang commited on
Commit
e3436cb
·
verified ·
1 Parent(s): 04312f0

Add PLE (Physical-Engineering's Last Exam) benchmark score

Browse files

Hi MiniMax team,

This PR adds `.eval_results/ple.yaml` with the [Physical-Engineering's Last Exam (PLE)](https://huggingface.co/datasets/google/physical-engineerings-last-exam) score for `MiniMaxAI/MiniMax-M2`.

- Score: `4.8` / 100 (std 1.8)
- Harness: Pi (the best of the 7 harnesses we ran with this model)
- Task id: `overall-score` (dev split)
- Results date: `2026-10-20`
- Leaderboard: https://ple-bench.com/leaderboard?set=dev
- Paper: https://arxiv.org/abs/2610.30537

PLE tests AI agents on structural, civil, electrical, and mechanical engineering tasks. The score is the mean reward over the 29 public dev tasks, averaged over 3 runs, with the model at its highest reasoning setting (where it has one) and a 6-hour limit per task. Each task runs in an isolated Harbor container with the project's files. A separate verifier grades the deliverables with automated checks, an expert rubric judged by Claude Opus 5.5, or both.

Thank you!

Files changed (1) hide show
  1. .eval_results/ple.yaml +9 -0
.eval_results/ple.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: google/physical-engineerings-last-exam
3
+ task_id: overall-score
4
+ value: 4.8
5
+ date: "2026-10-20"
6
+ source:
7
+ url: https://ple-bench.com/leaderboard?set=dev
8
+ name: PLE leaderboard
9
+ notes: "Harness: Pi. Mean of 3 runs on the 29 dev tasks (std 1.8)."