Spaces:
Running
Running
| title: README | |
| emoji: 🧪 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| # MatMasterBench: Beyond Knowledge and Reasoning—A Benchmark for Agentic Materials Research Execution | |
| **Open benchmarks and evaluation resources for materials-science Agents.** | |
| MatMaster-DP is maintained by [DP Technology](https://www.dp.tech/en). We develop benchmarks for evaluating how AI Agents reason about materials, construct atomistic models, prepare and analyze scientific calculations, reproduce paper-grounded research workflows, and plan laboratory operations. | |
| ## Public benchmarks | |
| | Benchmark | What is evaluated | Public package | | |
| |---|---|---| | |
| | [MATTER-604](https://huggingface.co/datasets/MatMaster-DP/MATTER-604) | 604 materials-science Agent tasks covering structure construction and retrieval, scientific analysis, simulation input generation, workflow execution, data diagnosis, and research constraints | Task prompts, redistributable input fixtures, submission schema, and evaluation guide | | |
| | [Materials AutoLab 144](https://huggingface.co/datasets/MatMaster-DP/Materials-AutoLab-144) | 144 tasks for materials-experiment planning and laboratory operation through a public OPC interface contract | Task packages, action and capability catalogs, schemas, and evaluation guide | | |
| | [Paper2Arm-5135](https://huggingface.co/datasets/MatMaster-DP/Paper2Arm-5135) | 5,135 paper-grounded task instances across 975 workflow families, evaluating whether an Agent can turn a reproduction objective into code, calculation inputs, analysis procedures, and structured research outputs | Instructions, runtime definitions, paper titles and DOI metadata, submission schema, and evaluation guide | | |
| | [E2EBench-118](https://huggingface.co/datasets/MatMaster-DP/E2EBench-118) | 118 curated end-to-end computational materials-science tasks covering atomistic simulation, electronic structure, thermodynamics, kinetics, mechanics, magnetic and optical models, scientific data analysis, and machine-learning interatomic potentials | Instructions, runtime definitions, solver-visible resources, bibliographic metadata, and evaluation guide | | |
| Each dataset card documents the scientific scope, package structure, and evaluation workflow. Machine-readable task records and public inputs are provided by the corresponding release. | |
| ## Evaluation access | |
| Reference answers and evaluation materials are separated from the public questions to reduce direct answer exposure. Qualified benchmark maintainers may request access through the manual-gated repositories: | |
| - [MATTER-604 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/MATTER-604-evaluator-access) | |
| - [Materials AutoLab 144 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/Materials-AutoLab-144-evaluator-access) | |
| - [Paper2Arm-5135 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/Paper2Arm-5135-evaluator-access) | |
| - [E2EBench-118 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/E2EBench-118-evaluator-access) | |
| Please disclose benchmark exposure, Agent and model versions, prompts, tools, runtime configuration, and any task-specific failures when reporting results. | |
| ## Evaluation result interpretation | |
| See [MatMaster Capability Atlas](http://kpha1548662.bohrium.tech:50001/) | |
| ## Trying MatMaster | |
| MatMaster: https://matmaster.bohrium.com/matmaster/ | |
| ## About DP Technology | |
| [DP Technology](https://www.dp.tech/en) develops AI-for-Science platforms and scientific software for research in materials, chemistry, life sciences, and related fields. | |