README.md

---
title: README
emoji: 🧪
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---

# MatMasterBench: Beyond Knowledge and Reasoning—A Benchmark for Agentic Materials Research Execution

**Open benchmarks and evaluation resources for materials-science Agents.**

MatMaster-DP is maintained by [DP Technology](https://www.dp.tech/en). We develop benchmarks for evaluating how AI Agents reason about materials, construct atomistic models, prepare and analyze scientific calculations, reproduce paper-grounded research workflows, and plan laboratory operations.

## Public benchmarks

| Benchmark | What is evaluated | Public package |
|---|---|---|
| [MATTER-604](https://huggingface.co/datasets/MatMaster-DP/MATTER-604) | 604 materials-science Agent tasks covering structure construction and retrieval, scientific analysis, simulation input generation, workflow execution, data diagnosis, and research constraints | Task prompts, redistributable input fixtures, submission schema, and evaluation guide |
| [Materials AutoLab 144](https://huggingface.co/datasets/MatMaster-DP/Materials-AutoLab-144) | 144 tasks for materials-experiment planning and laboratory operation through a public OPC interface contract | Task packages, action and capability catalogs, schemas, and evaluation guide |
| [Paper2Arm-5135](https://huggingface.co/datasets/MatMaster-DP/Paper2Arm-5135) | 5,135 paper-grounded task instances across 975 workflow families, evaluating whether an Agent can turn a reproduction objective into code, calculation inputs, analysis procedures, and structured research outputs | Instructions, runtime definitions, paper titles and DOI metadata, submission schema, and evaluation guide |
| [E2EBench-118](https://huggingface.co/datasets/MatMaster-DP/E2EBench-118) | 118 curated end-to-end computational materials-science tasks covering atomistic simulation, electronic structure, thermodynamics, kinetics, mechanics, magnetic and optical models, scientific data analysis, and machine-learning interatomic potentials | Instructions, runtime definitions, solver-visible resources, bibliographic metadata, and evaluation guide |

Each dataset card documents the scientific scope, package structure, and evaluation workflow. Machine-readable task records and public inputs are provided by the corresponding release.

## Evaluation access

Reference answers and evaluation materials are separated from the public questions to reduce direct answer exposure. Qualified benchmark maintainers may request access through the manual-gated repositories:

- [MATTER-604 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/MATTER-604-evaluator-access)
- [Materials AutoLab 144 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/Materials-AutoLab-144-evaluator-access)
- [Paper2Arm-5135 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/Paper2Arm-5135-evaluator-access)
- [E2EBench-118 Evaluator Access](https://huggingface.co/datasets/MatMaster-DP/E2EBench-118-evaluator-access)

Please disclose benchmark exposure, Agent and model versions, prompts, tools, runtime configuration, and any task-specific failures when reporting results.

## Evaluation result interpretation
See [MatMaster Capability Atlas](http://kpha1548662.bohrium.tech:50001/)

## Trying MatMaster
MatMaster: https://matmaster.bohrium.com/matmaster/

## About DP Technology

[DP Technology](https://www.dp.tech/en) develops AI-for-Science platforms and scientific software for research in materials, chemistry, life sciences, and related fields.