Anonymous Reasoning Math

This repository contains data accompanying an anonymous TMLR submission. It provides 59,520 sampled attempts from four model configurations across five competition-math benchmarks. Each model was sampled 80 times per question.

Contents

The data is stored in a Hugging Face Storage Bucket. Each Parquet file contains the 80 attempts for one model and one question, ordered by seed.

data/<model>/<task>/qNN.parquet

The bucket contains 744 Parquet files totaling about 166.8 GiB. The four configuration keys are Qwen3.6-35B-A3B, gpt-oss-20b_low, gpt-oss-20b_medium, and gpt-oss-20b_high.

Loading

pip install -U datasets huggingface_hub pyarrow

from datasets import load_dataset

data_files = {
    "cmimc_2025": "data/gpt-oss-20b_high/cmimc_2025/*.parquet",
}
dataset = load_dataset(
    "buckets/AnonymizedTMLRSubmission/reasoning-math",
    data_files=data_files,
    split="cmimc_2025",
    streaming=True,
)
record = next(iter(dataset))

Data Format

Math-specific fields include extracted_answer, boxed_answer_raw, has_box, has_valid_box, num_boxes, box_status, and boxed_is_correct.

The tokens struct includes prompt and completion log-probability summaries, realized token arrays, and top-20 candidate distributions. Each candidate contains token, token_id, logprob, and rank. The first completion candidate is the sampled token; remaining candidates are ordered by log probability. Use completion_rank_list for the sampled token's vocabulary rank.

Reproducibility

Rows are ordered by seed within each question. Length-truncated responses are graded incorrect because they do not contain a final boxed answer. Report the truncation policy used in comparisons. Verifier outputs should be treated as measurements with their own uncertainty.

License

The artifact is provided under the MIT License. Problem statements from the underlying competitions remain subject to their applicable terms.

Citation

Citation information will be added after anonymous review.