README.md

metadata

dataset_info:
  features:
    - name: task
      dtype: large_string
    - name: prompt
      dtype: large_string
    - name: answer
      dtype: large_string
    - name: metadata
      dtype: large_string
    - name: cot
      dtype: large_string
    - name: level
      dtype: int64
    - name: mode
      dtype: large_string
  splits:
    - name: train
      num_bytes: 7321098048
      num_examples: 3113154
    - name: test
      num_bytes: 74185594
      num_examples: 31445
  download_size: 2599070882
  dataset_size: 7395283642
configs:
  - config_name: default
    data_files:
      - split: train
        path: data/train-*
      - split: test
        path: data/test-*
license: mit
task_categories:
  - question-answering
language:
  - en
tags:
  - SFT
  - reasoning-traces
  - logic
  - reasoning
  - procedural
  - formal
  - synthetic
  - pretraining
  - pre-training
  - corpus
  - formal-pretraining
size_categories:
  - 10M<n<100M

Reasoning-Core : Procedural Pre-Training Pile (PPTP) ◉

PPTP is designed for formal/symbolic pre-training, mid-training and SFT.
The data is procedurally generated on cpu and can be scaled to trillion tokens, and the difficulty is also adjustable with a single knob.
Unlike LLM-generated synthetic data, the answers are correct by design.

Task Categories

📐 Formal Reasoning: planning • conjecture_entailment • proof_reconstruction
📜 Formal Semantics, Logic: logic_nli • evidence_retrieval
🔢 Mathematical computation: equation_system • arithmetics • symbolic_arithmetics • sequential_induction
💻 Code & Execution: code_execution • diff_prediction • diff_patching
🕸️ Graph Theory: graph_pathfinding • graph_node_centrality • graph_cycle_detection • graph_isomorphism
🎲 Probabilistic: bayesian_association • bayesian_intervention
📝 Language Parsing, Syntax: regex_following • regex_induction • parsability • parsing • continuation
📋 Table Processing: table_qa • table_conversion
🔎 Set Operations, Retrieval: set_intersection • set_missing_element • set_equality

Task Modes

We provide three modes for most tasks, all in SFT/pretraining suitable format:
➡️ Instruct mode: Direct prompt/answer format
🧠 Trace mode: Most tasks include reasoning traces to bake-in chain-of-thought reasoning patterns
✅ Verification mode: Tasks framed as prompt/candidate: valid (yes/no)? 10% of the time, to strengthen reasoning self-verification capabilities

🧪 Paper: Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
📦 Code: GitHub Repository (An updated paper for pre-training results is coming.)

RLVR version

See rc1 for the post-training/RLVR version

Abstract

We introduce Reasoning Core, a new scalable environment for Reinforcement Learning with Verifiable Rewards (RLVR), designed to advance foundational symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks that focus on games or isolated puzzles, Reasoning Core procedurally generates problems across core formal domains, including PDDL planning, first-order logic, context-free grammar parsing, causal reasoning, and system equation solving. The environment is built on key design principles of high-generality problem distributions, verification via external tools, and continuous difficulty control, which together provide a virtually infinite supply of novel training instances. Initial zero-shot evaluations with frontier LLMs confirm the difficulty of Reasoning Core's tasks, positioning it as a promising resource to improve the reasoning capabilities of future models.

Usage

ds = load_dataset("reasoning-core/symbolic-pretraining-pile")

Citation

@article{reasoningcore2026,
  title={Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training},
  author={Lacombe, Valentin and Quesnel, Valentin and Sileo, Damien},
  journal={arXiv preprint arXiv:2603.02208},
  year={2026},
  url={https://arxiv.org/abs/2603.02208}
}