Reasoning-Core : Procedural Pre-Training Pile (PPTP) ◉

PPTP is designed for formal/symbolic pre-training, mid-training and SFT.
The data is procedurally generated on cpu and can be scaled to trillion tokens, and the difficulty is also adjustable with a single knob.
Unlike LLM-generated synthetic data, the answers are correct by design.

Task Categories

📐 Formal Reasoning: planning • conjecture_entailment • proof_reconstruction
📜 Formal Semantics, Logic: logic_nli • evidence_retrieval
🔢 Mathematical computation: equation_system • arithmetics • symbolic_arithmetics • sequential_induction
💻 Code & Execution: code_execution • diff_prediction • diff_patching
🕸️ Graph Theory: graph_pathfinding • graph_node_centrality • graph_cycle_detection • graph_isomorphism
🎲 Probabilistic: bayesian_association • bayesian_intervention
📝 Language Parsing, Syntax: regex_following • regex_induction • parsability • parsing • continuation
📋 Table Processing: table_qa • table_conversion
🔎 Set Operations, Retrieval: set_intersection • set_missing_element • set_equality

Task Modes

We provide three modes for most tasks, all in SFT/pretraining suitable format:
➡️ Instruct mode: Direct prompt/answer format
🧠 Trace mode: Most tasks include reasoning traces to bake-in chain-of-thought reasoning patterns
✅ Verification mode: Tasks framed as prompt/candidate: valid (yes/no)? 10% of the time, to strengthen reasoning self-verification capabilities

🧪 Paper: Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning
📦 Code: GitHub Repository (An updated paper for pre-training results is coming.)

RLVR version

See rc1 for the post-training/RLVR version

Abstract

We introduce Reasoning Core, a new scalable environment for Reinforcement Learning with Verifiable Rewards (RLVR), designed to advance foundational symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks that focus on games or isolated puzzles, Reasoning Core procedurally generates problems across core formal domains, including PDDL planning, first-order logic, context-free grammar parsing, causal reasoning, and system equation solving. The environment is built on key design principles of high-generality problem distributions, verification via external tools, and continuous difficulty control, which together provide a virtually infinite supply of novel training instances. Initial zero-shot evaluations with frontier LLMs confirm the difficulty of Reasoning Core's tasks, positioning it as a promising resource to improve the reasoning capabilities of future models.

Usage

ds = load_dataset("reasoning-core/symbolic-pretraining-pile")

Citation

@article{reasoningcore2026,
  title={Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training},
  author={Lacombe, Valentin and Quesnel, Valentin and Sileo, Damien},
  journal={arXiv preprint arXiv:2603.02208},
  year={2026},
  url={https://arxiv.org/abs/2603.02208}
}