README.md

---
dataset_info:
  features:
  - name: task
    dtype: large_string
  - name: prompt
    dtype: large_string
  - name: answer
    dtype: large_string
  - name: metadata
    dtype: large_string
  - name: cot
    dtype: large_string
  - name: level
    dtype: int64
  - name: mode
    dtype: large_string
  splits:
  - name: train
    num_bytes: 7321098048
    num_examples: 3113154
  - name: test
    num_bytes: 74185594
    num_examples: 31445
  download_size: 2599070882
  dataset_size: 7395283642
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: test
    path: data/test-*
license: mit
task_categories:
- question-answering
language:
- en
tags:
- SFT
- reasoning-traces
- logic
- reasoning
- procedural
- formal
- synthetic
- pretraining
- pre-training
- corpus
- formal-pretraining
size_categories:
- 10M<n<100M
---

# Reasoning-Core  : Procedural Pre-Training Pile (PPTP) ◉

PPTP is designed for formal/symbolic pre-training, mid-training and SFT.  
The data is procedurally generated on cpu and can be scaled to trillion tokens, and the difficulty is also adjustable with a single knob.  
Unlike LLM-generated synthetic data, the answers are correct by design.

## Task Categories  
📐 **Formal Reasoning**: planning • conjecture_entailment • proof_reconstruction  
📜 **Formal Semantics, Logic**: logic_nli • evidence_retrieval  
🔢 **Mathematical computation**: equation_system • arithmetics • symbolic_arithmetics • sequential_induction  
💻 **Code & Execution**: code_execution • diff_prediction • diff_patching  
🕸️ **Graph Theory**: graph_pathfinding • graph_node_centrality • graph_cycle_detection • graph_isomorphism  
🎲 **Probabilistic**: bayesian_association • bayesian_intervention  
📝 **Language Parsing, Syntax**: regex_following • regex_induction • parsability • parsing • continuation  
📋 **Table Processing**: table_qa • table_conversion  
🔎 **Set Operations, Retrieval**: set_intersection • set_missing_element • set_equality  

## Task Modes  
We provide three modes for most tasks, all in SFT/pretraining suitable format:  
➡️ **Instruct mode**: Direct prompt/answer format  
🧠 **Trace mode**: Most tasks include reasoning traces to bake-in chain-of-thought reasoning patterns  
✅ **Verification mode**: Tasks framed as prompt/candidate: valid (yes/no)? 10% of the time, to strengthen reasoning self-verification capabilities


🧪 [Paper: Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning](https://huggingface.co/papers/2509.18083)  
📦 [Code: GitHub Repository](https://github.com/sileod/reasoning_core) *(An updated paper for pre-training results is coming.)*


## RLVR version
See [rc1](https://huggingface.co/datasets/reasoning-core/rc1/) for the post-training/RLVR version
 


## Abstract
We introduce Reasoning Core, a new scalable environment for Reinforcement
Learning with Verifiable Rewards (RLVR), designed to advance foundational
symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks
that focus on games or isolated puzzles, Reasoning Core procedurally generates
problems across core formal domains, including PDDL planning, first-order
logic, context-free grammar parsing, causal reasoning, and system equation
solving. The environment is built on key design principles of high-generality
problem distributions, verification via external tools, and continuous
difficulty control, which together provide a virtually infinite supply of novel
training instances. Initial zero-shot evaluations with frontier LLMs confirm
the difficulty of Reasoning Core's tasks, positioning it as a promising
resource to improve the reasoning capabilities of future models.

## Usage
`ds = load_dataset("reasoning-core/symbolic-pretraining-pile")`

# Citation
```
@article{reasoningcore2026,
  title={Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training},
  author={Lacombe, Valentin and Quesnel, Valentin and Sileo, Damien},
  journal={arXiv preprint arXiv:2603.02208},
  year={2026},
  url={https://arxiv.org/abs/2603.02208}
}
```