|
|
--- |
|
|
license: mit |
|
|
task_categories: |
|
|
- question-answering |
|
|
- code-understanding |
|
|
language: |
|
|
- en |
|
|
pretty_name: MiniTorch GraphQA |
|
|
tags: |
|
|
- code reasoning |
|
|
- graph-based QA |
|
|
- symbolic reasoning |
|
|
- minitorch |
|
|
- trm |
|
|
size_categories: |
|
|
- n<1K |
|
|
source_datasets: |
|
|
- original |
|
|
--- |
|
|
|
|
|
# MiniTorch GraphQA |
|
|
|
|
|
A dataset of 100 developer-style questions over the [MiniTorch](https://github.com/atgctg/minitorch) codebase, designed for **structured, graph-based code reasoning**. Each question maps to a precise function or class in the source code and is intended for use with **symbolic reasoning models** like the Tiny Recursion Model (TRM). |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
This dataset is designed to evaluate **lightweight, recursive reasoning agents** that operate over **retrieved subgraphs of code** (e.g., 10-node neighborhoods). It is **not** for code generation or language modeling. |
|
|
|
|
|
## Data Format |
|
|
|
|
|
Each line in `questions.jsonl` is a JSON object: |
|
|
|
|
|
```json |
|
|
{ |
|
|
"id": "minitorch-001", |
|
|
"question": "Which function computes the derivative of scalar addition?", |
|
|
"answer_node": "minitorch.scalar.Scalar.add_back", |
|
|
"module": 1, |
|
|
"reasoning_type": "autodiff" |
|
|
} |
|
|
id: unique identifier |
|
|
question: natural language developer question |
|
|
answer_node: fully qualified name of the ground-truth function/class |
|
|
module: MiniTorch module number (0–6) |
|
|
reasoning_type: category of reasoning required |
|
|
|
|
|
|
|
|
License |
|
|
|
|
|
MIT License. |
|
|
Citation |
|
|
|
|
|
If you use this dataset, please cite: |
|
|
|
|
|
@dataset{minitorch_graphqa_2025, |
|
|
author = {Sarosh Quraishi}, |
|
|
title = {MiniTorch GraphQA: A Symbolic Reasoning Benchmark for Code}, |
|
|
year = {2025}, |
|
|
publisher = {Hugging Face}, |
|
|
url = {https://huggingface.co/datasets/saroshq/minitorch-graphqa} |
|
|
} |
|
|
|
|
|
|
|
|
Acknowledgements |
|
|
|
|
|
MiniTorch: https://github.com/atgctg/minitorch |
|
|
|
