---
license: cc-by-nc-sa-4.0
---

# ChronoQA

ChronoQA is a **passage-grounded** benchmark that tests whether retrieval-augmented generation (RAG) systems can keep **temporal** and **causal** facts straight when reading long-form narratives (novels, scripts, etc.).  
Instead of giving the entire book to the model, ChronoQA forces a RAG pipeline to *retrieve the right snippets* and reason about evolving characters and event sequences.

|                               |                                    |
|-------------------------------|------------------------------------|
| **Instances**                 | 1,028 question–answer pairs        |
| **Narratives**                | 18 public-domain stories           |
| **Reasoning facets**          | 8 (causal, character, setting, …)  |
| **Evidence**                  | Exact byte-offsets for each answer |
| **Language**                  | English                            |
| **Intended use**              | Evaluate/train RAG systems that need chronology & causality |
| **License (annotations)**     | CC-BY-NC-SA-4.0                    |

---

## Dataset Description

### Motivation  
Standard RAG pipelines often lose chronological order and collapse every mention of an entity into a single node. ChronoQA highlights the failures that follow. Example:

*"Who was jinxing Harry's broom during his **first** Quidditch match?"* – a system that only retrieves early chapters may wrongly answer *Snape* instead of *Quirrell*.

### Source Stories  
All texts come from Project Gutenberg (public domain in the US).

| ID | Title | # Q |
|----|-------|----|
| 1 | *A Study in Scarlet* | 67 |
| 2 | *The Hound of the Baskervilles* | 55 |
| 3 | *Harry Potter and the Chamber of Secrets* | 30 |
| 4 | *Harry Potter and the Sorcerer's Stone* | 25 |
| 5 | *Les Misérables* | 72 |
| 6 | *The Phantom of the Opera* | 70 |
| 7 | *The Sign of the Four* | 62 |
| 8 | *The Wonderful Wizard of Oz* | 82 |
| 9 | *The Adventures of Sherlock Holmes* | 34 |
| 10 | *Lady Susan* | 88 |
| 11 | *Dangerous Connections* | 111 |
| 12 | *The Picture of Dorian Gray* | 27 |
| 13 | *The Diary of a Nobody* | 39 |
| 14 | *The Sorrows of Young Werther* | 58 |
| 15 | *The Mysterious Affair at Styles* | 69 |
| 16 | *Pride and Prejudice* | 54 |
| 17 | *The Secret Garden* | 61 |
| 18 | *Anne of Green Gables* | 24 |

### Reasoning Facets  
1. **Causal Consistency**  
2. **Character & Behavioural Consistency**  
3. **Setting, Environment & Atmosphere**  
4. **Symbolism, Imagery & Motifs**  
5. **Thematic, Philosophical & Moral**  
6. **Narrative & Plot Structure**  
7. **Social, Cultural & Political**  
8. **Emotional & Psychological**

---

## Dataset Structure

| Field | Type | Description |
|-------|------|-------------|
| `story_id`     | `string`  | ID of the narrative |
| `question_id`  | `int32`   | QA index within that story |
| `category`     | `string`  | One of the 8 reasoning facets |
| `query`        | `string`  | Natural-language question |
| `ground_truth` | `string`  | Gold answer |
| `passages`     | **`sequence` of objects** | Each object contains: <br> • `start_sentence` `string` <br> • `end_sentence` `string` <br> • `start_byte` `int32` <br> • `end_byte` `int32` <br> • `excerpt` `string` |
| `story_title`* | `string`  | Human-readable title (optional, present in processed splits) |

\*The raw JSONL released with the paper does **not** include `story_title`; it is added automatically in the hosted HF dataset for convenience.

There is a single **all** split (1,028 rows). Create your own train/validation/test splits if needed (e.g. by story or by reasoning facet).

---

## Usage Example

```python
from datasets import load_dataset

ds = load_dataset("your-org/chronoqa", split="all")
example = ds[0]

print("Question:", example["query"])
print("Answer  :", example["ground_truth"])
print("Evidence:", example["passages"][0]["excerpt"][:300], "…")
```

## Citation Information
```
@article{zhang2025respecting,
  title={Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation},
  author={Zhang, Ze Yu and Li, Zitao and Li, Yaliang and Ding, Bolin and Low, Bryan Kian Hsiang},
  journal={arXiv preprint arXiv:2506.05939},
  year={2025}
}
```