| # Hugging Face data provenance |
|
|
| - Dataset: rajpurkar/squad (SQuAD 1.1). |
| - Dataset URL: https://huggingface.co/datasets/rajpurkar/squad |
| - Split: train. |
| - Revision: 7b6d24c440a36b6815f21b70d25016731768db1f. |
| - Runtime access: datasets.load_dataset with streaming=True. |
| - Sample: first 30 unique contexts meeting the previous semantic-search lesson's rules within the first 6,000 rows. |
| - Selection: at most three contexts per display title, title at most 35 characters, excluded title Genocide, title plus context at most 200 embedding-model tokenizer tokens. |
| - Embedding tokenizer: sentence-transformers/all-MiniLM-L6-v2. |
| - Changes: strip surrounding context whitespace and replace underscores in display titles with spaces. Preserve original SQuAD row IDs and P01-P30 source labels. |
| - Only passage titles and text enter embeddings. Questions and answer spans stay outside the embedding index. |
| - Seven answerable evaluation questions derive from the loaded rows. The Montana question explicitly names the state. Three instructor-written challenges test live information, a missing private detail, and ambiguity. |
| - Licence: CC BY-SA 4.0. Retain DATA_ATTRIBUTION.md. |
|
|
| No passage or evaluation JSON file is shipped or read. The application builds Python lists in memory from the Hub dataset. Hugging Face libraries may maintain their own download cache. |
|
|
