Hugging Face data provenance
- Dataset: rajpurkar/squad (SQuAD 1.1).
- Dataset URL: https://huggingface.co/datasets/rajpurkar/squad
- Split: train.
- Revision: 7b6d24c440a36b6815f21b70d25016731768db1f.
- Runtime access: datasets.load_dataset with streaming=True.
- Sample: first 30 unique contexts meeting the previous semantic-search lesson's rules within the first 6,000 rows.
- Selection: at most three contexts per display title, title at most 35 characters, excluded title Genocide, title plus context at most 200 embedding-model tokenizer tokens.
- Embedding tokenizer: sentence-transformers/all-MiniLM-L6-v2.
- Changes: strip surrounding context whitespace and replace underscores in display titles with spaces. Preserve original SQuAD row IDs and P01-P30 source labels.
- Only passage titles and text enter embeddings. Questions and answer spans stay outside the embedding index.
- Seven answerable evaluation questions derive from the loaded rows. The Montana question explicitly names the state. Three instructor-written challenges test live information, a missing private detail, and ambiguity.
- Licence: CC BY-SA 4.0. Retain DATA_ATTRIBUTION.md.
No passage or evaluation JSON file is shipped or read. The application builds Python lists in memory from the Hub dataset. Hugging Face libraries may maintain their own download cache.