wikiparse 2606 data

Large-file delivery for the wikiparse 2606 technical milestone, derived from English Wikipedia 2026-06-01.

The versioned robbd/wikiparse-2606 dataset is the catalog: ten compact Parquet tables, checksums, schemas, provenance, and five small analysis cards. This bucket carries the larger context surfaces.

Contents

The content release is partitioned into 1,003 disjoint 256-stream shards, with one shorter terminal shard. Each shard has one Parquet file per priority and an exact manifest; the root manifest and receipt bind every shard and file.

Priorities

These labels are mutually exclusive delivery hints. They are not quality judgments, exclusion authority, or a claim that p2/p3 are useless. Categories and detector signals remain attached so consumers can write a different policy.

Download

hf buckets sync \
  hf://buckets/robbd/wikiparse-2606-data/context/index/v1 \
  ./wikiparse-2606-index

hf buckets sync \
  hf://buckets/robbd/wikiparse-2606-data/context/content/v1 \
  ./wikiparse-2606-content

# Or fetch only the science-first content slice and its receipts.
hf buckets sync \
  hf://buckets/robbd/wikiparse-2606-data/context/content/v1 \
  ./wikiparse-2606-p0 \
  --include "shards/*/p0.parquet" \
  --include "shards/*/manifest.json" \
  --include "manifest.json" \
  --include "receipt.json"

The context files copy retained Markdown from the existing compact corpus. They do not rerun the parser, call a model, or alter the source store.

Boundary

• Technical milestone only; announcement and outreach remain held.
• Citation records are article-level and deduplicated, not inline claim markers.
• Parser-quality and detector fields are operational evidence, not adjudicated truth.
• No model weights are published here.
• Data is distributed under CC BY-SA 4.0; attribution and modification notes are in the versioned dataset.