Simplify dataset download instructions

README.md

@@ -64,34 +64,18 @@ The four downstream benchmarks are released as separate dataset repositories, ea
 
 ```python
 # pip install huggingface_hub
-from pathlib import Path
 from huggingface_hub import snapshot_download
 
-def flatten_shards(root):
-    """Move shard_NN/* back into images/ resp. labels/ (D4LA, M6Doc only)."""
-    root = Path(root)
-    for sub in ("images", "labels"):
-        d = root / sub
-        for shard in sorted(d.glob("shard_*")):
-            for f in shard.iterdir():
-                target = d / f.name
-                if target.exists():
-                    raise FileExistsError(target)  # names are unique across shards
-                f.rename(target)
-            shard.rmdir()
-
 for repo, local in [
     ("juliozhao/D4LA",      "D4LA"),        # keep these exact directory names
-    ("juliozhao/M6Doc",     "m6doc"),       # (the pipeline aliases depend on them)
+    ("juliozhao/M6Doc",     "m6doc"),       # (the val/train scripts depend on them)
     ("juliozhao/HJDataset", "HJDataset"),
     ("juliozhao/PRIMA-LAD", "prima-lad"),
 ]:
-    snapshot_download(repo, repo_type="dataset", local_dir=local,
-                      ignore_patterns=["v1.zip"])
-    flatten_shards(local)
+    snapshot_download(repo, repo_type="dataset", local_dir=local)
 ```
 
-> **Shard note.** D4LA and M6Doc exceed HuggingFace's 10,000-files-per-directory limit, so overflow files are stored in `shard_NN/` subdirectories. The `flatten_shards` helper above restores the standard flat layout after download. HJDataset's `v1.zip` is excluded by the same code — it is a redundant re-pack of the same pages and is not needed.
+Each repository is ready to use as-is: the split lists point to the actual file locations, so no post-processing is needed after download.
 
 ## Usage