Buckets:

glennmatlin's picture
|
download
raw
5.36 kB

HCAI-Lab HuggingFace Naming Convention

Last updated: 2026-05-24.

This document defines the naming convention for HuggingFace repositories (buckets, datasets, models, spaces) under the HCAI-Lab organization. Apply to every new repo. Existing repos were migrated to this convention in PR 3 (2026-05-24).

The rule

Lowercase letters, digits, and hyphens only. No underscores. No uppercase.

✅ HCAI-Lab/dolma3-6t-sample-500-docs
✅ HCAI-Lab/trackstar-scores-base-olmes-4bench
✅ HCAI-Lab/olmes-eval-olmo3-7b-base
✅ HCAI-Lab/archive-dolma3-pool-150b

❌ HCAI-Lab/dolma3_6T_sample_500_docs       (underscores + uppercase)
❌ HCAI-Lab/Trackstar-Scores                 (uppercase)
❌ HCAI-Lab/dolma3_6t_doc_ids                (underscores)

Reasons:

  • Already used by all buckets and most newer datasets — smaller migration than the alternative
  • URL-friendly: no escaping needed in shell, browser, or markdown
  • Case-insensitive friendly: works on macOS / Windows / case-insensitive filesystems
  • Matches the de facto HF Hub / GitHub convention

Repo-name structure

HCAI-Lab/<prefix>-<purpose-tokens>[-<axis-tokens>]

Where:

  • <prefix> is one of:

    • dolma3- — Dolma 3 corpus, samples, manifests, sidecars, dedup state
    • trackstar- — TrackStar / Bergson attribution artifacts (gradient indices, query gradients, scores, preconditioners, top-K results, analysis)
    • olmes-eval- — OLMES evaluation result datasets
    • archive- — pre-6T-era datasets and other legacy work kept for reproducibility
    • (project-specific, e.g. tombench-) — other research projects under HCAI-Lab
  • <purpose-tokens> describes what the repo holds. Examples: gradient-index, query-gradients, scores, corpus-manifest, bloom-index, bin-characterization.

  • <axis-tokens> optional disambiguator for variants. Order: artifact-type → model → benchmark. Examples: base-olmes-4bench, instruct-cot-noprecond-mmlu-ss, olmo3-7b-base, gsm8k-arc, bbh.

Anti-patterns

  • Do not embed SOC ticket numbers in repo names. Tickets are transient; the repo outlives them. Record SOC lineage in the repo README's "Provenance" section instead. (We dropped SOC# from 20 repos in PR 2 / PR 3 for this reason.)
  • Do not use camelCase or PascalCase. Always lowercase.
  • Do not use underscores. Always hyphens.
  • Do not put project name as prefix when the org name already covers it. HCAI-Lab/data-attribution-olmes-... is redundant; we're already in the data-attribution org.
  • Do not version numbers via -v2 etc. for incremental updates. Use the same name and let HF Hub track history. If a true breaking change is needed, prefix with archive- on the old version.

Dataset + bucket pairs

Some artifacts have both a Dataset surface (for dataset viewer + snapshot_download access) and a Bucket surface (for hf buckets sync / S3-style access). When both exist for the same data:

  • Use the same canonical name across both surfaces.
  • Both repos share the URL handle HCAI-Lab/<name>. HF treats datasets and buckets as separate repo types in the same namespace; they coexist without collision.
  • Document the dual surface in the repo README so consumers know they can pick either path.

Example: the 500-docs sample is accessible as both:

  • Dataset: https://huggingface.co/datasets/HCAI-Lab/dolma3-6t-sample-500-docs
  • Bucket: hf://buckets/HCAI-Lab/dolma3-6t-sample-500-docs

Era markers

  • Current: no prefix marker. Active work uses purpose-themed names without era hint.
  • Archive: prefix archive-. Used for pre-6T-era datasets, deprecated query versions, and other legacy. Sorts visually below current work in alphabetical listings.

Provenance for renames

Every time a repo is renamed:

  1. Add a ## Provenance section to its README listing old name, SOC lineage (if any), and rename date.
  2. Add an old_names field to its entry in docs/data_home/inventory.json with the previous name(s).

This lets future readers trace any historical reference back to the current repo.

Open mappings (R2 ↔ HF)

The R2 bucket soc127-dedup still uses SOC-numbered prefixes (e.g., soc91-labels/, soc91-stats/, soc139-quality-sidecars/). Three HF datasets mirror these R2 prefixes and were intentionally left at their R2-matching names to preserve HF↔R2 parity:

  • HCAI-Lab/soc91-labels ↔ R2 soc91-labels/
  • HCAI-Lab/soc91-stats ↔ R2 soc91-stats/
  • HCAI-Lab/soc139-quality-sidecars ↔ R2 soc139-quality-sidecars/

When R2 reorganization is undertaken (see docs/R2_FUTURE_REORG.md), these will be brought into the dashes-lowercase convention together with the R2 source. Until then, they are an intentional exception.

Enforcement

  • New repo creation: follow this convention. If a name doesn't fit, propose the new pattern in a doc PR first.
  • Renames are infrequent and require a Provenance README update + inventory.json update.
  • The docs/data_home/inventory.json file is the canonical source of truth for what exists and what it's called.

Future home

The user has flagged a possible separate hcai-lab-org GitHub repository for org-level documentation in the future. When that exists, this naming-convention doc should move there so it applies across all HCAI-Lab projects, not just data-attribution.

Xet Storage Details

Size:
5.36 kB
·
Xet hash:
2742997ec56545831bd52b35bac42b308b16fdedbf8cb89ba468184d347cbe7d

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.