Pretraining Data - a WillHeld Collection

Models
Datasets
Spaces
Buckets new
Docs
Enterprise
Pricing
Log In
Sign Up

WillHeld 's Collections

Pretraining Data

Pretraining Data

updated 14 days ago

opencsg/Fineweb-Edu-Chinese-V2.1

Viewer • Updated Jan 28 • 958M • 55.6k • 65
allenai/dolma3_pool

Preview • Updated 17 days ago • 28.7k • 33
nvidia/Nemotron-CC-v2.1

Viewer • Updated Dec 22, 2025 • 3.8B • 3.43k • 109
allenai/dolma3_dolmino_pool

Updated Jan 5 • 15.3k • 8
allenai/dolma3_longmino_pool

Updated Jan 5 • 3.31k • 12
HuggingFaceFW/finepdfs

Viewer • Updated Jan 9 • 476M • 33.4k • 824
HuggingFaceFW/fineweb-2

Viewer • Updated Oct 27, 2025 • 4.48B • 70.6k • 767
HuggingFaceFW/finewiki

Viewer • Updated Oct 22, 2025 • 61.6M • 7.37k • 287
nyuuzyou/gitee-code

Viewer • Updated Jan 8 • 819M • 5.21k • 12
Common Pile v0.1 Filtered Data

Collection

An LLM pre-training dataset produced by filtering and deduplicating the raw text collected in the Common Pile v0.1 • 31 items • Updated Jun 6, 2025 • 23
tokyotech-llm/swallow-code-v2

Viewer • Updated Nov 8, 2025 • 147M • 5.32k • 34
ByteDance-Seed/Code-Contests-Plus

Viewer • Updated Nov 6, 2025 • 49.2k • 5.03k • 60
nvidia/Nemotron-Math-v2

Viewer • Updated 29 days ago • 7.09M • 3.59k • 167
nvidia/Nemotron-Pretraining-Code-v2

Viewer • Updated Dec 22, 2025 • 836M • 1.4k • 108
nvidia/Nemotron-Pretraining-Specialized-v1

Viewer • Updated Dec 22, 2025 • 60.7M • 2.41k • 75
nvidia/Nemotron-CC-Math-v1

Viewer • Updated Dec 23, 2025 • 190M • 3.32k • 67
nvidia/Nemotron-Pretraining-SFT-v1

Viewer • Updated Dec 23, 2025 • 299M • 1.79k • 62
microsoft/rStar-Coder

Viewer • Updated Jul 20, 2025 • 1.86M • 3.9k • 232
EssentialAI/essential-web-v1.0

Preview • Updated Oct 2, 2025 • 12k • 218
EssentialAI/eai-taxonomy-stem-w-dclm

Preview • Updated Jun 22, 2025 • 19.2k • 6
EssentialAI/eai-taxonomy-med-w-dclm

Viewer • Updated Jun 22, 2025 • 81.2M • 3.25k • 8
EssentialAI/eai-taxonomy-code-w-dclm

Viewer • Updated Jun 22, 2025 • 274M • 181 • 9
EssentialAI/eai-taxonomy-math-w-fm

Viewer • Updated Jun 22, 2025 • 21.6M • 5.99k • 5
GeorgyGUF/mathe-preview

Viewer • Updated Jun 27, 2025 • 27.9B • 73 • 3
DataMuncher-Labs/UltiMath

Viewer • Updated Jan 18 • 32.9B • 7.28k • 43
HuggingFaceFW/finetranslations

Viewer • Updated Jan 9 • 3.33B • 25.7k • 276
PleIAs/common_corpus

Viewer • Updated 21 days ago • 69.9k • 119k • 386
JetBrains-Research/commit-chronicle

Viewer • Updated Oct 5, 2023 • 10.9M • 1.64k • 11
ASSERT-KTH/repairllama-datasets

Viewer • Updated Jan 24, 2025 • 460k • 38 • 2
bigcode/commitpack

Updated Feb 4, 2025 • 15.1k • 78
ASSERT-KTH/RunBugRun

Viewer • Updated Jul 15, 2025 • 778k • 35
Maxscha/commitbench_long

Viewer • Updated Feb 14, 2024 • 3.5M • 219
nick007x/github-code-2025

Viewer • Updated Oct 15, 2025 • 147M • 4.33k • 115
tokyotech-llm/swallow-code

Viewer • Updated 12 days ago • 129M • 1.82k • 63
loubnabnl/github-code-duplicate

Viewer • Updated Jun 27, 2022 • 115M • 396 • 1
macrocosm-os/code-parrot-github-code

Viewer • Updated Oct 30, 2024 • 115M • 326 • 13
nyuuzyou/google-code-archive

Viewer • Updated Feb 1 • 65.8M • 1.29k • 72
ad6398/Deepmind-CodeContest-Unrolled

Viewer • Updated Jan 7, 2024 • 13.2M • 35 • 2
datablations/python-megatron

Updated May 22, 2023 • 10k • 1
nomic-ai/cornstack-python-v1

Viewer • Updated Mar 27, 2025 • 23.6M • 2.61k • 21
simmo/python-fim

Viewer • Updated Sep 26, 2024 • 22.6M • 869
susnato/python_PRs

Viewer • Updated Jan 15, 2024 • 552k • 235
utter-project/github-code-2025-above-2-stars

Viewer • Updated Nov 22, 2025 • 103M • 375 • 3
tokyotech-llm/swallow-math-v2

Viewer • Updated Nov 6, 2025 • 17.4M • 3.45k • 28
openbmb/UltraData-Math

Viewer • Updated 20 days ago • 181M • 77.5k • 259
hchautran/react_repos

Viewer • Updated Jun 9, 2022 • 513k • 54 • 6
jobs-git/HPLT2.0_cleaned

Updated Mar 7, 2025 • 151k
Zyphra/Zyda-2

Preview • Updated Aug 6, 2025 • 113k • 87
CodedotAI/code_clippy_github

Viewer • Updated Aug 5, 2022 • 2.4M • 6.37k • 20
Lichess/standard-chess-games

Viewer • Updated Oct 16, 2025 • 7.14B • 366k • 64
jablonkagroup/chempile-mlift

Viewer • Updated Jul 27, 2025 • 51.5M • 12.4k • 14
ChemRAG/ChemRAG-Corpus

Viewer • Updated Apr 11, 2025 • 49.8M • 25
jablonkagroup/chempile-code

Viewer • Updated Aug 13, 2025 • 2.27M • 277 • 4
jablonkagroup/chempile-paper

Viewer • Updated Aug 13, 2025 • 11.7M • 347 • 6
jablonkagroup/chempile-education

Viewer • Updated Jun 23, 2025 • 66.9k • 273 • 6
marianna13/biorxiv

Viewer • Updated Mar 2, 2023 • 164k • 17 • 2
institutional/institutional-books-1.0

Viewer • Updated Jun 16, 2025 • 983k • 4.93k • 268
yongchanskii/OCW

Viewer • Updated 30 days ago • 34.5k • 50
common-pile/youtube_filtered

Viewer • Updated Jun 6, 2025 • 986k • 44 • 5
common-pile/ubuntu_irc_filtered

Viewer • Updated Jun 6, 2025 • 216k • 20 • 1
common-pile/pubmed_filtered

Viewer • Updated Jun 6, 2025 • 4.77M • 461 • 3
common-pile/github_archive_filtered

Viewer • Updated Jun 6, 2025 • 23.3M • 238 • 1
common-pile/biodiversity_heritage_library_filtered

Viewer • Updated Jun 6, 2025 • 16.5M • 57 • 1
common-pile/uspto_filtered

Viewer • Updated Jun 6, 2025 • 14.4M • 948 • 3
common-pile/usgpo_filtered

Viewer • Updated Jun 6, 2025 • 2.34M • 93 • 1
soda-research/emilia-mm-pretrain-fix

Viewer • Updated Dec 6, 2025 • 12.6M • 6.08k
fineinstructions/fineinstructions_nemotron

Viewer • Updated Jan 30 • 1.23B • 4.66k • 5

Collection guide
Browse collections

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs