brian-learns/cdx-cc-news
Viewer • Updated • 1.46B • 1.57k • 5
CC-NEWS
Dataset, space, and buckets for Common Crawl News indexing explorations
Note Dataset with cdxj, parquet, and a viewer 10.57967/hf/9637
Common Crawl News Index (CC-NEWS)
🗞Search over 1.4 Billion archived news URLs from CC-NEWS
Note API endpoint /lookup for ?url=...&exact=false&limit=100
brian-learns/cc-news-cdx-server-storage
75.5 GBNote Bucket with RocksDB sst files and database