applied-ai-018
/

peacock-data-public-evaluation

Model card Files Files and versions

peacock-data-public-evaluation / Megatron-DeepSpeed /tools /openwebtext

57.6 kB

1 contributor

History: 1 commit

applied-ai-018's picture

Add files using upload-large-folder tool

ced76ab verified about 1 year ago

README.md

3.43 kB

Add files using upload-large-folder tool about 1 year ago
add_id.py

1.64 kB

Add files using upload-large-folder tool about 1 year ago
blacklist_urls.py

6.78 kB

Add files using upload-large-folder tool about 1 year ago
cleanup_dataset.py

3.68 kB

Add files using upload-large-folder tool about 1 year ago
cleanup_fix_dataset.py

6.66 kB

Add files using upload-large-folder tool about 1 year ago
filter_ngrams.py

18.3 kB

Add files using upload-large-folder tool about 1 year ago
find_duplicates.py

11.5 kB

Add files using upload-large-folder tool about 1 year ago
group_duplicate_url.py

2.66 kB

Add files using upload-large-folder tool about 1 year ago
merge_jsons.py

1.02 kB

Add files using upload-large-folder tool about 1 year ago
remove_group_duplicates.py

1.98 kB

Add files using upload-large-folder tool about 1 year ago