sleepyhead111
/

vllm_fairseq

Model card Files Files and versions

vllm_fairseq / mosesdecoder /scripts /tokenizer

Ctrl+K

Ctrl+K

1 contributor

History: 2 commits

sleepyhead111's picture

Add files using upload-large-folder tool

e9f8719 verified about 1 year ago

mosestokenizer
Add files using upload-large-folder tool about 1 year ago
basic-protected-patterns

267 Bytes
Add files using upload-large-folder tool about 1 year ago
deescape-special-chars-PTB.perl

760 Bytes
Add files using upload-large-folder tool about 1 year ago
deescape-special-chars.perl

729 Bytes
Add files using upload-large-folder tool about 1 year ago
delete-long-words.perl

310 Bytes
Add files using upload-large-folder tool about 1 year ago
detokenizer.perl

12.5 kB
Add files using upload-large-folder tool about 1 year ago
escape-special-chars.perl

847 Bytes
Add files using upload-large-folder tool about 1 year ago
lowercase.perl

383 Bytes
Add files using upload-large-folder tool about 1 year ago
normalize-punctuation.perl

1.91 kB
Add files using upload-large-folder tool about 1 year ago
pre-tok-clean.perl

1.43 kB
Add files using upload-large-folder tool about 1 year ago
pre-tokenizer.perl

967 Bytes
Add files using upload-large-folder tool about 1 year ago
pre_tokenize_cleaning.py

2.96 kB
Add files using upload-large-folder tool about 1 year ago
remove-non-printing-char.perl

549 Bytes
Add files using upload-large-folder tool about 1 year ago
replace-unicode-punctuation.perl

872 Bytes
Add files using upload-large-folder tool about 1 year ago
tokenizer.perl

18.4 kB
Add files using upload-large-folder tool about 1 year ago
tokenizer_PTB.perl

12.3 kB
Add files using upload-large-folder tool about 1 year ago