Spaces:

suricodes
/

hindi-sindhi-docker

Paused

App Files Files Community

hindi-sindhi-docker / mosesdecoder /scripts /tokenizer

68.2 kB

1 contributor

History: 1 commit

suricodes's picture

Upload folder using huggingface_hub

fd49381 verified about 1 year ago

mosestokenizer
Upload folder using huggingface_hub about 1 year ago
basic-protected-patterns

267 Bytes

Upload folder using huggingface_hub about 1 year ago
deescape-special-chars-PTB.perl

760 Bytes

Upload folder using huggingface_hub about 1 year ago
deescape-special-chars.perl

729 Bytes

Upload folder using huggingface_hub about 1 year ago
delete-long-words.perl

310 Bytes

Upload folder using huggingface_hub about 1 year ago
detokenizer.perl

12.5 kB

Upload folder using huggingface_hub about 1 year ago
escape-special-chars.perl

847 Bytes

Upload folder using huggingface_hub about 1 year ago
lowercase.perl

383 Bytes

Upload folder using huggingface_hub about 1 year ago
normalize-punctuation.perl

1.91 kB

Upload folder using huggingface_hub about 1 year ago
pre-tok-clean.perl

1.43 kB

Upload folder using huggingface_hub about 1 year ago
pre-tokenizer.perl

967 Bytes

Upload folder using huggingface_hub about 1 year ago
pre_tokenize_cleaning.py

2.96 kB

Upload folder using huggingface_hub about 1 year ago
remove-non-printing-char.perl

549 Bytes

Upload folder using huggingface_hub about 1 year ago
replace-unicode-punctuation.perl

872 Bytes

Upload folder using huggingface_hub about 1 year ago
tokenizer.perl

18.4 kB

Upload folder using huggingface_hub about 1 year ago
tokenizer_PTB.perl

12.3 kB

Upload folder using huggingface_hub about 1 year ago