balochiml
/

balochi-tokenizer

Model card Files Files and versions

balochi-tokenizer / src

Commit History

more spacy

e938b17

strickvl commited on Jun 3, 2023

add spacy trainer

b1d540a

strickvl commited on Jun 3, 2023

update trainer script

e8868b3

strickvl commited on Jun 3, 2023

update tokenizer script

57c829b

strickvl commited on Jun 3, 2023

add tokenization process

24469e0

strickvl commited on Jun 3, 2023

process and clean files

b3db2b8

strickvl commited on Jun 3, 2023

Get all text paths

470e696

strickvl commited on Jun 3, 2023

add data folder to gitignore

c9036ae

strickvl commited on Jun 3, 2023