Spaces:

Chintan-Shah
/

HindiTokenizerFromScratch

Sleeping

App Files Files Community

HindiTokenizerFromScratch

319 kB

1 contributor

History: 38 commits

Chintan-Shah's picture

Update app.py

847ca07 verified over 1 year ago

.gitattributes
1.52 kB

initial commit over 1 year ago
180548df513ce4f4.css
18.9 kB

Upload 180548df513ce4f4.css over 1 year ago
README.md
244 Bytes

initial commit over 1 year ago
app.py
14.4 kB

Update app.py over 1 year ago
merges.pkl
106 kB
xet

Updated merges and vocab with 10000 vocab size and merging the 3 byte char tokens for devnagari script first over 1 year ago
requirements.txt
0 Bytes

Update requirements.txt over 1 year ago
vocab.pkl
179 kB
xet

Updated merges and vocab with 10000 vocab size and merging the 3 byte char tokens for devnagari script first over 1 year ago