Ethosoft
/

NedoTurkishTokenizer

Model card Files Files and versions

NedoTurkishTokenizer

32.4 MB

Ctrl+K

Ctrl+K

6 contributors

History: 20 commits

nmstech's picture

Zemberek LFS ile eklendi

fae9ef5 about 1 month ago

.claude
Fix broken placeholder mechanism: replace with segment-based tokenization about 1 month ago
nedo_turkish_tokenizer
Rename project from TurkTokenizer to NedoTurkishTokenizer about 1 month ago
.gitattributes

1.64 kB
Zemberek LFS ile eklendi about 1 month ago
.gitignore

75 Bytes
Fix broken placeholder mechanism: replace with segment-based tokenization about 1 month ago
README.md

6.5 kB
Rename project from TurkTokenizer to NedoTurkishTokenizer about 1 month ago
pyproject.toml

1.16 kB
Rename project from TurkTokenizer to NedoTurkishTokenizer about 1 month ago
tokenization_nedo_turkish.py

6.45 kB
Rename project from TurkTokenizer to NedoTurkishTokenizer about 1 month ago
tokenizer_config.json

388 Bytes
Rename project from TurkTokenizer to NedoTurkishTokenizer about 1 month ago