Spaces:
Sleeping
Sleeping
| title: LanguageBPE | |
| emoji: 🌖 | |
| colorFrom: yellow | |
| colorTo: gray | |
| sdk: gradio | |
| sdk_version: 4.37.1 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| # App to visualize results of BPE tokenizer trained on wikipedia and opus dataset. | |
| ## Features | |
| - Input/output text: Write some word(s) and see tokenized output | |
| - Tokenizer: Select tokenizer | |
| ## Usage | |
| - Enter sentence in Hindi, English or both languages | |
| - Select tokenizer model | |
| - Hit submit and see result | |