Spaces:
Runtime error
Runtime error
File size: 506 Bytes
090668b 0285034 090668b 0285034 090668b 0285034 090668b 0285034 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | ---
title: Tokenizer Visualizer
emoji: 🔤
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: "4.44.0"
app_file: app.py
pinned: false
license: apache-2.0
tags: [tokenizer, visualization, mcp, dispatchai]
---
# 🔤 dispatchAI Tokenizer Visualizer
**MCP Server** — visualize how tokenizers split text.
## Tools
- `tokenize_text(text, method)` — Tokenize with chosen method
- `compare_methods(text)` — Compare token counts across methods
🚀 [dispatchAI](https://huggingface.co/dispatchAI)
|