Hugging Face's logo

jescy525
/
archon-tokenizer-v2

Model card Files
xet
 Community

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

ARCHON Tokenizer v2

BPE 32K + 8 special tokens ChatML/tool-calling.

Tokens ajoutes:

  • <|im_start|>
  • <|im_end|>
  • <|system|>
  • <|user|>
  • <|assistant|>
  • <|tool_call|>
  • <|tool_result|>
  • <|task_type|>

Base: ARCHON brain tokenizer (vocab BPE 32000). Compatibilite IDs preserve.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support