docs/source/tokenizer/index.rst

.. _tokenizer:

Tokenizer models
================

.. needtable::
   :types: tokenizer
   :style: datatable
   :columns: ID;language;license
   :class: table-striped
   :colwidths: 50;25;25

.. toctree::
   :hidden:

   Japanese/index.rst