Morpheus: A Morphology-Aware Neural Tokenizer and Word Embedder for Turkish

Turkish is agglutinative: meaning is carried
by morphemes, yet the subword tokenizers
that drive modern language models split words
by corpus statistics, fragmenting semantically
loaded suffixes and—in the case of WordPiece
and rule-based analyzers—failing to decode
their output back to the original text. This
paper presents Morpheus, a neural morpheme-
boundary model for Turkish that is at once a
lossless, morphology-aware tokenizer and a
word-embedding producer. A differentiable
Poisson–binomial dynamic program turns
per-character boundary probabilities into soft
morpheme memberships during training and
exact segments at inference, with no string
normalization, so decode(encode(w)) = w
holds by construction. Because the model is
neural, the same forward pass that tokenizes
also emits a structured word embedding.
Among reversible tokenizers—the only ones
valid for generation—Morpheus attains the
lowest bits-per-character (1.425), roughly dou-
bles the gold morphological alignment of the
subword family (MorphScore macro-F1 0.61
vs. ∼0.32), and uses ∼19% less GPU memory
than 64K-vocabulary subword tokenizers. As
an embedder, frozen Morpheus vectors lead on
lexical retrieval (root-family MAP 0.85) and
same-root verification (ROC-AUC 1.00), sur-
passing the multilingual retriever BGE-M3 and
BERTurk; on context- and inflection-dependent
tasks (NER, case/number probing) the heav-
ier contextual encoders remain ahead—a
trade-off we attribute to Morpheus’s root-
centric geometry.