Byt5_OOV_finetuned

A ByT5-small model fine-tuned to predict morpheme boundaries in Nepali words, with a specific focus on generalizing to roots never seen during training (out-of-vocabulary / OOV generalization).

Given a raw Nepali surface word, the model outputs the word split into root and affix, separated by +:

Input:  अँकाइनु
Output: अँका+इनु

Model Details

Model Description

Nepali morphology means a single dictionary root can surface in many inflected/derived forms. This model is trained to segment a word into its root and affix components at the byte level, which sidesteps the risk of a subword tokenizer fragmenting Devanagari's multi-byte, combining-mark-heavy sequences inconsistently.

• Developed by: w4ashabii
• Model type: Encoder-decoder (T5-family, byte-level), fine-tuned
• Language: Nepali (ne)
• License: MIT
• Finetuned from model: google/byt5-small

Model Sources

• Training data: w4ashabii/root_affix_dictionary

Uses

Direct Use

Segmenting a Nepali surface word into root + affix, e.g. as a preprocessing step for morphological analysis, lemmatization, or downstream NLP pipelines that benefit from root-level normalization.

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("w4ashabii/Byt5_OOV_finetuned")
model = AutoModelForSeq2SeqLM.from_pretrained("w4ashabii/Byt5_OOV_finetuned")

word = "अँकाइनु"
input_ids = tokenizer(word, return_tensors="pt").input_ids
output_ids = model.generate(input_ids, max_length=40)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
# अँका+इनु

Out-of-Scope Use

Not intended for full morphological analysis beyond root/affix segmentation (e.g. no POS tagging, no gloss/meaning output), and not evaluated on languages other than Nepali.

Bias, Risks, and Limitations

• Training data includes both dictionary-verified segmentations and heuristically-inferred ones (see the dataset card); heuristic-derived examples may carry systematic rule-engine errors that the model can inherit.
• Evaluated on a root-disjoint OOV test set drawn from the same dictionary source (10th edition); performance on genuinely novel coinages, loanwords, or dialectal forms outside that source is untested.
• The model slightly under-splits relative to over-splitting (see boundary precision vs. recall below) — it more often misses a boundary than invents a spurious one.

Training Details

Training Data

Fine-tuned on w4ashabii/root_affix_dictionary, split root-disjoint (not just word-disjoint) via grouped splitting on the Root column, so no root appears in more than one split — this is what makes the test set a genuine test of generalization to unseen roots rather than memorization.

• Train: 42,647 examples
• Validation: 5,212 examples
• Test: 5,260 examples

Training Procedure

• Base model: google/byt5-small
• Precision: fp32 (ByT5/T5 models are numerically unstable in fp16; bf16 was unavailable on the training GPU)
• Epochs: trained for 10, best checkpoint selected by validation exact match — peaked at epoch 4 and mildly overfit afterward
• Batch size: 128 train / 64 eval
• Learning rate: 3e-4
• Selection: best checkpoint by validation exact-match accuracy

Hardware

Single NVIDIA T4 GPU (Google Colab)

Evaluation

Evaluated on the root-disjoint OOV test split (5,260 words never seen — by root — during training).

Metrics

Two metrics, matching what matters for partial credit vs. strict correctness:

• Exact Match — predicted segmented string equals gold exactly
• Boundary F1 — treats each + as a boundary at a character offset in the reconstructed word and scores predicted vs. gold boundary positions with precision/recall/F1, so a model that gets 3 of 4 boundaries right on a word scores better than one that gets 0

Results

Metric	Score
Exact Match	0.8375
Boundary Precision	0.8094
Boundary Recall	0.7401
Boundary F1	0.7732

The gap between boundary precision and recall indicates the model is somewhat more conservative than aggressive — it's more likely to miss a true boundary on an unseen root than to hallucinate one that isn't there.

Citation

BibTeX:

@misc{byt5_oov_finetuned,
  author = {w4ashabii},
  title = {Byt5\_OOV\_finetuned: A ByT5 model for Nepali morpheme boundary prediction},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/w4ashabii/Byt5_OOV_finetuned}}
}