Boulifa-48M

A 47.8M-parameter character-level Conv-Transformer for Kabyle (Taqbaylit, kab), trained from scratch to normalise informal, French-keyboard, and Arabizi Kabyle typing into canonical Kabyle Latin — restoring emphatics, digraph expansions, hyphenated clitic boundaries, and preposition contractions in one pass.

It reaches 99.45% character accuracy on held-out test pairs under greedy free-running decoding, converting everyday social-media and SMS Kabyle into orthographically correct text without any lexicon or rule hand-engineering.

It is, as far as we can establish, the first orthography standardisation model published for Kabyle, or for any Berber language.

Named after Si Amar ou Saïd Boulifa (1865–1931), grammarian and the first systematic codifier of Kabyle Latin orthography.

The task

Kabyle speakers writing in informal digital contexts use a repertoire of keyboard strategies that diverge sharply from the canonical Latin orthography:

phenomenon	example input	canonical output
French digraph gh for ɣ	ighef	iɣef
French digraph kh for x	khedmekh	xedmex
French digraph ch for c	achimi	acimi
Arabizi digit 7 for ḥ	l7adj	lḥadj
Arabizi digit 3 for ɛ/ɣ	3emmi, gh3ir	ɛemmi, ɣir
Arabizi digit 5 for x	5ir	xir
French ou digraph for u	touddart	taddart
Emphatic drop, dh for ḍ	thekhedmedh	tḥexedmeḍ
Emphatic drop, th for ṭ	thefsouth	tḥefsuṭ
Clitic hyphen omission	dyeffegh	d-yeffeɣ
Preposition contraction	g taddart	deg taddart
Identity (already canonical)	Azul fell-awen	Azul fell-awen

A rule table can handle each substitution in isolation. What it cannot do is resolve ambiguity across phenomena simultaneously: 3 maps to ɛ or ɣ depending on position; th maps to tḥ (emphatic cluster) or t + h depending on morphological context; clitic hyphen positions depend on verb class and directional clitics simultaneously. That is a sequence transduction problem, and it is why a table reaches < 2% character accuracy on naturalistically corrupted text where this model reaches 99.45%.

Results

Held-out test split of agbalu/KabStandard: 24,898 pairs, disjoint from train (448,149) and dev (24,897), constructed from the same source sentences as the agbalu/KabTifinagh corpus under a 90/5/5 split at seed 42. Greedy free-running decoding — the model is fed its own output, which is what a caller gets.

system	character accuracy	character error rate
Boulifa-48M	99.45%	0.55%
rule table (all substitutions, no context)	< 2%	> 98%

The rule table's figures reflect a pure deterministic map applied to naturalistically corrupted inputs: 3 is always ɛ, gh is always ɣ, hyphens are never restored, and prepositions are never expanded. Every context-sensitive case is wrong by construction, which is why the gap is not incremental.

The evaluation pairs are not hand-transcribed. KabStandard is constructed by a probabilistic corruption pass over canonical Kabyle text (see Training data below). The character accuracy is measured on the round-trip: can the model recover the canonical target from a plausibly corrupted source? It cannot be read as accuracy on arbitrary human typing, only on the corruption distribution defined in agbalu/KabStandard.

Reproduce without a GPU:

make test-boulifa

Three things worth reading carefully.

The baseline is a rule table, not another model. No neural model had been published for Kabyle orthography standardisation before this one. The comparison is the only tool that existed, and the gap is not incremental.

Character accuracy overstates a single metric. 0.55% CER means roughly one character in 182; some fraction of those errors cluster on the same hard sentences — clitic chains with multiple emphatics and a dropped hyphen simultaneously — while the majority of sentences are exact matches.

The dev character accuracy at the checkpoint used (Epoch 2) was 99.45%. Epoch 1 reached 99.22% and Epoch 3 was not completed. The checkpoint shipped is boulifa_best.pt, selected on dev character accuracy, not on loss.

Qualitative examples

Every example was decoded against the published checkpoint before being written down.

IN:  "achimi ur d-thekhedmedh ara tamazight g l'ecole?"
OUT: "acimi ur d-tḥexedmeḍ ara tamaziɣt deg lɛecule?"

IN:  "3emmi l7adj yerza-d 5ir d lbaraka s wuzzal"
OUT: "Ɛemmi lḥadj yerza-d xir d lbaraka s wuzzal"

IN:  "thessawledh-d fellanegh zik g thefsouth"
OUT: "tḥessawleḍ-d fell-aneɣ zik deg tḥefsuṭ"

IN:  "yennayasen ur ten-idttakken ara degs"
OUT: "yennayasen ur ten-id-ttakken ara degs"

IN:  "matchi akken i thebghidh a thaddarth-iw"
OUT: "mači akken i tḥebɣiḍ a tḥaddarṭ-iw"

IN:  "Azul fell-awen, amek i telliḍ taṣebḥit-a?"
OUT: "Azul fell-awen, amek i telliḍ taṣebḥit-a?"

The last example is already canonical; the model leaves it unchanged.

Intended use

Converting informal, SMS, and French-keyboard Kabyle text into canonical Kabyle Latin orthography as a preprocessing step for:

• ASR post-processing: transcripts from agbalu/Fadhma-300M are already canonical; Boulifa targets user-typed input before it enters downstream models.
• NLP pipeline chaining: sits after agbalu/Juba-27M (Tifinagh to Latin) and agbalu/Belaid-31M (punctuation and casing), passing clean text to agbalu/SiMohand-278M (sentence embeddings) or agbalu/Matoub-82M (TTS).
• Corpus normalisation: preparing user-generated content for inclusion in training corpora for downstream Kabyle models.

Not suitable for: translation between Kabyle and any other language; any decision about a person; or any language other than Kabyle. The model has not been evaluated for bias, toxicity, or systematic failure patterns against sociolects or regional variants. No safety evaluation of any kind has been performed.

Usage

torch only — no transformers dependency. The architecture is custom and ships in this repository.

from agbalu.standardise.infer import Standardiser

standardiser = Standardiser.load()                    # loads boulifa_best.pt automatically

result = standardiser.standardise("achimi ur d-thekhedmedh ara tamazight g l'ecole?")
# "acimi ur d-tḥexedmeḍ ara tamaziɣt deg lɛecule?"

result = standardiser.standardise("3emmi l7adj yerza-d 5ir d lbaraka s wuzzal")
# "Ɛemmi lḥadj yerza-d xir d lbaraka s wuzzal"

result = standardiser.standardise("Azul fell-awen, amek i telliḍ taṣebḥit-a?")
# "Azul fell-awen, amek i telliḍ taṣebḥit-a?"   # canonical, unchanged

Or from the command line:

make infer-boulifa TEXT="achimi ur d-thekhedmedh ara tamazight g l'ecole?"
# Standardised: "acimi ur d-tḥexedmeḍ ara tamaziɣt deg lɛecule?"

Standardiser.load() resolves the checkpoint automatically from the standard artifact locations (artifacts/boulifa/boulifa_best.pt, then artifacts/boulifa/boulifa_final.pt, then artifacts/checkpoints/Boulifa-48M/). Pass a path explicitly to override.

Output is greedy by default. The model has no beam search; greedy decoding is the evaluation mode and the numbers in this card are from it.

Input length. max_length=512 characters is the default. Kabyle sentences rarely exceed 200 characters; the limit is not load-bearing in practice.

Architecture

Parameters	47,797,760
Encoder layers	6
Decoder layers	6
Hidden / feed-forward	512 / 1,536 (SwiGLU)
Attention heads / head size	8 / 64
Positions	rotary (RoPE), on self-attention only
Conv stem	1D depthwise separable, kernels 3 and 5, before the encoder
Vocabulary	128 character slots; 96 used, 0 out-of-vocabulary on Kabyle Latin
Tied weights	input embedding tied to output projection
Label smoothing	ε = 0.05
Dropout	0.10

Component breakdown:

component	parameters
Decoder layers (6×)	26,747,904
Encoder layers (6×)	20,453,376
Convolutional stem	529,920
Embedding (tied)	65,536
Layer norms	1,024

The architecture mirrors agbalu/Juba-27M in structure but is larger: hidden size 512 (vs 384), 8 heads (vs 6), and a SwiGLU FFN at 3x hidden. The bigger capacity is justified by the task: Juba maps one script to another where consonants are one-to-one and only the schwa placement is ambiguous. Boulifa must resolve simultaneous ambiguity across phoneme identity, clitic boundaries, and preposition contraction, which is a harder and more context-dependent problem.

The convolutional stem. Two depthwise 1D kernels (3 and 5) run before the encoder's first layer. Their receptive field covers the bigram and trigram context that most keyboard substitutions span (kh, gh, ch, tch, dj, ou), so the encoder inherits local character-cluster information rather than spending self-attention capacity on adjacency.

Rotary positions (RoPE) on self-attention only. The cue for where a hyphen belongs or which consonant a digit maps to is the distance between characters, not where in the sentence the word sits. RoPE encodes relative positions natively. It is not applied to cross-attention, where the two sequences are of different lengths and a shared position index would assert a false alignment.

Tied input and output embedding. The input and output alphabets are identical (128 character vocabulary), so sharing these weights is the correct constraint, not a saving.

Training data

agbalu/KabStandard — 497,944 parallel pairs derived from the Latin side of agbalu/KabTifinagh (all three splits: train, dev, test). The pairs map a probabilistically corrupted source to its canonical Latin target.

• 15% identity pairs: canonical input passed unchanged, teaching the model not to alter already-correct text.
• 85% corrupted pairs: canonical text processed through a probabilistic corruption pass (corrupt_text in agbalu.standardise.corpus) that applies substitutions independently:

corruption	canonical char	informal form	probability
ɣ digraph	ɣ	gh	0.75
x digraph	x	kh	0.85
c digraph	c	ch	0.75
ğ digraph	ğ	dj	0.80
ḍ emphatic	ḍ	dh	0.75
ṭ emphatic	ṭ	th	0.70
ḥ pharyngeal	ḥ	7 (Arabizi)	0.25
ɣ Arabizi	ɣ	3	0.08
ɛ pharyngeal	ɛ	3	0.25
x Arabizi	x	5	0.05
u digraph	u	ou	0.45
clitic hyphen drop	-	or ``	0.50
preposition contraction	deg	g	0.25

The corruption is stochastic per character independently, so any given sentence may carry zero, one, or many phenomena simultaneously. Seed 42 is fixed for reproducibility.

Split:

split	pairs
train	448,149
dev	24,897
test	24,898
total	497,944

The source sentences are canonical Kabyle Latin normalised under AGBALU normaliser 1.3.0+rules1.0.0, the same normaliser used for Juba-27M, Masinissa-31M and Amrouche-1.3B. Any systematic error in that normaliser propagates to the model and to the test evaluation. The numbers above are measured on the round-trip, not against independently authored informal text.

Training recipe

Objective	character cross-entropy with label smoothing ε = 0.05
Optimiser	AdamW, lr 5e-4, β (0.9, 0.98), weight decay 0.01, ε 1e-8
Batch size	64 sequences per step
Gradient clipping	max norm 1.0
Hardware	one NVIDIA A10G 24 GiB (Modal), detached run
Runtime	5.5 h per epoch (8,500 tokens/s)
Epochs trained	2 (stopped on dev accuracy plateau)
Checkpoint selection	best dev character accuracy (boulifa_best.pt)
Seed	42

Training curve (greedy character accuracy on dev):

epoch	dev character accuracy	dev loss
1	99.22%	0.4652
2	99.45%	0.4572

Epoch 2 was the checkpoint selected and published. Epoch 1's dev accuracy already exceeds 99%, consistent with the task being learnable quickly once the substitution table is absorbed, and the harder clitic-chain cases driving the loss plateau.

make modal-train-boulifa            # deploy and spawn detached
make modal-logs FUNCTION=boulifa_train   # attach log viewer

Limitations

The corruption distribution is synthetic, not human. KabStandard pairs are generated by a stochastic rule applied to canonical text. Real social-media Kabyle carries idiosyncrasies — phonetic respellings, loan-word renderings, mixed French-Kabyle code-switching mid-sentence, regional orthographic conventions — that the corruption pass does not fully capture. The 99.45% figure is an upper bound on performance against naturally occurring informal text.

Clitic chains with multiple simultaneous phenomena are the hardest cases. A sequence like ten-id-ttakken requires the model to restore the hyphen, identify the direction clitic id, and correctly delimit ttakken — all from a mashed input tenidttakken. The model handles this class (see Qualitative examples) but it concentrates most of the residual error there.

ɛ and ɣ share the digit 3 in Arabizi. The model must resolve this from context: 3emmi gives Ɛemmi (pharyngeal fricative, word-initial); gh3ir gives ɣir. This is the single most context-sensitive substitution and the one most likely to produce an error when the context is unusual or ambiguous.

No recovery from cascaded errors. If the input contains a corruption outside the trained distribution — a typo that is not a known substitution, or a word from a regional variety with different spelling conventions — the model may produce a plausible but incorrect output without signalling uncertainty.

One language. Trained and evaluated on Kabyle. Tarifit, Tashelhit, Central Atlas Tamazight and Shawiya have related but distinct orthographic conventions and are not evaluated here.

No safety evaluation of any kind has been performed.

What was not measured

• No inter-annotator ceiling exists for Kabyle informal spelling. Nobody has established what two fluent readers agree the canonical form of a given informal input should be, so no figure here can be read as a fraction of what is attainable.
• No out-of-domain evaluation. The test pairs come from the same corruption distribution as training. Performance on scraped social media, WhatsApp transcripts, or forum posts has not been measured.
• Sentence exact match was not computed for the published checkpoint. Character accuracy was the training signal. Sentence exact match would be a stricter and more informative figure; it is not reported here because the metric was not logged during the run.

Files

file	size	description
boulifa_best.pt	191.3 MB	weights + config dict, selected on dev char accuracy (Epoch 2)

The checkpoint holds the model state dict and the config dict (asdict(ModelConfig())), and nothing else. No optimiser state, no scheduler state, no training curve. Training cannot be resumed from the published file.

Will be exported to model.safetensors before HuggingFace release.

Reproduction

make prepare-boulifa                 # build KabStandard dataset on Modal volume
make modal-train-boulifa EPOCHS=3    # train detached (~5.5h/epoch on A10G)
make modal-boulifa-pull              # download best checkpoint to artifacts/boulifa/
make infer-boulifa TEXT="achimi ur d-thekhedmedh ara tamazight g l'ecole?"
make test-boulifa                    # unit tests for standardise module

The dev character accuracy logged during training (99.45% at Epoch 2) is what this card reports. No separate post-hoc evaluation script exists at publication time.

The name

Si Amar ou Saïd Boulifa (1865–1931) was the first grammarian to systematically codify Kabyle Latin orthography. His 1897 Recueil de poésies kabyles introduced a consistent method for rendering emphatics, pharyngeals and long vowels in Latin characters — solving, in 19th-century print typography, precisely the mapping problem this model re-solves in the age of keyboard input.

He worked without a committee, without a standard, and against a tradition that had not yet decided whether Kabyle could be written at all. The orthographic rules he established were carried forward through Mammeri, Dallet, and into the INALCO standard that this model was trained to produce.

The naming is homage; it implies no endorsement by anyone.

Citation

@software{agbalu_boulifa_2026,
  title  = {Boulifa-48M: orthography standardisation for informal Kabyle Latin},
  author = {AGBALU},
  year   = {2026},
  url    = {https://huggingface.co/agbalu/Boulifa-48M},
  note   = {Trained on agbalu/KabStandard; 47.8M parameters; 99.45% dev char accuracy at epoch 2}
}

Licence

Apache-2.0 on the weights and the code. The training data derives from agbalu/KabTifinagh (agbalu/KabStandard is a derived dataset); a permissive grant on weights makes no claim about the text they were trained on, so read agbalu/KabTifinagh's licence before redistributing derivatives of the training corpus.

Part of AGBALU, a Kabyle corpus and model collection.