English card: live demo up top, tested Python example

README.md

@@ -3,75 +3,157 @@ license: mit
 language: [ar]
 library_name: onnx
 pipeline_tag: text-generation
-tags: [arabic, story-generation, scaling-laws, onnx]
+tags: [arabic, story-generation, scaling-laws, onnx, small-language-model]
 ---
 
-# RuqLM
+# RuqLM — small Arabic language models
 
-سلّم من خمسة نماذج لغة عربية، من **0.74M** إلى **29.89M** بارامتر، مدرَّبة من
-الصفر على كوربوس من **50,003 قصة عربية** بميزانية توكنات ثابتة (76.8M توكن).
+**▶ Try it live: [ruqlm.web.app](https://ruqlm.web.app)** — runs entirely in your
+browser, no server, no signup needed for guest access.
 
-الأوزان بصيغة ONNX **بدقة كاملة (fp32)** — بلا تكميم. قِيست الحيرة على مجموعة
-التحقّق وطابقت PyTorch إلى ثلاث خانات عشرية (9.2009 مقابل 9.2009)، فالمخرجات
-هي مخرجات النموذج الأصلي نفسه.
+A ladder of five Arabic language models, **0.74M to 29.89M parameters**, trained
+from scratch on a corpus of **50,003 Arabic stories** under a fixed budget of 76.8M
+training tokens. The tokenizer, the corpus and the weights were all built for this
+project.
 
-تعمل **داخل المتصفح** بلا خادم عبر ONNX Runtime Web.
+Weights are **full-precision ONNX** — no quantisation. Perplexity matches the
+original PyTorch checkpoints to three decimals (9.2009 vs 9.2009), so these are the
+trained models, not an approximation of them.
 
-## السلّم
+---
+
+## The ladder
 
-| النموذج | بارامتر | d_model | طبقات | رؤوس | الحجم |
+| Model | Parameters | d_model | Layers | Heads | Size |
 |---|---|---|---|---|---|
-| `ruq-30m.onnx`  | 29.89M | 512 | 8 | 8 | 114.8MB |
-| `ruq-15m.onnx`  | 13.77M | 384 | 6 | 6 | 53.1MB |
-| `ruq-5m.onnx`   | 5.31M  | 256 | 4 | 4 | 20.7MB |
-| `ruq-2m.onnx`   | 1.90M  | 128 | 4 | 4 | 7.7MB |
-| `ruq-0.7m.onnx` | 0.74M  | 64  | 4 | 4 | 3.2MB |
+| `ruq-30m.onnx`  | 29.89M | 512 | 8 | 8 | 114.8 MB |
+| `ruq-15m.onnx`  | 13.77M | 384 | 6 | 6 | 53.1 MB |
+| `ruq-5m.onnx`   | 5.31M  | 256 | 4 | 4 | 20.7 MB |
+| `ruq-2m.onnx`   | 1.90M  | 128 | 4 | 4 | 7.7 MB |
+| `ruq-0.7m.onnx` | 0.74M  | 64  | 4 | 4 | 3.2 MB |
+
+Decoder-only transformer: RMSNorm, RoPE, SwiGLU, tied embeddings, 512-token context.
+Tokenizer is a byte-level BPE with an 8,192 vocabulary trained on the same corpus
+(1.411 tokens per word).
+
+---
+
+## Usage
+
+Install `onnxruntime`, `tokenizers` and `huggingface_hub`, then:
+
+```python
+import re, unicodedata
+import numpy as np, onnxruntime as ort
+from huggingface_hub import hf_hub_download
+from tokenizers import Tokenizer
+
+REPO, EOS = "Ruqiya/ruqlm", 2
+
+tok  = Tokenizer.from_file(hf_hub_download(REPO, "tokenizer.json"))
+sess = ort.InferenceSession(hf_hub_download(REPO, "ruq-30m.onnx"))
+
+# The tokenizer was trained on normalised text: diacritics and tatweel
+# removed. Skipping this step gives a different, worse tokenisation.
+_DIACRITICS = re.compile(r"[ً-ْٰٓ-ٕ]")
+_ZERO_WIDTH = re.compile(r"[​-‏‪-‮]")
+
+def normalize(text):
+    text = unicodedata.normalize("NFC", text)
+    text = _ZERO_WIDTH.sub("", text).replace("ـ", "")
+    return re.sub(r"\s+", " ", _DIACRITICS.sub("", text)).strip()
+
+def generate(prompt, max_new_tokens=120, temperature=0.85, top_k=50, seed=None):
+    """The model outputs logits for the last position only; sampling is here."""
+    rng = np.random.default_rng(seed)
+    ids = tok.encode(normalize(prompt)).ids
+    if ids and ids[-1] == EOS:          # drop the trailing </s> so it continues
+        ids = ids[:-1]
+
+    for _ in range(max_new_tokens):
+        logits = sess.run(None, {"input_ids": np.array([ids], dtype=np.int64)})[0][0]
+        logits = logits.astype(np.float64) / temperature
+
+        kth = np.partition(logits, -top_k)[-top_k]      # top-k filter
+        logits[logits < kth] = -np.inf
 
-المعمارية: Transformer فك-تشفير فقط، مع RMSNorm و RoPE و SwiGLU وتضمينات
-مربوطة. طول السياق 512. المُرمِّز BPE بمفردات 8,192 مدرَّب على الكوربوس نفسه
-(1.41 توكن/كلمة).
+        probs = np.exp(logits - logits.max())
+        probs /= probs.sum()
 
-## الواجهة
+        nxt = int(rng.choice(len(probs), p=probs))
+        if nxt == EOS:
+            break
+        ids.append(nxt)
 
-الرسم البياني يأخذ `input_ids` بشكل `[1, seq]` ويُخرج `logits` للموضع الأخير
-وحده بشكل `[1, 8192]`. حلقة أخذ العيّنات خارج النموذج، فتغيير الحرارة أو
-top-k لا يتطلب إعادة تصدير.
+    return tok.decode(ids)
+
+print(generate("كان يا ما كان"))
+```
+
+Output (`ruq-5m`, seed 0):
+
+> كان يا ما كان في حديقة بيت. كان النجار يعمل بجد ليطعم عائلته. وفي يوم من الأيام،
+> وأثناء عمله، وجد النجار سلحفاة صغيرة ضائعة…
+
+### Interface
+
+The graph takes `input_ids` of shape `[1, seq]` and returns `logits` for the **last
+position only**, shape `[1, 8192]`. The sampling loop lives outside the model, so
+temperature and top-k can change without re-exporting.
+
+### In the browser
 
 ```js
 const sess = await ort.InferenceSession.create('ruq-30m.onnx');
-const out  = await sess.run({ input_ids: new ort.Tensor('int64', ids, [1, ids.length]) });
+const out  = await sess.run({
+  input_ids: new ort.Tensor('int64', BigInt64Array.from(ids, BigInt), [1, ids.length])
+});
 ```
 
-## حدود النموذج
+---
+
+## Limitations
 
-هذه نماذج **إكمال نصّ** (base)، لا نماذج محادثة. دُرِّبت على قصص عربية فقط،
-فتُكمل ما تبدأه ولا تُجيب عن أسئلة ولا تتبع تعليمات — لم تمرّ بأي ضبط تعليمات
-(SFT) أو RLHF. توقّع منها سرداً قصصياً، لا إجابات.
+These are **base text-completion models, not chat models.** They were trained on
+Arabic stories only, so they continue what you start; they do not answer questions or
+follow instructions, having had no instruction tuning (SFT) or RLHF. Expect
+narrative, not answers.
 
-النحو والصرف قويّان؛ التماسك الدلالي والمرجعي أضعف — وهو ما يُتوقَّع عند هذا
-الحجم، ولا تلتقطه مقاييس التنوّع وحدها.
+Morphology and orthography are strong. **Semantic and referential coherence are
+weaker** — which is what the accompanying study measures, and what these models were
+built to quantify rather than to solve. Even the largest rung abandons 5.9% of the
+characters it introduces, where the training corpus abandons 0.0%.
 
-الكوربوس مُولَّد بمعلّمَين: **ALLaM 2.7B** (75%) و **Qwen 3.6-27B** (25%)،
-مع بوّابات جودة ترفض المحارف الأجنبية والافتتاحيات القالبية.
+The corpus is synthetic, generated by two teacher models: **ALLaM 2.7B** (75%) and
+**Qwen3.6-27B** (25%), with quality gates rejecting foreign characters and template
+openings.
+
+---
 
-## التقييم
+## What the models are for
 
-| | الكوربوس | ruq-30m | ruq-0.7m |
-|---|---|---|---|
-| تنوّع المفردات (TTR) | 0.736 | 0.708 | 0.698 |
-| تكرار الـ 5-gram | 0.2% | 0.1% | 0.0% |
+They were built to answer a measurement question: across a 40× range of parameters
+at fixed data, which linguistic competence scales and which saturates?
 
-## التجربة
+| Axis | Behaviour across the ladder |
+|---|---|
+| Lexical diversity (TTR) | **saturated** — 0.719 ± 0.005 → 0.721 ± 0.005 (0.4σ) |
+| Character tracking | **scales** — 31.6% → 5.9% abandoned (6.3σ) |
+| Orthography, agreement | saturated below 0.74M |
+
+Capacity buys coherence, not vocabulary.
+
+---
 
-<!-- TODO: رابط arXiv عند النشر -->
-الورقة قيد الإعداد. تجربة حيّة: https://ruqlm.web.app
+## Paper
 
-## الاستشهاد
+<!-- TODO: arXiv link on release -->
+*In preparation.* Corpus, metrics, checkpoints and every measurement are released.
 
 ```bibtex
 @misc{binsafi2026ruqlm,
-  title  = {Ruq-LM: A Scaling Study of Arabic Language Models
-            with Morphology-Aware Evaluation},
+  title  = {What Scales in a Small Arabic Language Model?
+            Morphology Saturates, Coherence Does Not},
   author = {Bin Safi, Ruqiya},
   year   = {2026},
   note   = {Preprint}