Upload 29 files

src/quran_muaalem.egg-info/PKG-INFO

@@ -0,0 +1,357 @@
+Metadata-Version: 2.4
+Name: quran-muaalem
+Version: 0.0.3
+Summary: Quran Phonetic Script with addional quarnic utils
+Author-email: Abdullah <abdullahamlyossef@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/obadx/quran-muaalem
+Project-URL: Issues, https://github.com/obadx/quran-muaalem/issues
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: diff-match-patch>=20241021
+Requires-Dist: numpy>=2.2.6
+Requires-Dist: quran-transcript>=0.1.0
+Requires-Dist: rich>=14.1.0
+Requires-Dist: torch>=2.7.0
+Requires-Dist: transformers>=4.55.0
+Provides-Extra: test
+Requires-Dist: librosa>=0.11.0; extra == "test"
+Requires-Dist: numba>=0.61.2; extra == "test"
+Requires-Dist: pytest; extra == "test"
+Provides-Extra: ui
+Requires-Dist: gradio>=5.43.1; extra == "ui"
+Requires-Dist: librosa>=0.11.0; extra == "ui"
+Requires-Dist: numba>=0.61.2; extra == "ui"
+Requires-Dist: moviepy>=2.2.1; extra == "ui"
+Dynamic: license-file
+
+# Quran Muaalem
+
+<div align="center">
+<strong>بعون الله وتوفيقه لا شريك له نقدم المعلم القرآني الذكي القادر على كشف أخطاء التلاوة والتجويد وصفات الحروف</strong>
+
+[![PyPI][pypi-badge]][pypi-url]
+[![Python Versions][python-badge]][python-url]
+[![Hugging Face Model][hf-model-badge]][hf-model-url]
+[![Hugging Face Dataset][hf-dataset-badge]][hf-dataset-url]
+[![Google Colab][colab-badge]][colab-url]
+[![arXiv][arxiv-badge]][arxiv-url]
+[![MIT License][mit-badge]][mit-url]
+[![Discord][discord-badge]][discord-url]
+
+</div>
+
+[pypi-badge]: https://img.shields.io/pypi/v/quran-muaalem.svg
+[pypi-url]: https://pypi.org/project/quran-muaalem/
+[mit-badge]: https://img.shields.io/github/license/obadx/quran-muaalem.svg
+[mit-url]: https://github.com/obadx/quran-muaalem/blob/main/LICENSE
+[python-badge]: https://img.shields.io/pypi/pyversions/quran-muaalem.svg
+[python-url]: https://pypi.org/project/quran-muaalem/
+[colab-badge]: https://img.shields.io/badge/Google%20Colab-Open%20in%20Colab-F9AB00?logo=google-colab&logoColor=white
+[colab-url]: https://colab.research.google.com/drive/1If0G9NtdXiSRu6PVGtIMvLwxizF2jspn?usp=sharing
+[hf-model-badge]: https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue
+[hf-model-url]: https://huggingface.co/obadx/muaalem-model-v3_0
+[hf-dataset-badge]: https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-orange
+[hf-dataset-url]: https://huggingface.co/datasets/obadx/muaalem-annotated-v3
+[arxiv-badge]: https://img.shields.io/badge/arXiv-Paper-<COLOR>.svg
+[arxiv-url]: https://arxiv.org/abs/2509.00094
+[discord-badge]: https://img.shields.io/badge/Discord-Join%20Community-7289da?logo=discord&logoColor=white
+[discord-url]: https://discord.gg/hJWW6fCH
+
+<div align="center" style="background-color: #f0f8ff; border-left: 5px solid #4CAF50; padding: 15px; margin: 20px 0; border-radius: 5px;">
+  <h3 style="color: #2c3e50; margin-top: 0;">📖 رابط لتجربة المعلم القرآني</h3>
+  <p style="margin: 10px 0;">يرجى الضغط على للتجربة:</p>
+  <a href="https://662a040e1863a5445c.gradio.live" style="display: inline-block; background-color: #4CAF50; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px; font-weight: bold; margin: 10px 0;">الرابط</a>
+  <p style="background-color: #ffeb3b; padding: 8px; border-radius: 3px; display: inline-block; margin: 10px 0;">
+    ⚠️ <strong>تنبيه:</strong> هذا الرابط سينتهي في <span style="color: #d32f2f; font-weight: bold;">27 أغسطس 2025</span>
+  </p>
+</div>
+
+[![ALT_TEXT](https://img.youtube.com/vi/CsFoznO08-Q/0.jpg)](https://www.youtube.com/watch?v=CsFoznO08-Q)
+
+
+## الممزيات
+
+* مدرب على الرسم الصوتي للقرآن الكريم: [quran-transcript](https://github.com/obadx/quran-transcript) القادر على كشف أخطاء الحروف والتجويد وصفات الحروف
+* نموذج معقول الحجم 660 MP 
+* يحتاج فقط إله 1.5 GB من ذاكرة معالج الرسوميات
+* معمارية مبتكرة: CTC متعدد المستويات
+
+## المعمارية
+معمارية مبتكرة: CTC متعدد المستويات. حيث كل مستوي يتدرب على وجه معين
+
+![multi-lvel-ctc](./assets/figures/mutli-level-ctc.png)
+
+## الخطوات المختصرة للتطوير
+
+* تجميع التلاوت القرآنية من القراء المتقنين: [prepare-quran-dataset](https://github.com/obadx/prepare-quran-dataset)
+* تقسيم التلاوت على حسب الوقف وليس الآية باستخدام [المقسم](https://github.com/obadx/recitations-segmenter)
+* الحصو على النص القرآني من المقاطع الصوتية باسخدام [نموذج ترتيل](https://huggingface.co/tarteel-ai/whisper-base-ar-quran)
+* تصحيح النصوص المستخرجة من ترتيل باستخدام  [خوارزمية التسميع](https://github.com/obadx/quran-transcript)
+* تحويل الرسم الإملائي للرسم العثماني: [quran-transcript](https://github.com/obadx/quran-transcript)
+* تحويل الرسم العثماني للرسم الصوتي للقرآني الكريم الذي يصف كل قواعد التجويد ما عدا الإشمام: [quran-transcript](https://github.com/obadx/quran-transcript)
+* تدريب النموذج على معمارية [Wav2Vec2BERT](https://huggingface.co/docs/transformers/model_doc/wav2vec2-bert)
+
+
+## استخدام النوذج
+
+
+### استخدام النموذج عن طريق واجهة gradio
+
+قم بتزيل  [uv](https://docs.astral.sh/uv/) 
+
+```bash
+pip install uv
+```
+أو
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+بعد ذلك قم بتنزيل `ffmpeg`
+
+```bash
+sudo apt-get update
+sudo apt-get install -y ffmpeg
+```
+
+أو من خلال `anaconda`
+```bash
+conda install ffmpeg
+```
+
+قم بتشغيل `gradio` ب command واحد فقط:
+```bash
+uvx --no-cache --from https://github.com/obadx/quran-muaalem.git[ui]  quran-muaalem-ui
+```
+او
+```bash
+uvx quran-muaalem[ui]  quran-muaalem-ui
+```
+
+### عن طريق python API
+
+
+#### Installation
+
+First, install the required dependencies:
+
+```bash
+# Install system dependencies
+sudo apt-get install -y ffmpeg libsndfile1 portaudio19-dev
+
+# Install Python packages
+pip install quran-muaalem librosa "numba>=0.61.2"
+```
+
+## Basic Usage Example
+
+```python
+"""
+Basic example of using the Quran Muaalem package for phonetic analysis of Quranic recitation.
+"""
+
+from dataclasses import asdict
+import json
+import logging
+
+from quran_transcript import Aya, quran_phonetizer, MoshafAttributes
+import torch
+from librosa.core import load
+
+# Import the main Muaalem class (adjust import based on your actual package structure)
+from quran_muaalem import Muaalem
+
+# Setup logging to see informative messages
+logging.basicConfig(level=logging.INFO)
+
+def analyze_recitation(audio_path):
+    """
+    Analyze a Quranic recitation audio file using the Muaalem model.
+    
+    Args:
+        audio_path (str): Path to the audio file to analyze
+    """
+    # Configuration
+    sampling_rate = 16000  # Must be 16000 Hz
+    device = "cuda" if torch.cuda.is_available() else "cpu"  # Use GPU if available
+    
+    # Step 1: Prepare the Quranic reference text
+    # Get the Uthmani script for a specific verse (Aya 8, Surah 75 in this example)
+    uthmani_ref = Aya(8, 75).get_by_imlaey_words(17, 9).uthmani
+    
+    # Step 2: Configure the recitation style (Moshaf attributes)
+    moshaf = MoshafAttributes(
+        rewaya="hafs",        # Recitation style (Hafs is most common)
+        madd_monfasel_len=2,  # Length of separated elongation
+        madd_mottasel_len=4,  # Length of connected elongation
+        madd_mottasel_waqf=4, # Length of connected elongation when stopping
+        madd_aared_len=2,     # Length of necessary elongation
+    )
+    # see: https://github.com/obadx/prepare-quran-dataset?tab=readme-ov-file#moshaf-attributes-docs
+    
+    # Step 3: Convert text to phonetic representation
+    # see docs for phnetizer: https://github.com/obadx/quran-transcript
+    phonetizer_out = quran_phonetizer(uthmani_ref, moshaf, remove_spaces=True)
+    
+    # Step 4: Initialize the Muaalem model
+    muaalem = Muaalem(device=device)
+    
+    # Step 5: Load and prepare the audio
+    wave, _ = load(audio_path, sr=sampling_rate, mono=True)
+    
+    # Step 6: Process the audio with the model
+    # The model analyzes the phonetic properties of the recitation
+    outs = muaalem(
+        [wave],           # Audio data
+        [phonetizer_out],          # Phonetic reference
+        sampling_rate=sampling_rate
+    )
+    
+    # Step 7: Display the results
+    for out in outs:
+        print("Predicted Phonemes:", out.phonemes.text)
+        
+        # Display detailed phonetic features for each phoneme
+        for sifa in out.sifat:
+            print(json.dumps(asdict(sifa), indent=2, ensure_ascii=False))
+            print("*" * 30)
+        print("-" * 40)
+
+    # Explaining Results
+    explain_for_terminal(
+        outs[0].phonemes.text,
+        phonetizer_out.phonemes,
+        outs[0].sifat,
+        phonetizer_out.sifat,
+    )
+
+
+if __name__ == "__main__":
+    # Replace with the path to your audio file
+    audio_path = "./assets/test.wav"
+    
+    try:
+        analyze_recitation(audio_path)
+    except Exception as e:
+        logging.error(f"Error processing audio: {e}")
+```
+
+Output:
+
+```bash
+ءِننننَللَااهَبِكُللِشَيءِنعَلِۦۦمُ۾۾۾بَرَااااءَتُممممِنَللَااهِوَرَسُۥۥلِه
+┏━━━━━━━━━━┳━━━━━��━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
+┃ Phonemes ┃ Tafashie       ┃ Qalqla       ┃ Ghonna       ┃ Hams Or Jahr ┃ Safeer    ┃ Tikraar     ┃ Tafkheem Or Taqeeq ┃ Istitala      ┃ Shidda Or Rakhawa ┃ Itbaq    ┃
+┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
+│ ءِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ ننننَ     │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ للَ       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ mofakham           │ not_mostateel │ between           │ monfateh │
+│ اا       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ mofakham           │ not_mostateel │ rikhw             │ monfateh │
+│ هَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ بِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ كُ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ للِ       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ شَ        │ motafashie     │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ ي        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ ءِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ ن        │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ عَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ لِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ ۦۦ       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ مُ        │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ ۾۾۾      │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ بَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ رَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ mokarar     │ mofakham           │ not_mostateel │ between           │ monfateh │
+│ اااا     │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ mofakham           │ not_mostateel │ rikhw             │ monfateh │
+│ ءَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ تُ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ shadeed           │ monfateh │
+│ ممممِ     │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ نَ        │ not_motafashie │ not_moqalqal │ maghnoon     │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ للَ       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ mofakham           │ not_mostateel │ between           │ monfateh │
+│ اا       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ mofakham           │ not_mostateel │ rikhw             │ monfateh │
+│ هِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ وَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ رَ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ mokarar     │ mofakham           │ not_mostateel │ between           │ monfateh │
+│ سُ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ safeer    │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ ۥۥ       │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+│ لِ        │ not_motafashie │ not_moqalqal │ not_maghnoon │ jahr         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ between           │ monfateh │
+│ ه        │ not_motafashie │ not_moqalqal │ not_maghnoon │ hams         │ no_safeer │ not_mokarar │ moraqaq            │ not_mostateel │ rikhw             │ monfateh │
+└──────────┴────────────────┴──────────────┴──────────────┴──────────────┴───────────┴─────────────┴────────────────────┴───────────────┴───────────────────┴──────────┘
+```
+
+### API Docs
+
+```python
+class Muaalem:
+    def __init__(
+        self,
+        model_name_or_path: str = "obadx/muaalem-model-v3_2",
+        device: str = "cpu",
+        dtype=torch.bfloat16,
+    ):
+        """
+        Initializing Muallem Model
+
+        Args:
+            model_name_or_path: the huggingface model name or path
+            device: the device to run model on
+            dtype: the torch dtype. Default is `torch.bfloat16` as the model was trained on
+        """
+
+    @torch.no_grad()
+    def __call__(
+        self,
+        waves: list[list[float] | torch.FloatTensor | NDArray],
+        ref_quran_phonetic_script_list: list[QuranPhoneticScriptOutput],
+        sampling_rate: int,
+    ) -> list[MuaalemOutput]:
+        """Infrence Funcion for the Quran Muaalem Project
+
+                waves: input waves  batch , seq_len with different formats described above
+                ref_quran_phonetic_script_list (list[QuranPhoneticScriptOutput]): list of the
+                    phonetized ouput of `quran_transcript.quran_phonetizer` with `remove_space=True`
+
+                sampleing_rate (int): has to be 16000
+
+        Returns:
+            list[MuaalemOutput]:
+                A list of output objects, each containing phoneme predictions and their
+                phonetic features (sifat) for a processed input.
+
+            Each MuaalemOutput contains:
+                phonemes (Unit):
+                    A dataclass representing the predicted phoneme sequence with:
+                        text (str): Concatenated string of all phonemes.
+                        probs (Union[torch.FloatTensor, list[float]]):
+                            Confidence probabilities for each predicted phoneme.
+                        ids (Union[torch.LongTensor, list[int]]):
+                            Token IDs corresponding to each phoneme.
+
+                sifat (list[Sifa]):
+                    A list of phonetic feature dataclasses (one per phoneme) with the
+                    following optional properties (each is a SingleUnit or None):
+                        - phonemes_group (str): the phonemes associated with the `sifa`
+                        - hams_or_jahr (SingleUnit): either `hams` or `jahr`
+                        - shidda_or_rakhawa (SingleUnit): either `shadeed`, `between`, or `rikhw`
+                        - tafkheem_or_taqeeq (SingleUnit): either `mofakham`, `moraqaq`, or `low_mofakham`
+                        - itbaq (SingleUnit): either `monfateh`, or `motbaq`
+                        - safeer (SingleUnit): either `safeer`, or `no_safeer`
+                        - qalqla (SingleUnit): eithr `moqalqal`, or `not_moqalqal`
+                        - tikraar (SingleUnit): either `mokarar` or `not_mokarar`
+                        - tafashie (SingleUnit): either `motafashie`, or `not_motafashie`
+                        - istitala (SingleUnit): either `mostateel`, or `not_mostateel`
+                        - ghonna (SingleUnit): either `maghnoon`, or `not_maghnoon`
+
+            Each SingleUnit in Sifa properties contains:
+                text (str): The feature's categorical label (e.g., "hams", "shidda").
+                prob (float): Confidence probability for this feature.
+                idx (int): Identifier for the feature class.
+        """
+```

src/quran_muaalem.egg-info/SOURCES.txt

@@ -0,0 +1,26 @@
+LICENSE
+README.md
+pyproject.toml
+src/quran_muaalem/__init__.py
+src/quran_muaalem/decode.py
+src/quran_muaalem/explain.py
+src/quran_muaalem/explain_gradio.py
+src/quran_muaalem/gradio_app.py
+src/quran_muaalem/inference.py
+src/quran_muaalem/muaalem_typing.py
+src/quran_muaalem.egg-info/PKG-INFO
+src/quran_muaalem.egg-info/SOURCES.txt
+src/quran_muaalem.egg-info/dependency_links.txt
+src/quran_muaalem.egg-info/entry_points.txt
+src/quran_muaalem.egg-info/requires.txt
+src/quran_muaalem.egg-info/top_level.txt
+src/quran_muaalem/modeling/__init__.py
+src/quran_muaalem/modeling/configuration_multi_level_ctc.py
+src/quran_muaalem/modeling/modeling_multi_level_ctc.py
+src/quran_muaalem/modeling/multi_level_tokenizer.py
+src/quran_muaalem/modeling/vocab.py
+tests/test_align_phonemes.py
+tests/test_best_match.py
+tests/test_exaplain_modules.py
+tests/test_modules.py
+tests/test_muaalem_infrence.py

src/quran_muaalem.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

src/quran_muaalem.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+quran-muaalem-ui = quran_muaalem.gradio_app:main

src/quran_muaalem.egg-info/requires.txt

@@ -0,0 +1,17 @@
+diff-match-patch>=20241021
+numpy>=2.2.6
+quran-transcript>=0.1.0
+rich>=14.1.0
+torch>=2.7.0
+transformers>=4.55.0
+
+[test]
+librosa>=0.11.0
+numba>=0.61.2
+pytest
+
+[ui]
+gradio>=5.43.1
+librosa>=0.11.0
+numba>=0.61.2
+moviepy>=2.2.1

src/quran_muaalem.egg-info/top_level.txt

@@ -0,0 +1 @@
+quran_muaalem

src/quran_muaalem/__init__.py

@@ -0,0 +1,13 @@
+from .inference import Muaalem
+from .muaalem_typing import MuaalemOutput, Unit, Sifa, SingleUnit
+from .explain import explain_for_terminal
+
+
+__all__ = [
+    "Muaalem",
+    "MuaalemOutput",
+    "Unit",
+    "Sifa",
+    "SingleUnit",
+    "explain_for_terminal",
+]

src/quran_muaalem/decode.py

@@ -0,0 +1,580 @@
+import logging
+from typing import Sequence, Any
+from dataclasses import dataclass
+import torch
+import numpy as np
+from numpy.typing import NDArray
+
+from .modeling.vocab import PAD_TOKEN_IDX
+from .muaalem_typing import Unit
+
+
+# def align_predicted_sequence(
+#     ref: Sequence[Any], predicted: Sequence[Any]
+# ) -> Sequence[Any]:
+#     """Aligns the preficeted sequence to the ref sequnce
+#
+#     Example (1): `predicted` length > `ref` length
+#         ref: abcde
+#         predicted: abcdef
+#         Returns: abcde
+#
+#     Example (2): `predicted` length <`ref` length
+#         ref: abcde
+#         predicted: abcd
+#         Returns: abcde
+#
+#     Returns:
+#         Sequnce[Any]: new precicted sequence that best matches the ref sequence
+#     """
+#     n = len(ref)
+#     m = len(predicted)
+#     if n == m:
+#         return predicted
+#     if n == 0:
+#         return []
+#     if m == 0:
+#         return ref
+#
+#     dp = [[0] * (m + 1) for _ in range(n + 1)]
+#
+#     for i in range(1, n + 1):
+#         dp[i][0] = 0
+#     for j in range(1, m + 1):
+#         dp[0][j] = 0
+#
+#     for i in range(1, n + 1):
+#         for j in range(1, m + 1):
+#             insertion = dp[i - 1][j]
+#             deletion = dp[i][j - 1]
+#             match_cost = dp[i - 1][j - 1] + (1 if ref[i - 1] != predicted[j - 1] else 0)
+#             dp[i][j] = min(insertion, deletion, match_cost)
+#
+#     i, j = n, m
+#     output_chars = []
+#     while i > 0 or j > 0:
+#         if (
+#             i > 0
+#             and j > 0
+#             and ref[i - 1] == predicted[j - 1]
+#             and dp[i][j] == dp[i - 1][j - 1]
+#         ):
+#             output_chars.append(predicted[j - 1])
+#             i -= 1
+#             j -= 1
+#         elif i > 0 and dp[i][j] == dp[i - 1][j]:
+#             output_chars.append(ref[i - 1])
+#             i -= 1
+#         elif j > 0 and dp[i][j] == dp[i][j - 1]:
+#             j -= 1
+#         else:
+#             output_chars.append(predicted[j - 1])
+#             i -= 1
+#             j -= 1
+#
+#     # return "".join(output_chars[::-1])
+#     return output_chars[::-1]
+
+
+# def align_predicted_sequence(ref, predicted):
+#     n = len(ref)
+#     m = len(predicted)
+#     if m == n:
+#         return predicted
+#
+#     INF = 10**9
+#     dp = [[0] * (m + 1) for _ in range(n + 1)]
+#     choice = [[0] * (m + 1) for _ in range(n + 1)]
+#
+#     for j in range(m + 1):
+#         dp[0][j] = 0
+#
+#     for i in range(1, n + 1):
+#         dp[i][0] = INF
+#
+#     for i in range(1, n + 1):
+#         for j in range(1, m + 1):
+#             # above
+#             option1 = dp[i][j - 1]
+#             # adjacent
+#             if j >= i:
+#                 cost = 0 if predicted[j - 1] == ref[i - 1] else 1
+#                 option2 = dp[i - 1][j - 1] + cost
+#             else:
+#                 option2 = INF
+#
+#             if option2 <= option1:
+#                 dp[i][j] = option2
+#                 choice[i][j] = 1
+#             else:
+#                 dp[i][j] = option1
+#                 choice[i][j] = 0
+#
+#     print(np.array(dp))
+#     print(np.array(choice))
+#
+#     res_chars = []
+#     i, j = n, m
+#     while i > 0 and j > 0:
+#         if choice[i][j] == 1:
+#             res_chars.append(predicted[j - 1])
+#             i -= 1
+#             j -= 1
+#         else:
+#             j -= 1
+#
+#     return res_chars[::-1]
+
+
+def align_chunked_phonemes_sequence(
+    ref: list[list[str]],
+    predicted: list[list[str]],
+) -> list[bool]:
+    """Aligns phonemes level to get mask that descripts what is missing
+
+    Returns the mask for the `ref` inputs that best matches the `predicted`
+    Note element wise comparison but retuns mask for best seqence (even with errors)
+
+    Example (1): `predicted` length > `ref` length
+        ref: abcde
+        predicted: abcdef
+        Returns: [T, T, T, T]
+
+    Example (2): `predicted` length <`ref` length
+        ref: abcde
+        predicted: abcd
+        Returns: [T, T, T, T, F]
+
+    Example (2): `predicted` length <`ref` length
+        ref: afcde
+        predicted: abcd
+        Returns: [T, T, T, T, F]
+
+
+    Len(mask] == Len(ref)
+
+    """
+
+    n = len(predicted)
+    m = len(ref)
+
+    if len(predicted) == len(ref):
+        return [True] * len(predicted)
+
+    if m == 0:
+        raise ValueError("`ref` length must not be zero length")
+
+    dp = [[0] * (m + 1) for _ in range(n + 1)]
+    choice = [[0] * (m + 1) for _ in range(n + 1)]
+
+    for j in range(m + 1):
+        dp[0][j] = 0
+
+    for i in range(1, n + 1):
+        dp[i][0] = i
+
+    for i in range(1, n + 1):
+        for j in range(1, m + 1):
+            option1 = dp[i][j - 1]
+            option2 = dp[i - 1][j] + 1
+            cost = 0 if predicted[i - 1][0] == ref[j - 1][0] else 1
+            option3 = dp[i - 1][j - 1] + cost
+
+            if option3 <= option1 and option3 <= option2:
+                dp[i][j] = option3
+                choice[i][j] = 3
+            elif option1 <= option2:
+                dp[i][j] = option1
+                choice[i][j] = 1
+            else:
+                dp[i][j] = option2
+                choice[i][j] = 2
+
+    i = n
+    j = m
+    mask = []
+    # res_chars = []
+    while i > 0 or j > 0:
+        if i > 0 and j > 0:
+            if choice[i][j] == 3:
+                # res_chars.append(ref[j - 1])
+                mask.append(True)
+                i -= 1
+                j -= 1
+            elif choice[i][j] == 2:
+                # res_chars.append(missing_placeholder)
+                i -= 1
+            else:
+                j -= 1
+                mask.append(False)
+        elif i > 0:
+            # res_chars.append(missing_placeholder)
+            i -= 1
+        else:
+            j -= 1
+            mask.append(False)
+
+    return mask[::-1]
+
+
+def align_predicted_sequence(
+    ref: Sequence[Any] | torch.LongTensor,
+    predicted: Sequence[Any] | torch.LongTensor,
+    missing_placeholder=-100,
+) -> Sequence[Any] | torch.LongTensor:
+    """Aligns the preficeted sequence to the ref sequnce
+
+    Example (1): `predicted` length > `ref` length
+        ref: abcde
+        predicted: abcdef
+        Returns: abcde
+
+    Example (2): `predicted` length <`ref` length
+        ref: abcde
+        predicted: abcd
+        Returns: abcd(missing_placeholder)
+
+    Returns:
+        Sequnce[Any]: new precicted sequence that best matches the ref sequence
+    """
+
+    n = len(ref)
+    m = len(predicted)
+
+    if len(ref) == len(predicted):
+        return predicted, [True] * len(ref)
+
+    if m == 0:
+        return [missing_placeholder] * n
+
+    dp = [[0] * (m + 1) for _ in range(n + 1)]
+    choice = [[0] * (m + 1) for _ in range(n + 1)]
+
+    for j in range(m + 1):
+        dp[0][j] = 0
+
+    for i in range(1, n + 1):
+        dp[i][0] = i
+
+    for i in range(1, n + 1):
+        for j in range(1, m + 1):
+            option1 = dp[i][j - 1]
+            option2 = dp[i - 1][j] + 1
+            cost = 0 if ref[i - 1] == predicted[j - 1] else 1
+            option3 = dp[i - 1][j - 1] + cost
+
+            if option3 <= option1 and option3 <= option2:
+                dp[i][j] = option3
+                choice[i][j] = 3
+            elif option1 <= option2:
+                dp[i][j] = option1
+                choice[i][j] = 1
+            else:
+                dp[i][j] = option2
+                choice[i][j] = 2
+
+    i = n
+    j = m
+    mask = []
+    res_chars = []
+    while i > 0 or j > 0:
+        if i > 0 and j > 0:
+            if choice[i][j] == 3:
+                res_chars.append(predicted[j - 1])
+                mask.append(True)
+                i -= 1
+                j -= 1
+            elif choice[i][j] == 2:
+                res_chars.append(missing_placeholder)
+                i -= 1
+            else:
+                j -= 1
+                mask.append(False)
+        elif i > 0:
+            res_chars.append(missing_placeholder)
+            i -= 1
+        else:
+            j -= 1
+            mask.append(False)
+
+    return res_chars[::-1], mask[::-1]
+
+
+@dataclass
+class CTCDecodeOut:
+    """
+    Both are 1D Tensors
+    """
+
+    ids: torch.LongTensor
+    p: torch.FloatTensor
+
+
+def ctc_decode(
+    batch_ids: torch.LongTensor,
+    batch_probs: torch.FloatTensor,
+    blank_id=PAD_TOKEN_IDX,
+    collapse_consecutive=True,
+) -> list[CTCDecodeOut]:
+    """
+    batch_ids (torch.LongTensor): batch on integer ids of shape: batch, sequecne_len
+    batch_probs (torch.LongTensor): batch on float32 ids of shape: batch, sequecne_len
+
+    Return:
+        list[tuple[list[int], float]]]:
+
+
+    """
+    outs = []
+    assert batch_ids.shape == batch_probs.shape
+    for seq_idx, seq in enumerate(batch_ids):
+        if collapse_consecutive:
+            tokens = []
+            probs = []
+            start = 0
+            end = 0
+            if len(seq) == 1 and seq[0] != blank_id:
+                tokens.append(seq[0])
+                probs.append(batch_probs[seq_idx][0])
+
+            for idx in range(len(seq) - 1):
+                curr = seq[idx]
+                next = seq[idx + 1]
+                # Last Item
+                if idx == len(seq) - 2 and curr != blank_id:
+                    if curr == next:
+                        end = idx + 2
+                        tokens.append(curr)
+                        probs.append(
+                            batch_probs[seq_idx][start:end].sum() / (end - start)
+                        )
+                    elif curr != next:
+                        end = idx + 1
+                        tokens.append(curr)
+                        probs.append(
+                            batch_probs[seq_idx][start:end].sum() / (end - start)
+                        )
+                        tokens.append(next)
+                        probs.append(batch_probs[seq_idx][idx + 1])
+                # Normal Case
+                elif curr != next and curr != blank_id:
+                    end = idx + 1
+                    tokens.append(curr)
+                    probs.append(batch_probs[seq_idx][start:end].sum() / (end - start))
+                    start = end
+                elif curr == blank_id:
+                    start = idx + 1
+
+            outs.append(
+                CTCDecodeOut(
+                    ids=torch.LongTensor(tokens),
+                    p=torch.FloatTensor(probs),
+                )
+            )
+        else:
+            mask = seq != blank_id
+            tokens = seq[mask]
+            probs = batch_probs[seq_idx][mask]
+            outs.append(CTCDecodeOut(ids=tokens, p=probs))
+    return outs
+
+
+# def multilevel_greedy_decode(
+#     level_to_probs: dict[str, torch.FloatTensor],
+#     level_to_id_to_vocab: dict[str, dict[int, str]],
+#     level_to_ref_ids: dict[str, torch.LongTensor],
+#     missing_placeholder=-100,
+#     pad_idx=PAD_TOKEN_IDX,
+# ) -> dict[str, list[Unit]]:
+#     level_to_units = {}
+#     for level in level_to_probs:
+#         batch_probs, batch_ids = level_to_probs[level].topk(1, dim=-1)
+#         decode_outs = ctc_decode(
+#             batch_ids.squeeze(-1), batch_probs.squeeze(-1), collapse_consecutive=True
+#         )
+#         level_to_units[level] = []
+#         for seq_idx, decode_out in enumerate(decode_outs):
+#             # Trying to align Ids of the sifat levels
+#             if level != "phonemes":
+#                 aligned_ids, mask = align_predicted_sequence(
+#                     level_to_ref_ids[level][seq_idx],
+#                     decode_out.ids,
+#                     missing_placeholder=missing_placeholder,
+#                 )
+#             else:
+#                 aligned_ids = decode_out.ids
+#
+#             probs = decode_out.p
+#             if len(aligned_ids) != len(decode_out.ids):
+#                 aligned_ids = torch.LongTensor(aligned_ids)
+#                 mask = torch.BoolTensor(mask)
+#
+#                 new_probs = torch.zeros(len(aligned_ids), dtype=torch.float32)
+#                 new_probs[aligned_ids != missing_placeholder] = probs[mask]
+#
+#                 aligned_ids[aligned_ids == missing_placeholder] = pad_idx
+#                 probs = new_probs
+#
+#             probs = decode_out.p.clone()
+#             text = ""
+#             for idx in aligned_ids:
+#                 text += level_to_id_to_vocab[level][int(idx)]
+#             level_to_units[level].append(
+#                 Unit(
+#                     text=text,
+#                     probs=probs,
+#                     ids=aligned_ids,
+#                 ),
+#             )
+#
+#     return level_to_units
+
+
+def phonemes_level_greedy_decode(
+    probs: torch.FloatTensor,
+    phonemes_level_vocab: dict[int, str],
+) -> list[Unit]:
+    """Decodes only phonemes level
+
+    Args:
+        probs (torch.FloatTensor) of shape batch, seq_len, num_classes
+        phonmes_level_vocab (dict[int, str]): mapping ids of phonemes to the
+            acutial string represnetation
+    """
+    batch_probs, batch_ids = probs.topk(1, dim=-1)
+    decode_outs = ctc_decode(
+        batch_ids.squeeze(-1), batch_probs.squeeze(-1), collapse_consecutive=True
+    )
+    units = []
+    for seq_idx, decode_out in enumerate(decode_outs):
+        text = ""
+        for idx in decode_out.ids:
+            text += phonemes_level_vocab[int(idx)]
+        units.append(
+            Unit(
+                text=text,
+                probs=decode_out.p,
+                ids=decode_out.ids,
+            ),
+        )
+    return units
+
+
+def multilevel_greedy_decode(
+    level_to_probs: dict[str, torch.FloatTensor],
+    level_to_id_to_vocab: dict[str, dict[int, str]],
+    level_to_ref_ids: dict[str, torch.LongTensor],
+    chunked_phonemes_batch: list[list[str]],
+    ref_chuncked_phonemes_batch: list[list[str]],
+    phonemes_units: list[Unit],
+    missing_placeholder=-100,
+    pad_idx=PAD_TOKEN_IDX,
+) -> dict[str, list[Unit]]:
+    level_to_units = {}
+    for level in level_to_probs:
+        if level == "phonemes":
+            continue
+        batch_probs, batch_ids = level_to_probs[level].topk(1, dim=-1)
+        decode_outs = ctc_decode(
+            batch_ids.squeeze(-1), batch_probs.squeeze(-1), collapse_consecutive=True
+        )
+        level_to_units[level] = []
+        for seq_idx, decode_out in enumerate(decode_outs):
+            # Trying to align Ids of the sifat levels
+            phonemes_mask = align_chunked_phonemes_sequence(
+                ref=ref_chuncked_phonemes_batch[seq_idx],
+                predicted=chunked_phonemes_batch[seq_idx],
+            )
+            phonemes_mask = torch.BoolTensor(phonemes_mask)
+
+            # NOTE:
+            # We want to align every level with predited phonme, but
+            # in some cases the length of every sifa level is > or < the
+            # length for the predited phonemes
+            # we slove this by two steps
+            # 1. Align the sifa level with length mismatch to the refrence sifa level
+            # 2. align the alinged sifa level back to the the length of prediced phonmes
+            if len(decode_out.ids) != len(chunked_phonemes_batch[seq_idx]) and (
+                len(chunked_phonemes_batch[seq_idx])
+                <= len(ref_chuncked_phonemes_batch[seq_idx])
+            ):
+                logging.info(f"Sequence: `{seq_idx}` has mismatch Level: {level}")
+                # 1. Align sifa level to the reference sifa level
+                ref_aligned_ids, mask = align_predicted_sequence(
+                    level_to_ref_ids[level][seq_idx],
+                    decode_out.ids,
+                    missing_placeholder=missing_placeholder,
+                )
+
+                probs = decode_out.p
+                ref_aligned_ids = torch.LongTensor(ref_aligned_ids)
+                mask = torch.BoolTensor(mask)
+
+                new_probs = torch.zeros(len(ref_aligned_ids), dtype=torch.float32)
+                new_probs[ref_aligned_ids != missing_placeholder] = probs[mask]
+
+                ref_aligned_ids[ref_aligned_ids == missing_placeholder] = pad_idx
+
+                # 2. Align the predicted aligned to the ref back to the predicted seqence
+                aligned_ids = ref_aligned_ids[phonemes_mask]
+                new_probs = ref_aligned_ids[phonemes_mask]
+
+                probs = new_probs
+            else:
+                aligned_ids = decode_out.ids
+                probs = decode_out.p
+
+            text = ""
+            for idx in aligned_ids:
+                text += level_to_id_to_vocab[level][int(idx)]
+            level_to_units[level].append(
+                Unit(
+                    text=text,
+                    probs=probs,
+                    ids=aligned_ids,
+                ),
+            )
+    level_to_units["phonemes"] = phonemes_units
+
+    return level_to_units
+
+
+def align_sequence(
+    seq: Sequence[int] | torch.LongTensor, target_len: int, min_repeat: int = 3
+) -> list[int]:
+    """Aligns a sequence by removing items from the longest repateted items
+
+    Returns:
+        list[int]: the ids which are goning to be deleted if longest_repeat > len(seq) - target_len
+
+    Example:
+                seq = [1, 0, 1, 0, 0, 0, 0, 1], target_len = 7
+                                ^  ^  ^
+    Longest Repeat              ^  ^  ^
+    Ouput: [3]
+    """
+
+    if len(seq) <= target_len:
+        return []
+
+    longest_start = 0
+    longest_repeat = 0
+    curr_repeat = 1
+    curr_start = 0
+    for idx in range(len(seq) - 1):
+        curr = seq[idx]
+        next = seq[idx + 1]
+        if curr == next:
+            curr_repeat += 1
+        if (curr != next) or (idx == len(seq) - 2):
+            if curr_repeat > longest_repeat and curr_repeat >= min_repeat:
+                longest_repeat = curr_repeat
+                longest_start = curr_start
+            curr_start = idx + 1
+            curr_repeat = 1
+
+    # logical case to remote only from the longest repeat
+    if longest_repeat > len(seq) - target_len:
+        return list(range(longest_start, longest_start + len(seq) - target_len))
+    else:
+        return []

src/quran_muaalem/explain.py

@@ -0,0 +1,250 @@
+from dataclasses import dataclass, asdict
+from typing import Literal
+import json
+
+from quran_transcript import SifaOutput
+import quran_transcript.alphabet as alph
+import diff_match_patch as dmp
+from rich import print
+from rich.text import Text
+from rich.table import Table
+from rich.console import Console
+
+from .muaalem_typing import Sifa
+from .modeling.vocab import SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS
+
+
+@dataclass
+class PhonemeGroup:
+    ref: str = ""
+    out: str = ""
+    ref_idx: int | None = None
+    out_idx: int | None = None
+    tag: Literal["exact", "partial", "insert", "delete"] | None = None
+
+    def get_tag(self):
+        if self.ref == "" and self.out == "":
+            raise ValueError("The Entire group is empty")
+        if self.ref == self.out:
+            self.tag = "exact"
+        elif self.ref != "" and self.out == "":
+            self.tag = "delete"
+        elif self.out != "" and self.ref == "":
+            self.tag = "insert"
+        else:
+            self.tag = "partial"
+        return self.tag
+
+
+def merge_same_phoneme_group(ph_groups: list[PhonemeGroup]) -> list[PhonemeGroup]:
+    outs = [ph_groups[0]]
+    prev_idx = 0
+    for curr_idx in range(1, len(ph_groups)):
+        # out is part of ref
+        if (
+            ph_groups[prev_idx].out_idx is not None
+            and ph_groups[curr_idx].ref_idx is not None
+            and ph_groups[prev_idx].out in ph_groups[curr_idx].ref
+        ):
+            del outs[-1]
+            outs.append(
+                PhonemeGroup(
+                    ref=ph_groups[curr_idx].ref,
+                    ref_idx=ph_groups[curr_idx].ref_idx,
+                    out=ph_groups[prev_idx].out,
+                    out_idx=ph_groups[prev_idx].out_idx,
+                )
+            )
+        # ref is part of out
+        elif (
+            ph_groups[prev_idx].ref_idx is not None
+            and ph_groups[curr_idx].out_idx is not None
+            and ph_groups[prev_idx].ref in ph_groups[curr_idx].out
+        ):
+            del outs[-1]
+            outs.append(
+                PhonemeGroup(
+                    ref=ph_groups[prev_idx].ref,
+                    ref_idx=ph_groups[prev_idx].ref_idx,
+                    out=ph_groups[curr_idx].out,
+                    out_idx=ph_groups[curr_idx].out_idx,
+                )
+            )
+        else:
+            outs.append(ph_groups[curr_idx])
+        prev_idx = curr_idx
+    return outs
+
+
+def segment_groups(
+    ref_groups: list[str],
+    groups: list[str],
+    diffs,
+) -> list[PhonemeGroup]:
+    """Join similar phonmes groups and diffrentiate between groups"""
+    ref_counter = 0
+    ref_ptr = 0
+    ref_group_idx = 0
+    out_counter = 0
+    out_ptr = 0
+    out_group_idx = 0
+
+    out_pairs = []
+    for op, data in diffs:
+        if op == 0:
+            ref_counter += len(data)
+            out_counter += len(data)
+        elif op == 1:
+            out_counter += len(data)
+        elif op == -1:
+            ref_counter += len(data)
+
+        ref_has_match = True
+        out_has_match = True
+        while ref_has_match or out_has_match:
+            pair = PhonemeGroup()
+            if ref_group_idx < len(ref_groups):
+                if (ref_counter - ref_ptr) >= len(ref_groups[ref_group_idx]):
+                    pair.ref = ref_groups[ref_group_idx]
+                    pair.ref_idx = ref_group_idx
+                    ref_ptr += len(ref_groups[ref_group_idx])
+                    ref_group_idx += 1
+                else:
+                    ref_has_match = False
+            else:
+                ref_has_match = False
+
+            if out_group_idx < len(groups):
+                if (out_counter - out_ptr) >= len(groups[out_group_idx]):
+                    pair.out = groups[out_group_idx]
+                    pair.out_idx = out_group_idx
+                    out_ptr += len(groups[out_group_idx])
+                    out_group_idx += 1
+                else:
+                    out_has_match = False
+            else:
+                out_has_match = False
+
+            if pair.ref or pair.out:
+                out_pairs.append(pair)
+    return merge_same_phoneme_group(out_pairs)
+
+
+def expalin_sifat(
+    sifat: list[Sifa],
+    exp_sifat: list[SifaOutput],
+    diffs,
+):
+    table = []
+    chunks = [s.phonemes_group for s in sifat]
+    exp_chunks = [s.phonemes for s in exp_sifat]
+
+    groups = segment_groups(ref_groups=exp_chunks, groups=chunks, diffs=diffs)
+    keys = set(asdict(sifat[0]).keys()) - {"phonemes_group"}
+    madd_group = alph.phonetics.alif + alph.phonetics.yaa_madd + alph.phonetics.waw_madd
+
+    for group in groups:
+        raw = {}
+        tag = group.get_tag()
+        if (tag == "exact") or (tag == "partial" and group.ref[0] in madd_group):
+            raw["tag"] = "exact"
+            raw["phonemes"] = sifat[group.out_idx].phonemes_group
+            raw["exp_phonemes"] = exp_sifat[group.ref_idx].phonemes
+            for key in keys:
+                if getattr(sifat[group.out_idx], key) is not None:
+                    raw[f"{key}"] = getattr(sifat[group.out_idx], key).text
+                else:
+                    raw[f"{key}"] = "None"
+
+                raw[f"exp_{key}"] = getattr(exp_sifat[group.ref_idx], key)
+        elif tag in {"partial", "insert"}:
+            raw["tag"] = "insert"
+            raw["phonemes"] = sifat[group.out_idx].phonemes_group
+            raw["exp_phonemes"] = ""
+            for key in keys:
+                if getattr(sifat[group.out_idx], key) is not None:
+                    raw[f"{key}"] = getattr(sifat[group.out_idx], key).text
+                else:
+                    raw[f"{key}"] = "None"
+
+                raw[f"exp_{key}"] = ""
+        if raw:
+            table.append(raw)
+
+    # print(json.dumps(table, indent=2, ensure_ascii=False))
+    return table
+
+
+def print_sifat_table(
+    table: list[dict],
+    lang: Literal["arabic", "english"] = "arabic",
+):
+    """Print the sifat comparison table with rich highlighting"""
+    if not table:
+        return
+
+    # Create a rich Table
+    rich_table = Table()
+
+    # Get base columns (non-exp keys without 'tag')
+    base_keys = [k for k in table[0].keys() if not k.startswith("exp_") and k != "tag"]
+
+    # Add columns
+    # rich_table.add_column("Tag", style="cyan")
+    for key in base_keys:
+        rich_table.add_column(key.replace("_", " ").title())
+
+    # Add rows
+    for row in table:
+        tag = row["tag"]
+        values = []
+        for key in base_keys:
+            exp_key = f"exp_{key}"
+            value = str(row[key])
+            if key != "phonemes" and lang == "arabic":
+                value = SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS[value]
+
+            # Apply styling based on tag and comparison
+            if tag == "exact" and row.get(exp_key) != row[key]:
+                values.append(f"[red]{value}[/red]")
+            elif tag == "insert":
+                values.append(f"[yellow]{value}[/yellow]")
+            else:
+                values.append(value)
+
+        rich_table.add_row(*values)
+
+    # Print the table
+    console = Console()
+    console.print(rich_table)
+
+
+def explain_for_terminal(
+    phonemes: str,
+    exp_phonemes: str,
+    sifat: list[Sifa],
+    exp_sifat: list[SifaOutput],
+    lang: Literal["arabic", "english"] = "english",
+):
+    # Create diff-match-patch object
+    dmp_obj = dmp.diff_match_patch()
+
+    # Calculate differences
+    diffs = dmp_obj.diff_main(exp_phonemes, phonemes)
+
+    # Create a Rich Text object for colored output
+    result = Text()
+
+    # Process each difference
+    for op, data in diffs:
+        if op == dmp_obj.DIFF_EQUAL:
+            result.append(data, style="white")
+        elif op == dmp_obj.DIFF_INSERT:
+            result.append(data, style="green")
+        elif op == dmp_obj.DIFF_DELETE:
+            result.append(data, style="red strike")
+
+    # Print the result
+    print(result)
+    sifat_table = expalin_sifat(sifat, exp_sifat, diffs)
+    print_sifat_table(sifat_table, lang=lang)  # Add this line to print the table

src/quran_muaalem/explain_gradio.py

@@ -0,0 +1,112 @@
+from typing import Literal
+import diff_match_patch as dmp
+
+from .explain import expalin_sifat
+from .modeling.vocab import SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS
+
+
+def explain_for_gradio(
+    phonemes: str,
+    exp_phonemes: str,
+    sifat: list,
+    exp_sifat: list,
+    lang: Literal["arabic", "english"] = "english",
+) -> str:
+    # Create diff-match-patch object
+    dmp_obj = dmp.diff_match_patch()
+
+    # Calculate differences using Google's diff-match-patch (same as terminal)
+    diffs = dmp_obj.diff_main(exp_phonemes, phonemes)
+
+    # Create HTML for phoneme differences
+    phoneme_html = explain_phonemes_html(dmp_obj, diffs)
+
+    # Create HTML for sifat table using your existing function
+    sifat_table = expalin_sifat(sifat, exp_sifat, diffs)
+    sifat_html = explain_sifat_html(sifat_table, lang)
+
+    # Combine both sections
+    html_output = f"""
+    <div style="font-family: monospace; width: 100%;">
+        <h3>مقارنة الحروف</h3>
+        {phoneme_html}
+        <h3>مقارنة صفات الحروف</h3>
+        {sifat_html}
+       <div class="color-legend">
+    </div>
+    """
+
+    return html_output
+
+
+def explain_phonemes_html(dmp_obj, diffs):
+    html_output = '<div style="background-color: #000; padding: 10px; border-radius: 5px; margin-bottom: 20px; font-size: 30px;">'
+
+    # Process each difference (same logic as terminal version)
+    for op, data in diffs:
+        if op == dmp_obj.DIFF_EQUAL:
+            html_output += f'<span style="color: #ffffff;">{data}</span>'
+        elif op == dmp_obj.DIFF_INSERT:
+            html_output += f'<span style="color: #00ff00;">{data}</span>'
+        elif op == dmp_obj.DIFF_DELETE:
+            html_output += f'<span style="color: #ff0000; text-decoration: line-through;">{data}</span>'
+
+    html_output += "</div>"
+    return html_output
+
+
+def explain_sifat_html(table, lang):
+    if not table:
+        return "<p>No sifat data available</p>"
+
+    # Create HTML table with full width
+    html_output = """
+    <table style="width: 100%; border-collapse: collapse; background-color: #000; color: #fff; margin-bottom: 20px;">
+        <thead>
+            <tr>
+    """
+
+    # Get base columns (non-exp keys without 'tag')
+    base_keys = [k for k in table[0].keys() if not k.startswith("exp_") and k != "tag"]
+
+    # Add columns
+    for key in base_keys:
+        html_output += f'<th style="border: 1px solid #444; padding: 8px; text-align: left;">{key.replace("_", " ").title()}</th>'
+
+    html_output += """
+            </tr>
+        </thead>
+        <tbody>
+    """
+
+    # Add rows
+    for row in table:
+        tag = row["tag"]
+        html_output += "<tr>"
+
+        for key in base_keys:
+            exp_key = f"exp_{key}"
+            value = str(row[key])
+
+            # Apply Arabic translation if needed
+            if key != "phonemes" and lang == "arabic":
+                value = SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS.get(value, value)
+
+            # Apply styling based on tag and comparison
+            if tag == "exact" and row.get(exp_key) != row[key]:
+                html_output += f'<td style="border: 1px solid #444; padding: 8px; color: #ff0000;">{value}</td>'
+            elif tag == "insert":
+                html_output += f'<td style="border: 1px solid #444; padding: 8px; color: #ffff00;">{value}</td>'
+            else:
+                html_output += (
+                    f'<td style="border: 1px solid #444; padding: 8px;">{value}</td>'
+                )
+
+        html_output += "</tr>"
+
+    html_output += """
+        </tbody>
+    </table>
+    """
+
+    return html_output

src/quran_muaalem/gradio_app.py

@@ -0,0 +1,401 @@
+import logging
+from dataclasses import asdict
+import json
+from typing import Literal, Optional, Any, get_origin, get_args
+
+from quran_transcript import Aya, quran_phonetizer, MoshafAttributes
+from quran_transcript.utils import PartOfUthmaniWord
+from quran_transcript.phonetics.moshaf_attributes import (
+    get_arabic_attributes,
+    get_arabic_name,
+)
+from librosa.core import load
+from pydantic.fields import FieldInfo, PydanticUndefined
+import torch
+import gradio as gr
+
+from quran_muaalem.inference import Muaalem
+from quran_muaalem.muaalem_typing import MuaalemOutput
+from quran_muaalem.explain import explain_for_terminal
+from quran_muaalem.explain_gradio import explain_for_gradio
+
+# Initialize components
+REQUIRED_MOSHAF_FIELDS = [
+    "rewaya",
+    "takbeer",
+    "madd_monfasel_len",
+    "madd_mottasel_len",
+    "madd_mottasel_waqf",
+    "madd_aared_len",
+    "madd_alleen_len",
+    "ghonna_lam_and_raa",
+    "meem_aal_imran",
+    "madd_yaa_alayn_alharfy",
+    "saken_before_hamz",
+    "sakt_iwaja",
+    "sakt_marqdena",
+    "sakt_man_raq",
+    "sakt_bal_ran",
+    "sakt_maleeyah",
+    "between_anfal_and_tawba",
+    "noon_and_yaseen",
+    "yaa_ataan",
+    "start_with_ism",
+    "yabsut",
+    "bastah",
+    "almusaytirun",
+    "bimusaytir",
+    "tasheel_or_madd",
+    "yalhath_dhalik",
+    "irkab_maana",
+    "noon_tamnna",
+    "harakat_daaf",
+    "alif_salasila",
+    "idgham_nakhluqkum",
+    "raa_firq",
+    "raa_alqitr",
+    "raa_misr",
+    "raa_nudhur",
+    "raa_yasr",
+    "meem_mokhfah",
+]
+model_id = "obadx/muaalem-model-v3_2"
+logging.basicConfig(level=logging.INFO)
+device = "cuda" if torch.cuda.is_available() else "cpu"
+muaalem = Muaalem(model_name_or_path=model_id, device=device)
+sampling_rate = 16000
+
+# Load Sura information
+sura_idx_to_name = {}
+sura_to_aya_count = {}
+start_aya = Aya()
+for sura_idx in range(1, 115):
+    start_aya.set(sura_idx, 1)
+    sura_idx_to_name[sura_idx] = start_aya.get().sura_name
+    sura_to_aya_count[sura_idx] = start_aya.get().num_ayat_in_sura
+
+# Default moshaf settings
+default_moshaf = MoshafAttributes(
+    rewaya="hafs",
+    madd_monfasel_len=4,
+    madd_mottasel_len=4,
+    madd_mottasel_waqf=4,
+    madd_aared_len=4,
+)
+
+# Current moshaf settings (will be updated from settings page)
+current_moshaf = default_moshaf
+
+
+def get_field_name(field_name: str, field_info: FieldInfo) -> str:
+    """Return the Arabic name of the field if applicable else the field_name"""
+    label = field_name
+    arabic_name = get_arabic_name(field_info)
+    if arabic_name:
+        label = f"{arabic_name} ({field_name})"
+    return label
+
+
+def create_gradio_input_for_field(
+    field_name: str,
+    field_info: FieldInfo,
+    default_value: Any = None,
+    key_prefix="model_",
+    help: str | None = None,
+) -> Any:
+    """Create a gradio input field given a pydantic field info"""
+    # Extract Arabic name from field description if available
+    label = get_field_name(field_name, field_info)
+
+    if default_value is None:
+        if field_info.default != PydanticUndefined:
+            default_value = field_info.default
+
+    if help is None:
+        help = field_info.description
+
+    # Handle Literal types
+    if get_origin(field_info.annotation) is Literal:
+        choices = list(get_args(field_info.annotation))
+        arabic_attributes = get_arabic_attributes(field_info)
+
+        # Create choice list with Arabic labels if available
+        choice_list = []
+        for choice in choices:
+            if arabic_attributes and choice in arabic_attributes:
+                choice_list.append((arabic_attributes[choice], choice))
+            else:
+                choice_list.append((str(choice), choice))
+
+        return gr.Dropdown(
+            choices=choice_list,
+            value=default_value,
+            label=label,
+            info=help,
+            interactive=True,
+        )
+
+    # Handle different field types
+    if field_info.annotation in [str, Optional[str]]:
+        return gr.Textbox(value=default_value or "", label=label, info=help)
+    elif field_info.annotation in [int, Optional[int]]:
+        return gr.Number(value=default_value or 0, label=label, info=help, precision=0)
+    elif field_info.annotation in [float, Optional[float]]:
+        return gr.Number(
+            value=default_value or 0.0, label=label, info=help, precision=1
+        )
+    elif field_info.annotation in [bool, Optional[bool]]:
+        return gr.Checkbox(value=default_value or False, label=label, info=help)
+
+    raise ValueError(f"Unsupported field type for {label}: {field_info.annotation}")
+
+
+def update_aya_dropdown(sura_idx):
+    if not sura_idx:
+        sura_idx = 1
+    return gr.update(
+        choices=list(range(1, sura_to_aya_count[int(sura_idx)] + 1)), value=1
+    )
+
+
+def update_uthmani_ref(sura_idx, aya_idx, start_idx, num_words):
+    if not all([sura_idx, aya_idx, start_idx is not None, num_words is not None]):
+        return ""
+    try:
+        uthmani_ref = (
+            Aya(int(sura_idx), int(aya_idx))
+            .get_by_imlaey_words(int(start_idx), int(num_words))
+            .uthmani
+        )
+        return uthmani_ref
+    except PartOfUthmaniWord as e:
+        return f"⚠️ Warning: You've selected part of a Uthmani word. Please adjust the number of words to include complete words only.\n\nError details: {str(e)}"
+    except Exception as e:
+        return f"Error: {str(e)}"
+
+
+def process_audio(audio, sura_idx, aya_idx, start_idx, num_words):
+    global current_moshaf
+
+    if audio is None:
+        return "Please upload an audio file first"
+
+    try:
+        # Get Uthmani reference text
+        uthmani_ref = (
+            Aya(int(sura_idx), int(aya_idx))
+            .get_by_imlaey_words(int(start_idx), int(num_words))
+            .uthmani
+        )
+        phonetizer_out = quran_phonetizer(
+            uthmani_ref, current_moshaf, remove_spaces=True
+        )
+
+        # Process audio
+        wave, _ = load(audio, sr=sampling_rate, mono=True)
+        outs = muaalem(
+            [wave],
+            [phonetizer_out],
+            sampling_rate=sampling_rate,
+        )
+
+        # # Prepare output
+        # output_text = f"Phonemes: {outs[0].phonemes}\n\n"
+        # for sifa in outs[0].sifat:
+        #     output_text += json.dumps(asdict(sifa), indent=2, ensure_ascii=False) + "\n"
+        #     output_text += "*" * 30 + "\n"
+        # output_text += "-" * 40 + "\n\n"
+
+        # Add explanation
+        explanation_html = explain_for_gradio(
+            outs[0].phonemes.text,
+            phonetizer_out.phonemes,
+            outs[0].sifat,
+            phonetizer_out.sifat,
+            lang="arabic",
+        )
+
+        return explanation_html
+
+    except PartOfUthmaniWord as e:
+        return f"⚠️ Error: The selected word range includes partial Uthmani words. Please adjust the number of words to include complete words only.\n\nError details: {str(e)}"
+    # except Exception as e:
+    #     return f"Error processing audio: {str(e)}"
+
+
+def update_moshaf_settings(*args):
+    """Update the global moshaf settings with values from the settings page"""
+    global current_moshaf, field_names
+
+    try:
+        # Create a dictionary from the field names and values
+        settings_dict = dict(zip(field_names, args))
+
+        # Create a new MoshafAttributes object with the updated values
+        current_moshaf = MoshafAttributes(**settings_dict)
+        return "✅ تم حفظ الإعدادات بنجاح - Settings saved successfully!"
+    except Exception as e:
+        return f"❌ خطأ في حفظ الإعدادات - Error saving settings: {str(e)}"
+
+
+def reset_settings():
+    """Reset all settings to default values"""
+    global current_moshaf
+
+    try:
+        current_moshaf = default_moshaf
+        # Return default values for all fields
+        default_values = [
+            getattr(default_moshaf, field_name) for field_name in field_names
+        ]
+        return default_values + [
+            "✅ تم إعادة التعيين إلى الإعدادات الافتراضية - Reset to default settings successfully!"
+        ]
+    except Exception as e:
+        return [getattr(current_moshaf, field_name) for field_name in field_names] + [
+            f"❌ Error resetting settings: {str(e)}"
+        ]
+
+
+# Create the Gradio app
+with gr.Blocks(title="المعلم القرآني") as app:
+    # Store current moshaf settings in session state
+    current_moshaf_state = gr.State(default_moshaf)
+
+    # Initialize field names list
+    field_names = []
+
+    with gr.Tab("التحليل الرئيسي - Main Analysis"):
+        gr.Markdown("# كشف أخطاء التلاوة والتجويد وصفات الحروف")
+        gr.Markdown("اختر المقطع القرآني المراد تعلمه")
+
+        with gr.Row():
+            with gr.Column(scale=1):
+                gr.Markdown("### التلاة المقارنة")
+
+                # Create sura dropdown with both index and name
+                sura_choices = [
+                    (f"{idx} - {sura_idx_to_name[idx]}", idx) for idx in range(1, 115)
+                ]
+                sura_dropdown = gr.Dropdown(
+                    choices=sura_choices,
+                    label="السورة",
+                    value=1,
+                    elem_id="sura_dropdown",
+                )
+
+                aya_dropdown = gr.Dropdown(
+                    choices=list(range(1, sura_to_aya_count[1] + 1)),
+                    label="رقم الآية",
+                    value=1,
+                    elem_id="aya_dropdown",
+                )
+                start_idx = gr.Number(
+                    value=0,
+                    label="رقمة الكلمة بداية من صفر (Word Index)",
+                    minimum=0,
+                    step=1,
+                    elem_id="start_idx",
+                )
+                num_words = gr.Number(
+                    value=4,
+                    label="عدد الكلمات",
+                    minimum=1,
+                    step=1,
+                    elem_id="num_words",
+                )
+                uthmani_text = gr.Textbox(
+                    label="الرسم العثماني",
+                    interactive=False,
+                    elem_id="uthmani_text",
+                )
+
+            with gr.Column(scale=2):
+                gr.Markdown("### فحص التلاوة القرآنية")
+                audio_input = gr.Audio(
+                    sources=["upload", "microphone"],
+                    label="Upload or Record Audio",
+                    type="filepath",
+                    elem_id="audio_input",
+                )
+                analyze_btn = gr.Button(
+                    "افحص التلاوة", variant="primary", elem_id="analyze_btn"
+                )
+                output_html = gr.HTML(
+                    label="نتيجة الفحص",
+                    elem_id="output_html",
+                )
+
+        # Initial update of uthmani text
+        app.load(
+            update_uthmani_ref,
+            inputs=[sura_dropdown, aya_dropdown, start_idx, num_words],
+            outputs=uthmani_text,
+        )
+
+        # Update aya dropdown when sura changes and reset aya_idx to 1
+        sura_dropdown.change(
+            update_aya_dropdown, inputs=sura_dropdown, outputs=aya_dropdown
+        ).then(
+            update_uthmani_ref,
+            inputs=[sura_dropdown, aya_dropdown, start_idx, num_words],
+            outputs=uthmani_text,
+        )
+
+        # Update uthmani text when any parameter changes
+        for component in [aya_dropdown, start_idx, num_words]:
+            component.change(
+                update_uthmani_ref,
+                inputs=[sura_dropdown, aya_dropdown, start_idx, num_words],
+                outputs=uthmani_text,
+            )
+
+        # Process audio when button is clicked
+        analyze_btn.click(
+            process_audio,
+            inputs=[audio_input, sura_dropdown, aya_dropdown, start_idx, num_words],
+            outputs=output_html,
+        )
+
+    with gr.Tab("إعدادات المصحف - Moshaf Settings"):
+        gr.Markdown("# إعدادات خصائص المصحف")
+        gr.Markdown("قم بتعديل خصائص المصحف حسب التلاوة المطلوبة")
+
+        # Create settings inputs directly in the tab
+        settings_components = []
+        fields = MoshafAttributes.model_fields
+
+        # Create inputs for all required fields
+        for field_name in REQUIRED_MOSHAF_FIELDS:
+            field_info = fields[field_name]
+            input_component = create_gradio_input_for_field(
+                field_name, field_info, getattr(default_moshaf, field_name, None)
+            )
+            settings_components.append(input_component)
+            field_names.append(field_name)
+
+        # Save button and status message
+        with gr.Row():
+            save_btn = gr.Button("حفظ الإعدادات - Save Settings", variant="primary")
+            reset_btn = gr.Button("إعادة التعيين - Reset to Default")
+
+        status_message = gr.Markdown()
+
+        # Save settings event
+        save_btn.click(
+            update_moshaf_settings, inputs=settings_components, outputs=status_message
+        )
+
+        # Reset to default event
+        reset_btn.click(
+            reset_settings, inputs=[], outputs=settings_components + [status_message]
+        )
+
+
+def main(app=app):
+    app.launch(server_name="0.0.0.0", share=True)
+
+
+if __name__ == "__main__":
+    main()
+    # app.launch(server_name="0.0.0.0", share=True)

src/quran_muaalem/inference.py

@@ -0,0 +1,188 @@
+import logging
+
+from quran_transcript import chunck_phonemes, QuranPhoneticScriptOutput
+from transformers import AutoFeatureExtractor
+import torch
+from numpy.typing import NDArray
+
+from .modeling.multi_level_tokenizer import MultiLevelTokenizer
+from .modeling.modeling_multi_level_ctc import Wav2Vec2BertForMultilevelCTC
+from .decode import (
+    multilevel_greedy_decode,
+    phonemes_level_greedy_decode,
+)
+from .muaalem_typing import Unit, SingleUnit, Sifa, MuaalemOutput
+
+
+def format_sifat(
+    level_to_units: dict[str, list[Unit]],
+    chunked_phonemes_batch: list[list[str]],
+    multi_level_tokenizer: MultiLevelTokenizer,
+) -> list[list[Sifa]]:
+    sifat_batch = []
+    for seq_idx in range(len(chunked_phonemes_batch)):
+        sifat = []
+        for idx, ph_group in enumerate(chunked_phonemes_batch[seq_idx]):
+            sifa_dict = {}
+            for level in level_to_units:
+                if level == "phonemes":
+                    continue
+                sifa_idx = idx
+                if sifa_idx < len(level_to_units[level][seq_idx].ids):
+                    label = int(level_to_units[level][seq_idx].ids[sifa_idx])
+                    text = multi_level_tokenizer.sifat_to_en_vocab[level][label]
+                    p = level_to_units[level][seq_idx].probs[sifa_idx]
+                    sifa_dict[level] = SingleUnit(
+                        text=text, prob=float(p), idx=int(label)
+                    )
+                else:
+                    logging.info(
+                        f"Sequence: `{seq_idx}` has short Level: {level} we will place it with `None`"
+                    )
+                    sifa_dict[level] = None
+            sifat.append(
+                Sifa(
+                    phonemes_group=chunked_phonemes_batch[seq_idx][idx],
+                    **sifa_dict,
+                )
+            )
+        sifat_batch.append(sifat)
+    return sifat_batch
+
+
+class Muaalem:
+    def __init__(
+        self,
+        model_name_or_path: str = "obadx/muaalem-model-v3_2",
+        device: str = "cpu",
+        dtype=torch.bfloat16,
+    ):
+        """
+        Initializing Muallem Model
+
+        Args:
+            model_name_or_path: the huggingface model name or path
+            device: the device to run model on
+            dtype: the torch dtype. Default is `torch.bfloat16` as the model was trained on
+        """
+        self.device = device
+        self.dtype = dtype
+
+        self.model = Wav2Vec2BertForMultilevelCTC.from_pretrained(model_name_or_path)
+        self.multi_level_tokenizer = MultiLevelTokenizer(model_name_or_path)
+        self.processor = AutoFeatureExtractor.from_pretrained(model_name_or_path)
+
+        self.model.to(device, dtype=dtype)
+
+    @torch.no_grad()
+    def __call__(
+        self,
+        waves: list[list[float] | torch.FloatTensor | NDArray],
+        ref_quran_phonetic_script_list: list[QuranPhoneticScriptOutput],
+        sampling_rate: int,
+    ) -> list[MuaalemOutput]:
+        """Infrence Funcion for the Quran Muaalem Project
+
+                waves: input waves  batch , seq_len with different formats described above
+                ref_quran_phonetic_script_list (list[QuranPhoneticScriptOutput]): list of the
+                    phonetized ouput of `quran_transcript.quran_phonetizer` with `remove_space=True`
+
+                sampleing_rate (int): has to be 16000
+
+        Returns:
+            list[MuaalemOutput]:
+                A list of output objects, each containing phoneme predictions and their
+                phonetic features (sifat) for a processed input.
+
+            Each MuaalemOutput contains:
+                phonemes (Unit):
+                    A dataclass representing the predicted phoneme sequence with:
+                        text (str): Concatenated string of all phonemes.
+                        probs (Union[torch.FloatTensor, list[float]]):
+                            Confidence probabilities for each predicted phoneme.
+                        ids (Union[torch.LongTensor, list[int]]):
+                            Token IDs corresponding to each phoneme.
+
+                sifat (list[Sifa]):
+                    A list of phonetic feature dataclasses (one per phoneme) with the
+                    following optional properties (each is a SingleUnit or None):
+                        - phonemes_group (str): the phonemes associated with the `sifa`
+                        - hams_or_jahr (SingleUnit): either `hams` or `jahr`
+                        - shidda_or_rakhawa (SingleUnit): either `shadeed`, `between`, or `rikhw`
+                        - tafkheem_or_taqeeq (SingleUnit): either `mofakham`, `moraqaq`, or `low_mofakham`
+                        - itbaq (SingleUnit): either `monfateh`, or `motbaq`
+                        - safeer (SingleUnit): either `safeer`, or `no_safeer`
+                        - qalqla (SingleUnit): eithr `moqalqal`, or `not_moqalqal`
+                        - tikraar (SingleUnit): either `mokarar` or `not_mokarar`
+                        - tafashie (SingleUnit): either `motafashie`, or `not_motafashie`
+                        - istitala (SingleUnit): either `mostateel`, or `not_mostateel`
+                        - ghonna (SingleUnit): either `maghnoon`, or `not_maghnoon`
+
+            Each SingleUnit in Sifa properties contains:
+                text (str): The feature's categorical label (e.g., "hams", "shidda").
+                prob (float): Confidence probability for this feature.
+                idx (int): Identifier for the feature class.
+        """
+
+        if sampling_rate != 16000:
+            raise ValueError(f"`sampling_rate` has to be 16000 got: `{sampling_rate}`")
+
+        # TODO: check input waves
+
+        # Tokanizing Ref
+        level_to_ref_ids = self.multi_level_tokenizer.tokenize(
+            [r.phonemes for r in ref_quran_phonetic_script_list],
+            [r.sifat for r in ref_quran_phonetic_script_list],
+            to_dict=True,
+            return_tensors="pt",
+            padding="longest",
+        )["input_ids"]
+
+        features = self.processor(
+            waves, sampling_rate=sampling_rate, return_tensors="pt"
+        )
+        features = {k: v.to(self.device, dtype=self.dtype) for k, v in features.items()}
+        outs = self.model(**features, return_dict=False)[0]
+
+        probs = {}
+        for level in outs:
+            probs[level] = (
+                torch.nn.functional.softmax(outs[level], dim=-1).cpu().to(torch.float32)
+            )
+
+        # Decoding only Phonemes Level
+        phonemes_units = phonemes_level_greedy_decode(
+            probs["phonemes"], self.multi_level_tokenizer.id_to_vocab["phonemes"]
+        )
+
+        chunked_phonemes_batch: list[list[str]] = []
+        for phonemes_unit in phonemes_units:
+            chunked_phonemes_batch.append(chunck_phonemes(phonemes_unit.text))
+
+        level_to_units = multilevel_greedy_decode(
+            level_to_probs=probs,
+            level_to_id_to_vocab=self.multi_level_tokenizer.id_to_vocab,
+            level_to_ref_ids=level_to_ref_ids,
+            chunked_phonemes_batch=chunked_phonemes_batch,
+            ref_chuncked_phonemes_batch=[
+                [s.phonemes for s in r.sifat] for r in ref_quran_phonetic_script_list
+            ],
+            phonemes_units=phonemes_units,
+        )
+
+        sifat_batch: list[list[Sifa]] = format_sifat(
+            level_to_units,
+            chunked_phonemes_batch,
+            self.multi_level_tokenizer,
+        )
+
+        outs = []
+        # looping over the batch
+        for idx in range(len(level_to_units["phonemes"])):
+            outs.append(
+                MuaalemOutput(
+                    phonemes=level_to_units["phonemes"][idx],
+                    sifat=sifat_batch[idx],
+                )
+            )
+        return outs

src/quran_muaalem/modeling/configuration_multi_level_ctc.py

@@ -0,0 +1,320 @@
+from transformers import PretrainedConfig
+
+
+class Wav2Vec2BertForMultilevelCTCConfig(PretrainedConfig):
+    r"""
+    This is the configuration class to store the configuration of a [`Wav2Vec2BertModel`]. It is used to
+    instantiate an Wav2Vec2Bert model according to the specified arguments, defining the model architecture.
+    Instantiating a configuration with the defaults will yield a similar configuration to that of the Wav2Vec2Bert
+    [facebook/wav2vec2-bert-rel-pos-large](https://huggingface.co/facebook/wav2vec2-bert-rel-pos-large)
+    architecture.
+
+    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
+    documentation from [`PretrainedConfig`] for more information.
+
+
+    Args:
+        level_to_vocab_size (`dict[str, int]`, *optional*):
+            Every level has its own vocabulary: {'phonemes': 44, 'hams_or_jahr': 3, ....}
+            Vocabulary size of the Wav2Vec2Bert model. Defines the number of different tokens that can be
+            represented by the `inputs_ids` passed when calling [`Wav2Vec2BertModel`]. Vocabulary size of the
+            model. Defines the different tokens that can be represented by the *inputs_ids* passed to the forward
+            method of [`Wav2Vec2BertModel`].
+        level_to_loss_weigth (`dict[str, int]`, *optional*):
+            Every level has its own loss weigth such that the sum of all levels adds to 1:
+            If you supply only one level: the rest of the level will have loss weigth of (1-given_loss_weigth) / nmber of rest of levels
+
+        hidden_size (`int`, *optional*, defaults to 1024):
+            Dimensionality of the encoder layers and the pooler layer.
+        num_hidden_layers (`int`, *optional*, defaults to 24):
+            Number of hidden layers in the Transformer encoder.
+        num_attention_heads (`int`, *optional*, defaults to 16):
+            Number of attention heads for each attention layer in the Transformer encoder.
+        intermediate_size (`int`, *optional*, defaults to 4096):
+            Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder.
+        feature_projection_input_dim (`int`, *optional*, defaults to 160):
+            Input dimension of this model, i.e the dimension after processing input audios with [`SeamlessM4TFeatureExtractor`] or [`Wav2Vec2BertProcessor`].
+        hidden_act (`str` or `function`, *optional*, defaults to `"swish"`):
+            The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,
+            `"relu"`, `"selu"`, `"swish"` and `"gelu_new"` are supported.
+        hidden_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout probability for all fully connected layers in the embeddings, encoder, and pooler.
+        activation_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for activations inside the fully connected layer.
+        attention_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout ratio for the attention probabilities.
+        feat_proj_dropout (`float`, *optional*, defaults to 0.0):
+            The dropout probability for the feature projection.
+        final_dropout (`float`, *optional*, defaults to 0.1):
+            The dropout probability for the final projection layer of [`Wav2Vec2BertForCTC`].
+        layerdrop (`float`, *optional*, defaults to 0.1):
+            The LayerDrop probability. See the [LayerDrop paper](see https://huggingface.co/papers/1909.11556) for more
+            details.
+        initializer_range (`float`, *optional*, defaults to 0.02):
+            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
+        layer_norm_eps (`float`, *optional*, defaults to 1e-05):
+            The epsilon used by the layer normalization layers.
+        apply_spec_augment (`bool`, *optional*, defaults to `True`):
+            Whether to apply *SpecAugment* data augmentation to the outputs of the feature encoder. For reference see
+            [SpecAugment: A Simple Data Augmentation Method for Automatic Speech
+            Recognition](https://huggingface.co/papers/1904.08779).
+        mask_time_prob (`float`, *optional*, defaults to 0.05):
+            Percentage (between 0 and 1) of all feature vectors along the time axis which will be masked. The masking
+            procedure generates `mask_time_prob*len(time_axis)/mask_time_length ``independent masks over the axis. If
+            reasoning from the probability of each feature vector to be chosen as the start of the vector span to be
+            masked, *mask_time_prob* should be `prob_vector_start*mask_time_length`. Note that overlap may decrease the
+            actual percentage of masked vectors. This is only relevant if `apply_spec_augment is True`.
+        mask_time_length (`int`, *optional*, defaults to 10):
+            Length of vector span along the time axis.
+        mask_time_min_masks (`int`, *optional*, defaults to 2):
+            The minimum number of masks of length `mask_feature_length` generated along the time axis, each time step,
+            irrespectively of `mask_feature_prob`. Only relevant if `mask_time_prob*len(time_axis)/mask_time_length <
+            mask_time_min_masks`.
+        mask_feature_prob (`float`, *optional*, defaults to 0.0):
+            Percentage (between 0 and 1) of all feature vectors along the feature axis which will be masked. The
+            masking procedure generates `mask_feature_prob*len(feature_axis)/mask_time_length` independent masks over
+            the axis. If reasoning from the probability of each feature vector to be chosen as the start of the vector
+            span to be masked, *mask_feature_prob* should be `prob_vector_start*mask_feature_length`. Note that overlap
+            may decrease the actual percentage of masked vectors. This is only relevant if `apply_spec_augment is
+            True`.
+        mask_feature_length (`int`, *optional*, defaults to 10):
+            Length of vector span along the feature axis.
+        mask_feature_min_masks (`int`, *optional*, defaults to 0):
+            The minimum number of masks of length `mask_feature_length` generated along the feature axis, each time
+            step, irrespectively of `mask_feature_prob`. Only relevant if
+            `mask_feature_prob*len(feature_axis)/mask_feature_length < mask_feature_min_masks`.
+        ctc_loss_reduction (`str`, *optional*, defaults to `"sum"`):
+            Specifies the reduction to apply to the output of `torch.nn.CTCLoss`. Only relevant when training an
+            instance of [`Wav2Vec2BertForCTC`].
+        ctc_zero_infinity (`bool`, *optional*, defaults to `False`):
+            Whether to zero infinite losses and the associated gradients of `torch.nn.CTCLoss`. Infinite losses mainly
+            occur when the inputs are too short to be aligned to the targets. Only relevant when training an instance
+            of [`Wav2Vec2BertForCTC`].
+        use_weighted_layer_sum (`bool`, *optional*, defaults to `False`):
+            Whether to use a weighted average of layer outputs with learned weights. Only relevant when using an
+            instance of [`Wav2Vec2BertForSequenceClassification`].
+        classifier_proj_size (`int`, *optional*, defaults to 768):
+            Dimensionality of the projection before token mean-pooling for classification.
+        tdnn_dim (`tuple[int]` or `list[int]`, *optional*, defaults to `(512, 512, 512, 512, 1500)`):
+            A tuple of integers defining the number of output channels of each 1D convolutional layer in the *TDNN*
+            module of the *XVector* model. The length of *tdnn_dim* defines the number of *TDNN* layers.
+        tdnn_kernel (`tuple[int]` or `list[int]`, *optional*, defaults to `(5, 3, 3, 1, 1)`):
+            A tuple of integers defining the kernel size of each 1D convolutional layer in the *TDNN* module of the
+            *XVector* model. The length of *tdnn_kernel* has to match the length of *tdnn_dim*.
+        tdnn_dilation (`tuple[int]` or `list[int]`, *optional*, defaults to `(1, 2, 3, 1, 1)`):
+            A tuple of integers defining the dilation factor of each 1D convolutional layer in *TDNN* module of the
+            *XVector* model. The length of *tdnn_dilation* has to match the length of *tdnn_dim*.
+        xvector_output_dim (`int`, *optional*, defaults to 512):
+            Dimensionality of the *XVector* embedding vectors.
+        pad_token_id (`int`, *optional*, defaults to 0): The id of the _beginning-of-stream_ token.
+        bos_token_id (`int`, *optional*, defaults to 1): The id of the _padding_ token.
+        eos_token_id (`int`, *optional*, defaults to 2): The id of the _end-of-stream_ token.
+        add_adapter (`bool`, *optional*, defaults to `False`):
+            Whether a convolutional attention network should be stacked on top of the Wav2Vec2Bert Encoder. Can be very
+            useful for warm-starting Wav2Vec2Bert for SpeechEncoderDecoder models.
+        adapter_kernel_size (`int`, *optional*, defaults to 3):
+            Kernel size of the convolutional layers in the adapter network. Only relevant if `add_adapter is True`.
+        adapter_stride (`int`, *optional*, defaults to 2):
+            Stride of the convolutional layers in the adapter network. Only relevant if `add_adapter is True`.
+        num_adapter_layers (`int`, *optional*, defaults to 1):
+            Number of convolutional layers that should be used in the adapter network. Only relevant if `add_adapter is
+            True`.
+        adapter_act (`str` or `function`, *optional*, defaults to `"relu"`):
+            The non-linear activation function (function or string) in the adapter layers. If string, `"gelu"`,
+            `"relu"`, `"selu"`, `"swish"` and `"gelu_new"` are supported.
+        use_intermediate_ffn_before_adapter (`bool`, *optional*, defaults to `False`):
+            Whether an intermediate feed-forward block should be stacked on top of the Wav2Vec2Bert Encoder and before the adapter network.
+             Only relevant if `add_adapter is True`.
+        output_hidden_size (`int`, *optional*):
+            Dimensionality of the encoder output layer. If not defined, this defaults to *hidden-size*. Only relevant
+            if `add_adapter is True`.
+        position_embeddings_type (`str`, *optional*, defaults to `"relative_key"`):
+            Can be specified to :
+                - `rotary`, for rotary position embeddings.
+                - `relative`, for relative position embeddings.
+                - `relative_key`, for relative position embeddings as defined by Shaw in [Self-Attention
+            with Relative Position Representations (Shaw et al.)](https://huggingface.co/papers/1803.02155).
+            If left to `None`, no relative position embeddings is applied.
+        rotary_embedding_base (`int`, *optional*, defaults to 10000):
+            If `"rotary"` position embeddings are used, defines the size of the embedding base.
+        max_source_positions (`int`, *optional*, defaults to 5000):
+            if `"relative"` position embeddings are used, defines the maximum source input positions.
+        left_max_position_embeddings (`int`, *optional*, defaults to 64):
+            If `"relative_key"` (aka Shaw) position embeddings are used, defines the left clipping value for relative positions.
+        right_max_position_embeddings (`int`, *optional*, defaults to 8):
+            If `"relative_key"` (aka Shaw) position embeddings are used, defines the right clipping value for relative positions.
+        conv_depthwise_kernel_size (`int`, *optional*, defaults to 31):
+            Kernel size of convolutional depthwise 1D layer in Conformer blocks.
+        conformer_conv_dropout (`float`, *optional*, defaults to 0.1):
+            The dropout probability for all convolutional layers in Conformer blocks.
+    Example:
+
+    ```python
+    >>> from transformers import Wav2Vec2BertConfig, Wav2Vec2BertModel
+
+    >>> # Initializing a Wav2Vec2Bert facebook/wav2vec2-bert-rel-pos-large style configuration
+    >>> configuration = Wav2Vec2BertConfig()
+
+    >>> # Initializing a model (with random weights) from the facebook/wav2vec2-bert-rel-pos-large style configuration
+    >>> model = Wav2Vec2BertModel(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```"""
+
+    model_type = "multi_level_ctc"
+
+    def __init__(
+        self,
+        level_to_vocab_size: dict[str, int] = {},
+        level_to_loss_weight: dict[str, float] = {"phonemes": 0.4},
+        hidden_size=1024,
+        num_hidden_layers=24,
+        num_attention_heads=16,
+        intermediate_size=4096,
+        feature_projection_input_dim=160,
+        hidden_act="swish",
+        hidden_dropout=0.0,
+        activation_dropout=0.0,
+        attention_dropout=0.0,
+        feat_proj_dropout=0.0,
+        final_dropout=0.1,
+        layerdrop=0.1,
+        initializer_range=0.02,
+        layer_norm_eps=1e-5,
+        apply_spec_augment=True,
+        mask_time_prob=0.05,
+        mask_time_length=10,
+        mask_time_min_masks=2,
+        mask_feature_prob=0.0,
+        mask_feature_length=10,
+        mask_feature_min_masks=0,
+        ctc_loss_reduction="sum",
+        ctc_zero_infinity=False,
+        use_weighted_layer_sum=False,
+        classifier_proj_size=768,
+        tdnn_dim=(512, 512, 512, 512, 1500),
+        tdnn_kernel=(5, 3, 3, 1, 1),
+        tdnn_dilation=(1, 2, 3, 1, 1),
+        xvector_output_dim=512,
+        pad_token_id=0,
+        bos_token_id=1,
+        eos_token_id=2,
+        add_adapter=False,
+        adapter_kernel_size=3,
+        adapter_stride=2,
+        num_adapter_layers=1,
+        adapter_act="relu",
+        use_intermediate_ffn_before_adapter=False,
+        output_hidden_size=None,
+        position_embeddings_type="relative_key",
+        rotary_embedding_base=10000,
+        max_source_positions=5000,
+        left_max_position_embeddings=64,
+        right_max_position_embeddings=8,
+        conv_depthwise_kernel_size=31,
+        conformer_conv_dropout=0.1,
+        **kwargs,
+    ):
+        super().__init__(
+            **kwargs,
+            pad_token_id=pad_token_id,
+            bos_token_id=bos_token_id,
+            eos_token_id=eos_token_id,
+        )
+        self.hidden_size = hidden_size
+        self.num_hidden_layers = num_hidden_layers
+        self.intermediate_size = intermediate_size
+        self.hidden_act = hidden_act
+        self.num_attention_heads = num_attention_heads
+        self.feature_projection_input_dim = feature_projection_input_dim
+        self.hidden_dropout = hidden_dropout
+        self.attention_dropout = attention_dropout
+        self.activation_dropout = activation_dropout
+        self.feat_proj_dropout = feat_proj_dropout
+        self.final_dropout = final_dropout
+        self.layerdrop = layerdrop
+        self.layer_norm_eps = layer_norm_eps
+        self.initializer_range = initializer_range
+        self.level_to_vocab_size = level_to_vocab_size
+        self.use_weighted_layer_sum = use_weighted_layer_sum
+        self.max_source_positions = max_source_positions
+
+        loss_weights_sum = sum(level_to_loss_weight.values())
+        if loss_weights_sum > 1:
+            raise ValueError(
+                f"The sum of loss weight per level has to be less than one! got: `{level_to_loss_weight}`"
+            )
+        unmentioned_loss_levels_count = len(
+            [l for l in self.level_to_vocab_size if l not in level_to_loss_weight]
+        )
+        for level in self.level_to_vocab_size:
+            if level not in level_to_loss_weight:
+                level_to_loss_weight[level] = (
+                    1 - loss_weights_sum
+                ) / unmentioned_loss_levels_count
+        self.level_to_loss_weight = level_to_loss_weight
+
+        if position_embeddings_type is not None and position_embeddings_type not in [
+            "rotary",
+            "relative",
+            "relative_key",
+        ]:
+            raise ValueError(
+                """
+                `position_embeddings_type` is not valid. It must be one of the following values:
+                `["rotary", "relative", "relative_key"]` or left as `None`.
+                """
+            )
+        self.position_embeddings_type = position_embeddings_type
+        self.rotary_embedding_base = rotary_embedding_base
+        self.left_max_position_embeddings = left_max_position_embeddings
+        self.right_max_position_embeddings = right_max_position_embeddings
+
+        # Conformer-block related
+        self.conv_depthwise_kernel_size = conv_depthwise_kernel_size
+        self.conformer_conv_dropout = conformer_conv_dropout
+
+        # fine-tuning config parameters for SpecAugment: https://huggingface.co/papers/1904.08779
+        self.apply_spec_augment = apply_spec_augment
+        self.mask_time_prob = mask_time_prob
+        self.mask_time_length = mask_time_length
+        self.mask_time_min_masks = mask_time_min_masks
+        self.mask_feature_prob = mask_feature_prob
+        self.mask_feature_length = mask_feature_length
+        self.mask_feature_min_masks = mask_feature_min_masks
+
+        # ctc loss
+        self.ctc_loss_reduction = ctc_loss_reduction
+        self.ctc_zero_infinity = ctc_zero_infinity
+
+        # adapter
+        self.add_adapter = add_adapter
+        self.adapter_kernel_size = adapter_kernel_size
+        self.adapter_stride = adapter_stride
+        self.num_adapter_layers = num_adapter_layers
+        self.adapter_act = adapter_act
+        self.output_hidden_size = (
+            output_hidden_size if output_hidden_size is not None else hidden_size
+        )
+        if use_intermediate_ffn_before_adapter and not add_adapter:
+            raise ValueError(
+                "`use_intermediate_ffn_before_adapter` is `True` but `add_adapter` is `False`."
+            )
+        self.use_intermediate_ffn_before_adapter = use_intermediate_ffn_before_adapter
+
+        # SequenceClassification-specific parameter. Feel free to ignore for other classes.
+        self.classifier_proj_size = classifier_proj_size
+
+        # XVector-specific parameters. Feel free to ignore for other classes.
+        self.tdnn_dim = list(tdnn_dim)
+        self.tdnn_kernel = list(tdnn_kernel)
+        self.tdnn_dilation = list(tdnn_dilation)
+        self.xvector_output_dim = xvector_output_dim
+
+    @property
+    def inputs_to_logits_ratio(self):
+        ratio = self.feature_projection_input_dim * 2
+        if self.add_adapter:
+            ratio = ratio * (self.adapter_stride**self.num_adapter_layers)
+        return ratio

src/quran_muaalem/modeling/modeling_multi_level_ctc.py

@@ -0,0 +1,148 @@
+from typing import Optional, Union
+
+from transformers.models.wav2vec2_bert.modeling_wav2vec2_bert import (
+    Wav2Vec2BertPreTrainedModel,
+    Wav2Vec2BertModel,
+    _HIDDEN_STATES_START_POSITION,
+)
+from transformers.utils import auto_docstring
+from transformers.modeling_outputs import CausalLMOutput
+import torch
+from torch import nn
+
+from .configuration_multi_level_ctc import Wav2Vec2BertForMultilevelCTCConfig
+
+
+class Wav2Vec2BertForMultilevelCTC(Wav2Vec2BertPreTrainedModel):
+    config_class = Wav2Vec2BertForMultilevelCTCConfig
+
+    def __init__(self, config):
+        super().__init__(config)
+
+        self.wav2vec2_bert = Wav2Vec2BertModel(config)
+        self.dropout = nn.Dropout(config.final_dropout)
+
+        if config.level_to_vocab_size == {}:
+            raise ValueError(
+                f"You are trying to instantiate {self.__class__} with a configuration that "
+                "does not define the vocabulary size of the language model head. Please "
+                "instantiate the model as follows: `Wav2Vec2BertForCTC.from_pretrained(..., level_to_vocab_size=level_to_vocab_size)`. "
+                "or define `level_to_vocab_size` of your model's configuration."
+            )
+        output_hidden_size = (
+            config.output_hidden_size
+            if hasattr(config, "add_adapter") and config.add_adapter
+            else config.hidden_size
+        )
+        self.level_to_lm_head = nn.ModuleDict(
+            {
+                level: nn.Linear(output_hidden_size, vocab_size)
+                for level, vocab_size in config.level_to_vocab_size.items()
+            }
+        )
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @auto_docstring
+    def forward(
+        self,
+        input_features: Optional[torch.Tensor],
+        attention_mask: Optional[torch.Tensor] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+        labels: Optional[dict[str, torch.Tensor]] = None,
+    ) -> Union[tuple, CausalLMOutput]:
+        r"""
+        labels (dict[`str`, `torch.LongTensor`] level_name to its labels of shape `(batch_size, target_length)`, *optional*):
+            Labels for connectionist temporal classification. Note that `target_length` has to be smaller or equal to
+            the sequence length of the output logits. Indices are selected in `[-100, 0, ..., config.vocab_size - 1]`.
+            All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ...,
+            config.vocab_size - 1]`.
+        """
+        if labels is not None:
+            if not isinstance(labels, dict):
+                raise ValueError(
+                    f"Label has to be a dict for level to its tartget labels got `{type(labels)}`"
+                )
+            for level in labels:
+                if labels[level].max() >= self.config.level_to_vocab_size[level]:
+                    raise ValueError(
+                        f"Label values must be <= vocab_size: {self.config.level_to_vocab_size[level]} for level: `{level}`"
+                    )
+
+        return_dict = (
+            return_dict if return_dict is not None else self.config.use_return_dict
+        )
+
+        outputs = self.wav2vec2_bert(
+            input_features,
+            attention_mask=attention_mask,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+        hidden_states = outputs[0]
+        hidden_states = self.dropout(hidden_states)
+
+        level_to_logits = {}
+        for level in self.level_to_lm_head:
+            level_to_logits[level] = self.level_to_lm_head[level](hidden_states)
+
+        loss = None
+        if labels is not None:
+            # retrieve loss input_lengths from attention_mask
+            attention_mask = (
+                attention_mask
+                if attention_mask is not None
+                else torch.ones(
+                    input_features.shape[:2],
+                    device=input_features.device,
+                    dtype=torch.long,
+                )
+            )
+            input_lengths = self._get_feat_extract_output_lengths(
+                attention_mask.sum([-1])
+            ).to(torch.long)
+
+            loss = 0.0
+            for level in labels:
+                # assuming that padded tokens are filled with -100
+                # when not being attended to
+                labels_mask = labels[level] >= 0
+                target_lengths = labels_mask.sum(-1)
+                flattened_targets = labels[level].masked_select(labels_mask)
+
+                # ctc_loss doesn't support fp16
+                log_probs = nn.functional.log_softmax(
+                    level_to_logits[level], dim=-1, dtype=torch.float32
+                ).transpose(0, 1)
+
+                with torch.backends.cudnn.flags(enabled=False):
+                    loss += self.config.level_to_loss_weight[
+                        level
+                    ] * nn.functional.ctc_loss(
+                        log_probs,
+                        flattened_targets,
+                        input_lengths,
+                        target_lengths,
+                        blank=self.config.pad_token_id,
+                        reduction=self.config.ctc_loss_reduction,
+                        zero_infinity=self.config.ctc_zero_infinity,
+                    )
+
+        if not return_dict:
+            output = (level_to_logits,) + outputs[_HIDDEN_STATES_START_POSITION:]
+            return ((loss,) + output) if loss is not None else output
+
+        return CausalLMOutput(
+            loss=loss,
+            logits=level_to_logits,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )
+
+
+__all__ = ["Wav2Vec2BertForMultilevelCTC"]

src/quran_muaalem/modeling/multi_level_tokenizer.py

@@ -0,0 +1,121 @@
+from typing import get_origin, Literal, Any
+
+from quran_transcript import SifaOutput, quran_phonetizer
+from transformers import Wav2Vec2CTCTokenizer
+
+from .vocab import PAD_TOKEN, PAD_TOKEN_IDX, SIFAT_ATTR_TO_ARABIC, SIFAT_ATTR_TO_ENGLISH
+
+
+def add_zero_between(L, x=PAD_TOKEN_IDX):
+    out = []
+    for i, item in enumerate(L):
+        out.append(item)
+        if i < len(L) - 1:  # Don't add zero after the last element
+            out.append(0)
+    return out
+
+
+class MultiLevelTokenizer:
+    def __init__(self, model_name_or_path: str):
+        self.levels = ["phonemes"]
+        for fieldname, fieldinfo in SifaOutput.model_fields.items():
+            if get_origin(fieldinfo.annotation) == Literal:
+                self.levels.append(fieldname)
+
+        self.level_to_tokenizer = {}
+        for level in self.levels:
+            self.level_to_tokenizer[level] = Wav2Vec2CTCTokenizer.from_pretrained(
+                model_name_or_path, pad_token=PAD_TOKEN, target_lang=level
+            )
+
+        self.level_to_id_vocab = self.get_level_to_id_to_voab()
+        self.sifat_level_to_id_to_en_vocab = self.get_sifat_levels_to_en_name()
+
+    def get_tokenizer(self):
+        return self.level_to_tokenizer["phonemes"]
+
+    @property
+    def vocab(self):
+        return self.get_tokenizer().vocab
+
+    @property
+    def id_to_vocab(self):
+        return self.level_to_id_vocab
+
+    @property
+    def sifat_to_en_vocab(self):
+        return self.sifat_level_to_id_to_en_vocab
+
+    def tokenize(
+        self,
+        phonetic_script: list[str] | str,
+        sifat: list[list[SifaOutput | dict]] | list[SifaOutput | dict],
+        to_dict=False,
+        **kwargs,
+    ) -> dict:
+        if isinstance(phonetic_script, str):
+            phonetic_script = [phonetic_script]
+        if not isinstance(sifat[0], list):
+            sifat = [sifat]
+
+        if isinstance(sifat[0][0], dict):
+            sifat = [[SifaOutput(**s) for s in inner_list] for inner_list in sifat]
+
+        level_to_text_list = {}
+        for level in self.levels:
+            if level == "phonemes":
+                text_list = phonetic_script
+            else:
+                text_list = [
+                    "".join(
+                        [SIFAT_ATTR_TO_ARABIC[getattr(s, level)] for s in inner_list]
+                    )
+                    for inner_list in sifat
+                ]
+            level_to_text_list[level] = text_list
+
+        level_to_tokenized = {}
+        for level in self.levels:
+            level_to_tokenized[level] = self.level_to_tokenizer[level](
+                level_to_text_list[level], **kwargs
+            )
+
+        if to_dict:
+            out_dict = {"input_ids": {}, "attention_mask": {}}
+            for level in level_to_tokenized:
+                for k in out_dict:
+                    out_dict[k][level] = level_to_tokenized[level][k]
+            return out_dict
+        return level_to_tokenized
+
+    def decode(
+        self, level_to_input_ids: dict[str, Any], place_zeros_in_between=False
+    ) -> dict[str, list[str] | str]:
+        level_to_decoded_outs = {}
+        for level in level_to_input_ids:
+            input_ids = level_to_input_ids[level]
+            if place_zeros_in_between:
+                input_ids = [add_zero_between(ids) for ids in input_ids]
+            level_to_decoded_outs[level] = self.level_to_tokenizer[level].batch_decode(
+                input_ids,
+            )
+        return level_to_decoded_outs
+
+    def get_level_to_id_to_voab(self):
+        vocab = self.get_tokenizer().vocab
+        level_to_ids_to_vocab = {}
+        for level in vocab:
+            level_to_ids_to_vocab[level] = {v: k for k, v in vocab[level].items()}
+        return level_to_ids_to_vocab
+
+    def get_sifat_levels_to_en_name(self):
+        level_to_id_to_vocab = self.get_level_to_id_to_voab()
+        level_to_id_to_en_vocab = {}
+        for level in level_to_id_to_vocab:
+            if level == "phonemes":
+                continue
+            level_to_id_to_en_vocab[level] = {
+                k: SIFAT_ATTR_TO_ENGLISH[v] if k != PAD_TOKEN_IDX else PAD_TOKEN
+                for k, v in level_to_id_to_vocab[level].items()
+            }
+        return level_to_id_to_en_vocab

src/quran_muaalem/modeling/vocab.py

@@ -0,0 +1,67 @@
+import json
+from dataclasses import asdict
+from typing import get_origin, get_args, Literal
+
+from quran_transcript import alphabet as alph
+from quran_transcript import SifaOutput
+
+SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS = {
+    "hams": "همس",
+    "jahr": "جهر",
+    "shadeed": "شديد",
+    "between": "بين الشدة والرخاوة",
+    "rikhw": "رخو",
+    "mofakham": "مفخم",
+    "moraqaq": "مرقق",
+    "low_mofakham": "أدنى المفخم",
+    "monfateh": "منفتح",
+    "motbaq": "مطبق",
+    "safeer": "صفير",
+    "no_safeer": "لا صفير",
+    "moqalqal": "مقلقل",
+    "not_moqalqal": "لا قلقلة",
+    "mokarar": "مكرر",
+    "not_mokarar": "لا تكرار",
+    "motafashie": "متفشي",
+    "not_motafashie": "لا تفشي",
+    "mostateel": "مستطيل",
+    "not_mostateel": "لا إستطالة",
+    "maghnoon": "مغن",
+    "not_maghnoon": "لا غنة",
+}
+SIFAT_ATTR_TO_ARABIC = {
+    k: f"[{v}]" for k, v in SIFAT_ATTR_TO_ARABIC_WITHOUT_BRACKETS.items()
+}
+SIFAT_ATTR_TO_ENGLISH = {v: k for k, v in SIFAT_ATTR_TO_ARABIC.items()}
+
+PAD_TOKEN = "[PAD]"
+PAD_TOKEN_IDX = 0
+
+
+def build_quran_phoneme_script_vocab(path: str):
+    level_to_token_to_idx = {}
+    # Phonemes level
+    phonemes = list(asdict(alph.phonetics).values())
+    level_to_token_to_idx["phonemes"] = {PAD_TOKEN: PAD_TOKEN_IDX}
+    idx = 0
+    for p in phonemes:
+        if idx == PAD_TOKEN_IDX:
+            idx += 1
+        level_to_token_to_idx["phonemes"][p] = idx
+        idx += 1
+
+    for field_name, fieldinfo in SifaOutput.model_fields.items():
+        if get_origin(fieldinfo.annotation) == Literal:
+            level = field_name
+            phonemes = get_args(fieldinfo.annotation)
+            phonemes = [SIFAT_ATTR_TO_ARABIC[p] for p in phonemes]
+            level_to_token_to_idx[level] = {PAD_TOKEN: PAD_TOKEN_IDX}
+            idx = 0
+            for p in phonemes:
+                if idx == PAD_TOKEN_IDX:
+                    idx += 1
+                level_to_token_to_idx[level][p] = idx
+                idx += 1
+
+    with open(path, "w+", encoding="utf-8") as f:
+        json.dump(level_to_token_to_idx, f, ensure_ascii=False, indent=2)

src/quran_muaalem/muaalem_typing.py

@@ -0,0 +1,78 @@
+from dataclasses import dataclass
+import torch
+
+
+@dataclass
+class Unit:
+    """
+    probs: 1D tensors
+    """
+
+    text: str
+    probs: torch.FloatTensor | list[float]
+    ids: torch.LongTensor | list[int]
+
+
+@dataclass
+class SingleUnit:
+    """
+    A dataclass representing the predicted phoneme sequence with:
+        text (str): Concatenated string of all phonemes.
+        probs (Union[torch.FloatTensor, list[float]]):
+            Confidence probabilities for each predicted phoneme (1D tensor).
+        ids (Union[torch.LongTensor, list[int]]) (1D tensor):
+            Token IDs corresponding to each phoneme.
+
+    """
+
+    text: str
+    prob: float
+    idx: int
+
+
+@dataclass
+class Sifa:
+    """
+    following optional properties (each is a SingleUnit or None):
+        - phonemes_group (str): the phonemes associated with the `sifa`
+        - hams_or_jahr (SingleUnit): either `hams` or `jahr`
+        - shidda_or_rakhawa (SingleUnit): either `shadeed`, `between`, or `rikhw`
+        - tafkheem_or_taqeeq (SingleUnit): either `mofakham`, `moraqaq`, or `low_mofakham`
+        - itbaq (SingleUnit): either `monfateh`, or `motbaq`
+        - safeer (SingleUnit): either `safeer`, or `no_safeer`
+        - qalqla (SingleUnit): eithr `moqalqal`, or `not_moqalqal`
+        - tikraar (SingleUnit): either `mokarar` or `not_mokarar`
+        - tafashie (SingleUnit): either `motafashie`, or `not_motafashie`
+        - istitala (SingleUnit): either `mostateel`, or `not_mostateel`
+        - ghonna (SingleUnit): either `maghnoon`, or `not_maghnoon`
+
+    Each SingleUnit in Sifa properties contains:
+        text (str): The feature's categorical label (e.g., "hams", "shidda").
+        prob (float): Confidence probability for this feature.
+        idx (int): Identifier for the feature class.
+
+    """
+
+    phonemes_group: str
+    hams_or_jahr: SingleUnit | None
+    shidda_or_rakhawa: SingleUnit | None
+    tafkheem_or_taqeeq: SingleUnit | None
+    itbaq: SingleUnit | None
+    safeer: SingleUnit | None
+    qalqla: SingleUnit | None
+    tikraar: SingleUnit | None
+    tafashie: SingleUnit | None
+    istitala: SingleUnit | None
+    ghonna: SingleUnit | None
+
+
+@dataclass
+class MuaalemOutput:
+    """
+    text (str): The feature's categorical label (e.g., "hams", "shidda").
+    prob (float): Confidence probability for this feature.
+    idx (int): Identifier for the feature class.
+    """
+
+    phonemes: Unit
+    sifat: list[Sifa]