Update README.md

README.md

@@ -49,18 +49,19 @@ pipeline_tag: automatic-speech-recognition
 
 ### Model Description
 
-This is an ASR (Automatic Speech Recognition) model inspired by [PARSeq](https://github.com/baudm/parseq/tree/main) and [Whisper](https://github.com/openai/whisper/tree/main). **TrorYongASR** is smaller than Whisper model of openai and supports only Khmer and English languages.
+TrorYongASR is an Encoder-Decoder model for Automatic Speech Recognition (ASR) task.
+It is inspired by [PARSeq](https://github.com/baudm/parseq/tree/main) and [Whisper](https://github.com/openai/whisper/tree/main): the auditory-lingual decoder has only one transformer block.
 
 <div align="center">
 
-| **Model Size**    | Tiny          | Small |
-|:-------------:|:-----------------:|:-----:|
-| **Parameters**    | 29M               | 136M |
+|  **Model Size**   |       Tiny        |        Small        |
+|:-----------------:|:-----------------:|:-------------------:|
+|  **Parameters**   |        29M        |        135M         |
 | **Audio Encoder** | 4 layers, 6 heads | 12 layers, 12 heads |
-| **Text Decoder**  | 1 layer, 12 heads | 1 layer, 24 heads |
-| **Embedding Dim** | 384 | 768 |
-| **Audio Context** | 1500 | 1500 |
-| **Text Context**  | 1024 | 1024 |
+| **Text Decoder**  | 1 layer, 12 heads |  1 layer, 24 heads  |
+| **Embedding Dim** |        384        |         768         |
+| **Audio Context** |       1500        |        1500         |
+| **Text Context**  |       1024        |        1024         |
 </div>
 
 **Note:** The audio array are processed to log-mel spectrogram with `80` mels (the same as Whisper models of the same size)
@@ -75,13 +76,12 @@ This is an ASR (Automatic Speech Recognition) model inspired by [PARSeq](https:/
 <!-- Provide the basic links for the model. -->
 
 - **Repository:** https://github.com/Kimang18/KrorngAI/tree/main/tror-yong-asr
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Blog Post:** https://kimang18.github.io/krorngai-blog/TrorYongASR/
+- **Demo [optional]:** TBA
 
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
 The evaluation assesses two capabilities — language detection and transcription — on two datasets ([`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh) for Khmer and [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) for English). All results are from the **test split** of each dataset, representing the model's generalization ability to unseen data.
 
 ### Testing Data
@@ -90,13 +90,13 @@ The evaluation assesses two capabilities — language detection and transcriptio
 
 <div align="center">
 
-| Dataset               | Language   | Testing examples | Description                                    |
-| ---------             | ---------- | ------------- | -                                                 |
-| **google/fleurs**     | Khmer      | 765           | Multi-lingual dataset with Khmer language samples |
-| **librispeech.clean** | English    | 2620          | Clean speech dataset for English transcription    |
+| Dataset               | Language   | Testing examples | Description                                       |
+| -------------         | ---------- | -------------    | -                                                 |
+| **google/fleurs**     | Khmer      | 765              | Multi-lingual dataset with Khmer language samples |
+| **librispeech.clean** | English    | 2620             | Clean speech dataset for English transcription    |
 </div>
 
-**Note:** All evaluation results below are from the **test split** of each dataset. Audios longer than `30 seconds` are excluded from the evaluation (that is why `google/fleurs` has 765 examples instead of 771).
+**Note:** Audios longer than `30 seconds` are excluded from the evaluation (that is why `google/fleurs` has 765 examples instead of 771).
 
 ### Metrics and Results
 
@@ -104,96 +104,84 @@ The evaluation assesses two capabilities — language detection and transcriptio
 
 #### Language Detection
 
-**Task:** Given audio input, detect the language.
+Language detection measures model’s capability to recognize the spoken language from audio input. Since TrorYongASR currently supports 2 languages, this task becomes binary classification task. Classic metrics are used:
 
-<div align="center">
-  
-| Metric | Description |
-|--------|-------------|
-| **Precision** | Proportion of predicted languages that are correct |
-| **Recall** | Proportion of actual language samples correctly identified |
-| **Accuracy** | Proportion of total predictions that are correct |
-| **F1-score** | Harmonic mean of precision and recall |
-</div>
+- **Precision**: Proportion of predicted languages that are correct
+- **Recall** : Proportion of actual language samples correctly identified
+- **F1-score** : Harmonic mean of precision and recall
 
 **Results:**
 
 <div align="center">
 
-| Model | Metrics | Khmer (`fleurs`) | English (`librispeech.clean`) |
-|-------|---------|------------------|-----------------------------|
-| Tiny | Precision | 100% | 100% |
-|  | Recall | 100% | 100% |
-|  | Accuracy | 100% | 100% |
-|  | F1-score | 100% | 100% |
-| Small | Precision | 100% | 100% |
-|  | Recall | 100% | 100% |
-|  | Accuracy | 100% | 100% |
-|  | F1-score | 100% | 100% |
+| Model | Metrics   | Khmer (`fleurs`) | English (`librispeech.clean`) |
+|-------|-----------|------------------|-------------------------------|
+| Tiny  | Precision | 100%             | 100%                          |
+|       | Recall    | 100%             | 100%                          |
+|       | F1-score  | 100%             | 100%                          |
+| Small | Precision | 100%             | 99%                          |
+|       | Recall    | 96%              | 100%                          |
+|       | F1-score  | 98%              | 99%                          |
 </div>
 
-**Key Finding:** Both model sizes achieved perfect language detection performance on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio.
+Tiny size achieved perfect language detection performance on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. Small size performs slightly worst by tending to predict English language.
 
-**Note on Language Detection Performance:** The 100% language detection scores may appear unusually high. This is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
+The 100% language detection scores may appear unusually high. This is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
 
 
 #### Transcription
 
-**Task:** Convert audio to text (transcription).
+For transcription task, 3 metrics below are used
 
-<div align="center">
-  
-| Metric | Description |
-|--------|-------------|
-| **Token Error Rate** | Proportion of incorrectly transcribed tokens |
-| **Character Error Rate (CER)** | Proportion of characters that are incorrect |
-| **Word Error Rate (WER)** | Proportion of words that are incorrect |
-</div>
+- **Token Error Rate (TER)** : Proportion of incorrectly transcribed tokens
+- **Character Error Rate (CER)** : Proportion of characters that are incorrect
+- **Word Error Rate (WER)** : Proportion of words that are incorrect
 
-**Note on Token Error Rate:** Token Error Rate measures model's capability in predicting the next token given the audio input and the current sequence of tokens. This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER) because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively. Token Error Rate is used here because Khmer text lacks word boundaries, making WER and CER calculations challenging without additional preprocessing.
+Token Error Rate (TER) measures model's capability in predicting the next token given the audio input and the current sequence of tokens. This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER) because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively. Token Error Rate is used here because Khmer text lacks word boundaries, making WER and CER calculations challenging without additional preprocessing.
 
-**Note on Translation Task:** The models are also trained for `translation` task, but evaluation is deferred to future work due to scarce data (there are only 2000 examples from Khmer audio to English text, and 1000 examples from English audio to Khmer text in the pre-training).
 
 **Transcription Results:**
 
 <div align="center">
 
-| Model | Metric | Khmer (`fleurs`) | English (`librispeech.clean`) | Mixed (Khmer + English) |
-|-------|--------|-------|---------|---------|
-| **Tiny** | Word Error Rate (WER) | 86.16% | 31.13% | 46.53% |
-| | Character Error Rate (CER) | 60.71% | 20.98% | 32.89% |
-| | Token Error Rate | 56% | 19% | 29% |
-| **Small** | Word Error Rate (WER) | 50.70% | 12.95% | 23.52% |
-| | Character Error Rate (CER) | 35.31% | 7.08% | 15.54% |
-| | Token Error Rate | 46% | 10% | 19% |
+| Model     | Metric                     | Khmer (`fleurs`) | English (`librispeech.clean`) | Mixed (Khmer + English) |
+|-----------|----------------------------|------------------|-------------------------------|-------------------------|
+| **Tiny**  | WER | 75.81%           | 54.33%                        | 60.36%                  |
+|           | CER | 54.99%           | 42.41%                        | 46.18%                  |
+|           | TER | 54%              | 17%                           | 27%                     |
+| **Small** | WER | 50.46%           | 21.75%                        | 29.78%                  |
+|           | CER | 35.89%           | 16.58%                        | 22.37%                  |
+|           | TER | 43%              | 8%                            | 18%                     |
 </div>
 
 **Key Observations:**
-- The tiny model shows strong performance on English (31.13% WER, 20.98% CER, 19% token error rate)
-- Performance drops significantly for Khmer (86.16% WER, 60.71% CER, 56% token error rate)
-- The small model shows strong performance on English (12.95% WER, 7.08% CER, 10% token error rate)
-- Performance for Khmer is moderate (50.70% WER, 35.31% CER, 46% token error rate)
-- The larger model benefits from increased embedding dimension (768 vs 384) and more layers (12 vs 4)
+
+- The tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER)
+- Performance drops significantly for Khmer (75.88% WER, 54.99% CER, 54% TER)
+- The small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER)
+- Performance for Khmer is moderate (50.46% WER, 35.89% CER, 43% TER)
+- The larger model benefits from increased embedding dimension (768 vs 384) and more layers for audio encoder (12 vs 4)
 
 **Note:** To compute `CER` and `WER`, whitespaces are added between words in Khmer text (Khmer text does not have word boundaries like English text). To do so, `khmercut` PyPI package is used to tokenize Khmer text into words, and then the words are joined back together with whitespaces.
 
 
 **WER Comparison with Whisper:**
 
-| Tiny | Parameters | Khmer (`fleurs`) | English (`librispeech.clean`) |
-|-------|--------|---------------------------| --- |
-| TrorYongASR | 29M | 86.16% | 31.13% |
-| Whisper | 39M | 100.6% | 7.6% |
+| Tiny        | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
+| -------     | --------   | --------------------------- | ---                           |
+| TrorYongASR | 29M        | 75.88%                      | 54.33%                        |
+| Whisper     | 39M        | 100.6%                      | 7.6%                          |
 
-| Small | Parameters | Khmer (`fleurs`) | English (`librispeech.clean`) |
-|-------|--------|---------------------------| --- |
-| TrorYongASR | 136M | 50.70% | 12.95% |
-| Whisper | 244M | 104.4% | 3.4% |
+| Small       | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
+| -------     | --------   | --------------------------- | ---                           |
+| TrorYongASR | 135M       | 50.46%                      | 21.75%                        |
+| Whisper     | 244M       | 104.4%                      | 3.4%                          |
 
 **Key Observations:**
-- Whisper models have more parameters for comparable sizes (39M vs 29M for Tiny, 244M vs 136M for Small)
-- Whisper shows significantly lower word error rates on English (7.6% vs 31.13% for Tiny, 3.4% vs 12.95% for Small)
-- Whisper performs worse on Khmer (100.6% vs 86.16% for Tiny, 104.4% vs 50.70% for Small)
+
+- Whisper models have more parameters for comparable sizes (39M vs 29M for Tiny, 244M vs 135M for Small)
+- Whisper shows significantly lower word error rates on English (7.6% vs 54.33% for Tiny, 3.4% vs 12.95% for Small)
+- Whisper performs worse on Khmer (100.6% vs 75.88% for Tiny, 104.4% vs 50.46% for Small)
 - Error rates > 100% for Whisper on Khmer indicate the model is overfitting to the training data
 
 **Note:** `WER` data of Whisper is taken from their [paper](https://arxiv.org/abs/2212.04356).
@@ -201,9 +189,11 @@ The evaluation assesses two capabilities — language detection and transcriptio
 
 ### Result Summary
 
-**Language Detection:** Both model sizes achieved perfect 100% performance across all metrics (Precision, Recall, Accuracy, F1-score) on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. This perfect score is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
+**Language Detection:** Both model sizes achieved great performance across all metrics (Precision, Recall, F1-score) on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. This high score is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
 
-**Transcription:** The Small model shows strong performance on English (12.95% WER, 7.08% CER, 10% token error rate) and moderate performance for Khmer (50.70% WER, 35.31% CER, 46% token error rate). The Tiny model shows strong performance on English (31.13% WER, 20.98% CER, 19% token error rate) but significantly lower performance for Khmer (86.16% WER, 60.71% CER, 56% token error rate). This shows that `TrorYongASR` can be scaled to get higher performance (both models have the same pre-training configuration. See Section Training Details below).
+**Transcription:** The Small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER) and moderate performance for Khmer (50.46% WER, 35.89% CER, 43% TER). The Tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER) but significantly lower performance for Khmer (75.88% WER, 54.99% CER, 54% TER). This shows that TrorYongASR can be scaled to get higher performance.
+
+**Note on Translation Task:** The models are also trained for `translation` task, but evaluation is deferred to future work due to scarce data (there are only 2000 examples from Khmer audio to English text, and 1000 examples from English audio to Khmer text in the pre-training).
 
 
 ## How to Get Started with the Model
@@ -220,7 +210,7 @@ from transformers import AutoProcessor
 from tror_yong_asr import TrorYongASRModel, transcribe, translate, detect_language
 
 
-model_id = "KrorngAI/tror-yong-asr-tiny"
+model_id = "KrorngAI/TrorYongASR-tiny"
 processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
 model = TrorYongASRModel.from_pretrained(model_id, trust_remote_code=True)
 
@@ -317,35 +307,45 @@ For translation task, the data was scarce: only 2000 examples for Khmer audio to
 #### Preprocessing
 
 Following `Whisper` model of openai, audios with duration longer than 30 seconds are filtered out.
-All audios have `16, 000` sample rate.
+All audios have `16000` sample rate.
 For English dataset, all texts are in lowercase.
 
 #### Training Hyperparameters
 
-- **Training regime:** bf16 mixed precision training using `LightningAI` package <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Training regime:** 16-mixed precision training using `LightningAI` package <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 - **Optimizer:** MuonAdamW (custom implementation)
-- **Learning rate:** Linear Warmup (40 optimizer steps) + CosineAnnealing (3960 optimizer steps)
+- **Learning rate:** Linear Warmup (38 optimizer steps) + CosineAnnealing (3774 optimizer steps)
 - **Weight decay:** 0.1
 - **Effective Batch size:** 64
-- **Number of optimizer steps:** 4000
+- **Number of optimizer steps:** 3812
 - **Number of epochs:** roughly 2 epochs
 - **Gradient Clip Value:** 0.5 (only for parameters trained by AdamW)
 
 
 #### Speeds, Sizes, Times
 
-The training was conducted over 4000 optimizer steps.
-For tiny variant, the training took around 10 hours on 1 Tesla T4 GPU.
-For small variant, the training took around 2 hours on 1 A100 GPU.
+The training was conducted over 3812 optimizer steps.
 
+- For tiny variant, the training took around 6 hours on 1 Tesla T4 GPU.
+- For small variant, the training took around 7 hours on 2 Tesla T4 GPU (using DDP strategy).
 
-## Citation [optional]
+
+## Citation
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
 **BibTeX:**
 
-[More Information Needed]
+```bibtex
+@online{khun2026,
+  author = {Khun, Kimang},
+  title = {TrorYongASR: {Permuted} {AutoRegressive} {Sequence}
+    {Modeling} for {Automatic} {Speech} {Recognition}},
+  date = {2026-05-07},
+  url = {https://kimang18.github.io/krorngai-blog/TrorYongASR/},
+  langid = {en}
+}
+```
 
 
 ## Model Card Author
@@ -361,8 +361,6 @@ So, huge thanks to [`LightningAI`](https://lightning.ai) and `Google Colab`.
 
 Thanks to the authors of [`PARSeq`](https://github.com/baudm/parseq/tree/main) and [`Whisper`](https://github.com/openai/whisper/tree/main) for their publicly available sourcecode.
 
-Thanks to [`openslr`](https://openslr.org), [Mozilla Data Collective](https://mozilladatacollective.com/datasets/cml9h5vgc01bxmn075sjeftek) and Google for their publicly available dataset.
-
 ## Model Card Contact
 
 If you have any questions, please reach out at [Facebook Page](https://www.facebook.com/profile.php?id=61582509385293).