Update README.md

README.md

@@ -1,199 +1,366 @@
 ---
 library_name: transformers
-tags: []
+license: other
+license_name: modified-mit
+datasets:
+- DDD-Cambodia/khm-asr-cultural
+- openslr/librispeech_asr
+- KrorngAI/fleurs-km-kh-openslr-SLR42
+language:
+- km
+- en
+metrics:
+- wer
+- cer
+- ter
+pipeline_tag: automatic-speech-recognition
 ---
+<div align="center">
+  <picture>
+      <img src="figures/krorngai.png" width="30%" alt="KrorngAI">
+  </picture>
+</div>
+<hr>
+<!--
+<div align="center" style="line-height:1">
+  <a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K2.6-ff6b6b?color=1783ff&logoColor=white"/></a>
+  <a href="https://www.facebook.com/profile.php?id=61582509385293" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Krorng%20AI-white?logoColor=white"/></a>
+</div>
+-->
+
+<div align="center" style="line-height: 1;">
+  <a href="https://huggingface.co/KrorngAI" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Krorng%20AI-ffc107?color=ffc107&logoColor=white"/></a>
+  <a href="https://youtube.com/@krorngai" target="_blank"><img alt="YouTube Channel" src="https://img.shields.io/badge/Youtube-Krorng%20AI-red?logoColor=red"/></a>
+  <a href="https://www.facebook.com/profile.php?id=61582509385293" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Facebook-Krorng%20AI-blue?logoColor=blue"/></a>
+  <a href="https://kimang18.github.io" target="_blank"><img alt="Personal" src="https://img.shields.io/badge/KHUN-white?logoColor=white"/></a>
+</div>
+<div align="center" style="line-height: 1;">
+  <a href="https://huggingface.co/moonshotai/Kimi-K2.6/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>
+</div>
+
+
+# TrorYongASR
+
+> [!Note]
+> This repository contains model weights and configuration files for the pre-trained model. 
+>
 
-# Model Card for Model ID
+## Model Details
 
-<!-- Provide a quick summary of what the model is/does. -->
+### Model Description
 
+TrorYongASR is an Encoder-Decoder model for Automatic Speech Recognition (ASR) task.
+It is inspired by [PARSeq](https://github.com/baudm/parseq/tree/main) and [Whisper](https://github.com/openai/whisper/tree/main): the auditory-lingual decoder has only one transformer block.
 
+<div align="center">
 
-## Model Details
+|  **Model Size**   |       Tiny        |        Small        |
+|:-----------------:|:-----------------:|:-------------------:|
+|  **Parameters**   |        29M        |        135M         |
+| **Audio Encoder** | 4 layers, 6 heads | 12 layers, 12 heads |
+| **Text Decoder**  | 1 layer, 12 heads |  1 layer, 24 heads  |
+| **Embedding Dim** |        384        |         768         |
+| **Audio Context** |       1500        |        1500         |
+| **Text Context**  |       1024        |        1024         |
+</div>
 
-### Model Description
+**Note:** The audio array are processed to log-mel spectrogram with `80` mels (the same as Whisper models of the same size)
 
-<!-- Provide a longer summary of what this model is. -->
+- **Developed by:** KHUN Kimang (Ph.D.)
+- **Shared by:** KrorngAI
+- **Model type:** ASR (Automatic Speech Recognition)
+- **Language(s) (NLP):** Khmer and English
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+### Model Sources
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+<!-- Provide the basic links for the model. -->
 
-### Model Sources [optional]
+- **Repository:** https://github.com/Kimang18/KrorngAI/tree/main/tror-yong-asr
+- **Blog Post:** https://kimang18.github.io/krorngai-blog/TrorYongASR/
+- **Demo [optional]:** TBA
 
-<!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+## Evaluation
 
-## Uses
+The evaluation assesses two capabilities — language detection and transcription — on two datasets ([`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh) for Khmer and [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) for English). All results are from the **test split** of each dataset, representing the model's generalization ability to unseen data.
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+### Testing Data
 
-### Direct Use
+<!-- This should link to a Dataset Card if possible. -->
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+<div align="center">
 
-[More Information Needed]
+| Dataset               | Language   | Testing examples | Description                                       |
+| -------------         | ---------- | -------------    | -                                                 |
+| **google/fleurs**     | Khmer      | 765              | Multi-lingual dataset with Khmer language samples |
+| **librispeech.clean** | English    | 2620             | Clean speech dataset for English transcription    |
+</div>
 
-### Downstream Use [optional]
+**Note:** Audios longer than `30 seconds` are excluded from the evaluation (that is why `google/fleurs` has 765 examples instead of 771).
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+### Metrics and Results
 
-[More Information Needed]
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
 
-### Out-of-Scope Use
+#### Language Detection
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+Language detection measures model’s capability to recognize the spoken language from audio input. Since TrorYongASR currently supports 2 languages, this task becomes binary classification task. Classic metrics are used:
 
-[More Information Needed]
+- **Precision**: Proportion of predicted languages that are correct
+- **Recall** : Proportion of actual language samples correctly identified
+- **F1-score** : Harmonic mean of precision and recall
 
-## Bias, Risks, and Limitations
+**Results:**
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+<div align="center">
 
-[More Information Needed]
+| Model | Metrics   | Khmer (`fleurs`) | English (`librispeech.clean`) |
+|-------|-----------|------------------|-------------------------------|
+| Tiny  | Precision | 100%             | 100%                          |
+|       | Recall    | 100%             | 100%                          |
+|       | F1-score  | 100%             | 100%                          |
+| Small | Precision | 100%             | 99%                          |
+|       | Recall    | 96%              | 100%                          |
+|       | F1-score  | 98%              | 99%                          |
+</div>
 
-### Recommendations
+Tiny size achieved perfect language detection performance on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. Small size performs slightly worst by tending to predict English language.
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+The 100% language detection scores may appear unusually high. This is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
-## How to Get Started with the Model
+#### Transcription
 
-Use the code below to get started with the model.
+For transcription task, 3 metrics below are used
 
-[More Information Needed]
+- **Token Error Rate (TER)** : Proportion of incorrectly transcribed tokens
+- **Character Error Rate (CER)** : Proportion of characters that are incorrect
+- **Word Error Rate (WER)** : Proportion of words that are incorrect
 
-## Training Details
+Token Error Rate (TER) measures model's capability in predicting the next token given the audio input and the current sequence of tokens. This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER) because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively. Token Error Rate is used here because Khmer text lacks word boundaries, making WER and CER calculations challenging without additional preprocessing.
 
-### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+**Transcription Results:**
 
-[More Information Needed]
+<div align="center">
 
-### Training Procedure
+| Model     | Metric                     | Khmer (`fleurs`) | English (`librispeech.clean`) | Mixed (Khmer + English) |
+|-----------|----------------------------|------------------|-------------------------------|-------------------------|
+| **Tiny**  | WER | 75.81%           | 54.33%                        | 60.36%                  |
+|           | CER | 54.99%           | 42.41%                        | 46.18%                  |
+|           | TER | 54%              | 17%                           | 27%                     |
+| **Small** | WER | 50.46%           | 21.75%                        | 29.78%                  |
+|           | CER | 35.89%           | 16.58%                        | 22.37%                  |
+|           | TER | 43%              | 8%                            | 18%                     |
+</div>
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+**Key Observations:**
 
-#### Preprocessing [optional]
+- The tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER)
+- Performance drops significantly for Khmer (75.88% WER, 54.99% CER, 54% TER)
+- The small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER)
+- Performance for Khmer is moderate (50.46% WER, 35.89% CER, 43% TER)
+- The larger model benefits from increased embedding dimension (768 vs 384) and more layers for audio encoder (12 vs 4)
 
-[More Information Needed]
+**Note:** To compute `CER` and `WER`, whitespaces are added between words in Khmer text (Khmer text does not have word boundaries like English text). To do so, `khmercut` PyPI package is used to tokenize Khmer text into words, and then the words are joined back together with whitespaces.
 
 
-#### Training Hyperparameters
+**WER Comparison with Whisper:**
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+| Tiny        | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
+| -------     | --------   | --------------------------- | ---                           |
+| TrorYongASR | 29M        | 75.88%                      | 54.33%                        |
+| Whisper     | 39M        | 100.6%                      | 7.6%                          |
 
-#### Speeds, Sizes, Times [optional]
+| Small       | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
+| -------     | --------   | --------------------------- | ---                           |
+| TrorYongASR | 135M       | 50.46%                      | 21.75%                        |
+| Whisper     | 244M       | 104.4%                      | 3.4%                          |
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+**Key Observations:**
 
-[More Information Needed]
+- Whisper models have more parameters for comparable sizes (39M vs 29M for Tiny, 244M vs 135M for Small)
+- Whisper shows significantly lower word error rates on English (7.6% vs 54.33% for Tiny, 3.4% vs 12.95% for Small)
+- Whisper performs worse on Khmer (100.6% vs 75.88% for Tiny, 104.4% vs 50.46% for Small)
+- Error rates > 100% for Whisper on Khmer indicate the model is overfitting to the training data
 
-## Evaluation
+**Note:** `WER` data of Whisper is taken from their [paper](https://arxiv.org/abs/2212.04356).
 
-<!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
+### Result Summary
 
-#### Testing Data
+**Language Detection:** Both model sizes achieved great performance across all metrics (Precision, Recall, F1-score) on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. This high score is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
 
-<!-- This should link to a Dataset Card if possible. -->
+**Transcription:** The Small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER) and moderate performance for Khmer (50.46% WER, 35.89% CER, 43% TER). The Tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER) but significantly lower performance for Khmer (75.88% WER, 54.99% CER, 54% TER). This shows that TrorYongASR can be scaled to get higher performance.
 
-[More Information Needed]
+**Note on Translation Task:** The models are also trained for `translation` task, but evaluation is deferred to future work due to scarce data (there are only 2000 examples from Khmer audio to English text, and 1000 examples from English audio to Khmer text in the pre-training).
 
-#### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+## How to Get Started with the Model
 
-[More Information Needed]
+First, install `tror-yong-asr` PyPI package:
+```bash
+pip install tror-yong-asr
+```
 
-#### Metrics
+Then, use the code below to get started with the model.
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+```python
+from transformers import AutoProcessor
+from tror_yong_asr import TrorYongASRModel, transcribe, translate, detect_language
+
+
+model_id = "KrorngAI/TrorYongASR-tiny"
+processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
+model = TrorYongASRModel.from_pretrained(model_id, trust_remote_code=True)
+
+result1 = detect_language('/path/to/audio_file.mp3', model, processor)
+print(result1)
+
+result2 = transcribe('/path/to/audio_file.mp3', model, processor, max_tokens=64)
+print(result2)
 
-[More Information Needed]
+result3 = translate('/path/to/audio_file.mp3', model, processor, max_tokens=64)
+print(result3)
+```
 
-### Results
+## Fine-tuning
 
-[More Information Needed]
+Notebook (TBA)
 
-#### Summary
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+The Tiny model can be used directly for:
+- **Speech-to-text transcription**: transcribe Khmer and English audio
+- **Speech-to-text translation**: translate Khmer audio to English text and English audio to Khmer text
+- **Language detection**: Identify whether audio is in Khmer or English (100% accuracy)
+- **Edge computing**: Deploy on mobile devices, IoT devices, and embedded systems due to its small size (29M parameters)
+- **Real-time applications**: Low latency inference suitable for real-time speech interfaces
 
 
+### Downstream Use [optional]
 
-## Model Examination [optional]
+The model can be integrated into:
+- **Mobile applications**: Android/iOS apps with speech recognition
+- **Web applications**: Browser-based speech-to-text using WebAssembly
+- **IoT devices**: Smart speakers, voice assistants
+- **Larger ASR systems**: As a component in multi-language ASR pipelines
 
-<!-- Relevant interpretability work for the model goes here -->
 
-[More Information Needed]
+## Bias, Risks, and Limitations
 
-## Environmental Impact
+**Technical Limitations:**
+- **No speech detection**: The model was not trained for this task. User needs to fine-tune the model for this task (TrorYongASRTokenizer has `<|nospeech|>` token.)
+- **Translate task**: The training data for translation task is scarce. User needs to fine-tune the model for better translation performance
+- **Noise robustness**: Performance may degrade in noisy environments
+- **No timestamp output**: The model does not support timestamp output
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+**Sociotechnical Limitations:**
+- **Accent variability**: May not perform well on diverse Khmer accents
+- **Background noise**: Limited robustness to background noise and reverberation
+- **Speaker variability**: May struggle with different speaking styles and rates
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+### Recommendations
 
-## Technical Specifications [optional]
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-### Model Architecture and Objective
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
-[More Information Needed]
 
-### Compute Infrastructure
+## Training Details
 
-[More Information Needed]
+To capture model's scalability, both tiny and small variants were trained using the same configuration detailed below.
 
-#### Hardware
+### Training Data
 
-[More Information Needed]
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-#### Software
+#### Transcription Task
 
-[More Information Needed]
+For transcription task, the model was trained on around 140 hours of Khmer audio and around 100 hours of English audio.
+Khmer datasets include [`DDD-Cambodia/khm-asr-cultural`](https://huggingface.co/datasets/DDD-Cambodia/khm-asr-cultural) (134.6 hours), [`openslr/openslr`](https://huggingface.co/datasets/Kimang18/openslr-SLR42/blob/main/README.md), and [`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh).
+Split `clean.100` of [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) was used for English dataset.
 
-## Citation [optional]
+<div align="center">
+  
+| Dataset                            | Language   | Training examples | Validation examples | Description                                       |
+| ---------                          | ---------- | ----------------- | ------------------- |-                                                  |
+| **DDD-Cambodia/khm-asr-cultural**  | Khmer      | 56716             | 0                   | Khmer ASR Cultural Dataset  (split `train`) |
+| **openslr/openslr**                | Khmer      | 2906              | 0                   | Multi-speaker TTS data for Khmer language (split `SLR42`) |
+| **google/fleurs**                  | Khmer      | 1675              | 324                 | TTS data for Khmer language (split `km_kh`) |
+| **librispeech\_asr.clean**              | English    | 28539             | 2703                | Clean speech dataset for English transcription    |
+</div>
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+#### Translation Task
 
-**BibTeX:**
+For translation task, the data was scarce: only 2000 examples for Khmer audio to English text, and only 1000 examples for English audio to Khmer text.
 
-[More Information Needed]
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing
+
+Following `Whisper` model of openai, audios with duration longer than 30 seconds are filtered out.
+All audios have `16000` sample rate.
+For English dataset, all texts are in lowercase.
+
+#### Training Hyperparameters
+
+- **Training regime:** 16-mixed precision training using `LightningAI` package <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Optimizer:** MuonAdamW (custom implementation)
+- **Learning rate:** Linear Warmup (38 optimizer steps) + CosineAnnealing (3774 optimizer steps)
+- **Weight decay:** 0.1
+- **Effective Batch size:** 64
+- **Number of optimizer steps:** 3812
+- **Number of epochs:** roughly 2 epochs
+- **Gradient Clip Value:** 0.5 (only for parameters trained by AdamW)
 
-**APA:**
 
-[More Information Needed]
+#### Speeds, Sizes, Times
+
+The training was conducted over 3812 optimizer steps.
+
+- For tiny variant, the training took around 6 hours on 1 Tesla T4 GPU.
+- For small variant, the training took around 7 hours on 2 Tesla T4 GPU (using DDP strategy).
+
+
+## Citation
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
 
-## Glossary [optional]
+```bibtex
+@online{khun2026,
+  author = {Khun, Kimang},
+  title = {TrorYongASR: {Permuted} {AutoRegressive} {Sequence}
+    {Modeling} for {Automatic} {Speech} {Recognition}},
+  date = {2026-05-07},
+  url = {https://kimang18.github.io/krorngai-blog/TrorYongASR/},
+  langid = {en}
+}
+```
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
 
-[More Information Needed]
+## Model Card Author
 
-## More Information [optional]
+- ឈ្មោះ: បណ្ឌិត ឃុន គីមអាង 
+- Name: KHUN Kimang (Ph.D.)
 
-[More Information Needed]
+## Acknowledgement
 
-## Model Card Authors [optional]
+[`LightningAI`](https://lightning.ai) and `Google Colab` did not specifically sponsor this project.
+But, both models are be trained thanks to their free credits.
+So, huge thanks to [`LightningAI`](https://lightning.ai) and `Google Colab`.
 
-[More Information Needed]
+Thanks to the authors of [`PARSeq`](https://github.com/baudm/parseq/tree/main) and [`Whisper`](https://github.com/openai/whisper/tree/main) for their publicly available sourcecode.
 
 ## Model Card Contact
 
-[More Information Needed]
+If you have any questions, please reach out at [Facebook Page](https://www.facebook.com/profile.php?id=61582509385293).