Align ASR model card with paper and simplify installation and training

README.md

@@ -5,75 +5,61 @@ language:
 - zh
 base_model:
 - Qwen/Qwen3-ASR-1.7B
+- FireRedTeam/FireRedASR2-AED
 tags:
 - wenetspeech-min
 - minnan
 - southern-min
 - speech-recognition
-- speech-translation
+- paired-transcript-asr
 - qwen3-asr
 ---
 
 # WenetSpeech-Min ASR Models
 
-WenetSpeech-Min ASR provides models for **Minnan (Southern Min) speech with two
-transcription targets**: a dialect-preserving Minnan transcript and a Mandarin
-transcript of the same speech.
+These ASR models accompany
+[WenetSpeech-Min: A Large-Scale Minnan Speech Corpus with Dual Transcriptions for Dialectal Speech Processing](https://arxiv.org/abs/2609.36834).
+WenetSpeech-Min comprises around **10,000 hours of Minnan speech** from online
+media, pairing each utterance with a Minnan transcript and a Mandarin transcript.
 
-The training corpus contains approximately **10,000 hours of Minnan audio**,
-with both Minnan and Mandarin transcripts for each utterance. The input audio
-is Minnan for both tasks; “Mandarin” describes the output transcript, not a
-separate Mandarin-audio corpus. **The training dataset has not been publicly
-released.** Model weights and training/inference recipes are provided separately
-from the training data.
+The models support **Minnan transcription**, **Mandarin transcription**, and
+**paired-transcript recognition**. Minnan transcripts retain dialectal wording;
+Mandarin transcripts express the same content in written Mandarin, without
+requiring word-by-word alignment with the spoken Minnan.
 
 [Code](https://github.com/ASLP-lab/WenetSpeech-Min-Repo) |
-[Qwen3-ASR base model](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) |
-[FireRedASR2-AED base model](https://huggingface.co/FireRedTeam/FireRedASR2-AED)
+[Paper](https://arxiv.org/abs/2609.36834) |
+[Qwen3-ASR](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) |
+[FireRedASR2-AED](https://huggingface.co/FireRedTeam/FireRedASR2-AED)
 
 ## Available Models
 
-| Model | Input | Output | Framework | Status |
-| --- | --- | --- | --- | --- |
-| [Qwen3-ASR-WSM-Min](https://huggingface.co/ASLP-lab/WS-Min-ASR/tree/main/Qwen3-ASR-WSM-Min) | Minnan speech | Minnan transcript | Qwen3-ASR-1.7B, fine-tuned with ms-swift | Available |
-| [Qwen3-ASR-WSM-Mandarin](https://huggingface.co/ASLP-lab/WS-Min-ASR/tree/main/Qwen3-ASR-WSM-Mandarin) | Minnan speech | Mandarin transcript | Qwen3-ASR-1.7B, fine-tuned with ms-swift | Available |
-| FireRedASR2-AED-WSM | Minnan speech | Minnan or Mandarin transcript, selected by task | FireRedASR2-AED, fine-tuned with a task-conditioned WeNet implementation | In training; weights not uploaded yet |
+| Model | Task | Fine-tuning framework |
+| --- | --- | --- |
+| [Qwen3-ASR-WSM-Min](https://huggingface.co/ASLP-lab/WS-Min-ASR/tree/main/Qwen3-ASR-WSM-Min) | Minnan transcription | ms-swift |
+| [Qwen3-ASR-WSM-Mandarin](https://huggingface.co/ASLP-lab/WS-Min-ASR/tree/main/Qwen3-ASR-WSM-Mandarin) | Mandarin transcription | ms-swift |
+| [FireRedASR2-AED-WSM](https://huggingface.co/ASLP-lab/WS-Min-ASR/tree/main/FireRedASR2-AED-WSM) | Paired-transcript recognition | WeNet |
 
-The two Qwen directories contain separate, complete fine-tuned models. Load the
-selected **subdirectory**, rather than this repository root. FireRedASR2-AED-WSM
-is a separate dual-task model; its future weights require the matching WeNet
-code and cannot be loaded using `Qwen3ASRModel`.
+All three models take Minnan speech as input. Each Qwen checkpoint provides one
+transcription target; FireRedASR2-AED-WSM supports both targets in one model.
 
 ## Install
 
-### Clone the code
-
-```bash
-git clone https://github.com/ASLP-lab/WenetSpeech-Min-Repo.git
-cd WenetSpeech-Min-Repo
-```
-
-The relevant recipe directories are:
+Follow the official installation instructions for the relevant framework:
 
-```text
-Qwen3-ASR-Min/          # Minnan output, ms-swift training and Qwen-ASR inference
-Qwen3-ASR-Mandarin/     # Mandarin output, ms-swift training and Qwen-ASR inference
-FireRedASR2-AED-WSM/    # Task-conditioned WeNet training and dual-transcript inference
-```
+- [Qwen3-ASR environment setup](https://github.com/QwenLM/Qwen3-ASR#environment-setup) for Qwen inference, including the vLLM backend.
+- [ms-swift installation](https://swift.readthedocs.io/en/latest/GetStarted/SWIFT-installation.html) for Qwen fine-tuning.
+- [WeNet installation for training and deployment](https://github.com/wenet-e2e/wenet#install-for-training--deployment) for FireRed fine-tuning and inference.
 
-### Create the Qwen environment
+Clone the training and inference recipes:
 
 ```bash
-conda create -n ws-min-qwen python=3.12 -y
-conda activate ws-min-qwen
-python -m pip install -r Qwen3-ASR-Min/requirements-vllm.txt
+git clone https://github.com/ASLP-lab/WenetSpeech-Min-Repo.git
+cd WenetSpeech-Min-Repo
 ```
 
-The inference recipe uses **qwen-asr 0.0.4**, **Transformers 4.57.6**, and
-**vLLM 0.14.0**. Install a PyTorch/CUDA build compatible with your GPU.
-For the Transformers-only backend, install `requirements.txt` instead of
-`requirements-vllm.txt`. Training dependencies and the separate FireRed
-Python 3.10 / WeNet environment are documented in each recipe directory.
+For FireRed paired-transcript recognition, run
+`bash FireRedASR2-AED-WSM/setup_wenet.sh` to prepare the matching WeNet task-embedding implementation.
 
 ### Model download
 
@@ -156,55 +142,102 @@ bash Qwen3-ASR-Min/infer.sh --backend transformers
 The output contains the audio identifier, path, transcription target, language,
 and recognized text. Several audio paths may follow `--audio` for batch inference.
 
-### FireRed dual-transcript inference
+### FireRed paired-transcript recognition
 
-FireRedASR2-AED-WSM is **still in training**. The code recipe supports inference
-with a local trained checkpoint, its training YAML, and tokenizer/CMVN assets:
+Download the FireRed checkpoint and its tokenizer/CMVN assets:
+
+```python
+from huggingface_hub import snapshot_download
+
+snapshot_download(
+    repo_id="ASLP-lab/WS-Min-ASR",
+    allow_patterns=["FireRedASR2-AED-WSM/*"],
+    local_dir="pretrained_models/WS-Min-ASR",
+)
+```
+
+Run inference from the GitHub repository root:
 
 ```bash
-cd FireRedASR2-AED-WSM
-bash setup_wenet.sh
-# Install the WeNet dependencies described in this directory's README first.
-CUDA_VISIBLE_DEVICES=0 bash infer.sh \
-  --model-dir /path/to/converted_model_assets \
-  --config /path/to/training_output/train.yaml \
-  --checkpoint /path/to/training_output/step_XXXXX.pt \
+CUDA_VISIBLE_DEVICES=0 bash FireRedASR2-AED-WSM/infer.sh \
+  --model-dir pretrained_models/WS-Min-ASR/FireRedASR2-AED-WSM \
   --task both
 ```
 
-`--task minnan` requests only the Minnan transcript, while `--task mandarin`
-requests only the Mandarin transcript. The dual-task implementation uses learned
-continuous task embeddings (`transcribe_minnan` and `translate_zh`) without
-adding task tokens to the output vocabulary. The fine-tuned checkpoint must
-contain the trained task embeddings. The public base model alone is insufficient.
+Use `--task minnan` for the Minnan transcript or `--task mandarin` for the
+Mandarin transcript. `--task both` returns `text_minnan` and `text_mandarin`.
+Override the included sample with `--audio /path/to/audio.wav`.
 
 ## Training
 
-Training requires access to suitable user-provided audio and transcripts; the
-approximately 10,000-hour WenetSpeech-Min training corpus is **not public yet**.
-A shared paired-manifest format is documented in all three recipe directories:
+The [GitHub repository](https://github.com/ASLP-lab/WenetSpeech-Min-Repo) contains
+the training recipes for all three models. Each recipe reads a JSONL manifest,
+with one training example per line. Use the corresponding format below for
+both training and validation manifests. Audio paths must resolve on the
+training machine.
+
+### Qwen3-ASR-WSM-Min: Minnan transcription
+
+[Training code](https://github.com/ASLP-lab/WenetSpeech-Min-Repo/tree/main/Qwen3-ASR-Min)
+
+Supervision uses the Minnan transcripts of WenetSpeech-Min. The ms-swift
+training format is:
 
 ```json
-{"utt":"example_001","wav_path":"audio/example_001.wav","text_minnan":"你欲食啥物？","text_mandarin":"你想吃什么？"}
+{"key":"example_001","messages":[{"role":"user","content":"<audio>"},{"role":"assistant","content":"language Chinese<asr_text>你欲食啥物？"}],"audios":["/path/to/audio/example_001.wav"]}
 ```
 
-This record is an illustrative schema, not a released training sample. The data
-converter for each Qwen recipe selects one transcript; the FireRed converter
-creates two task-labelled examples for each audio recording. Split the data
-before task expansion, and keep validation and final evaluation sets separate.
+```bash
+TRAIN_DATA=/path/to/train.minnan.jsonl \
+VAL_DATA=/path/to/validation.minnan.jsonl \
+bash Qwen3-ASR-Min/train.sh
+```
 
-| Recipe | Supervision | Default training settings |
-| --- | --- | --- |
-| Qwen3-ASR-Min | Minnan transcript | Full fine-tuning, BF16, learning rate `1e-5`, 4 epochs, DeepSpeed ZeRO-2 |
-| Qwen3-ASR-Mandarin | Mandarin transcript | Full fine-tuning, BF16, learning rate `1e-5`, 4 epochs, DeepSpeed ZeRO-2 |
-| FireRedASR2-AED-WSM | Both task-labelled transcripts | WeNet torch DDP, task-embedding learning rate `1e-4`, other parameters `1e-5` |
-
-The Qwen training entry point is `train.sh`, which delegates to the existing
-`run.sh`. Inference uses the new `infer.sh` / `infer.py` entry points. The original
-empty `inference.sh` placeholders are retained unchanged. See each directory's
-README for dependencies, exact configuration, data conversion, hardware-related
-settings, and commands. No training run or benchmark result is implied by this
-code release.
+### Qwen3-ASR-WSM-Mandarin: Mandarin transcription
+
+[Training code](https://github.com/ASLP-lab/WenetSpeech-Min-Repo/tree/main/Qwen3-ASR-Mandarin)
+
+Training combines MinSpeech with the Mandarin transcripts of WenetSpeech-Min.
+Use the same ms-swift schema with the Mandarin transcript as the assistant target:
+
+```json
+{"key":"example_001","messages":[{"role":"user","content":"<audio>"},{"role":"assistant","content":"language Chinese<asr_text>你想吃什么？"}],"audios":["/path/to/audio/example_001.wav"]}
+```
+
+```bash
+TRAIN_DATA=/path/to/train.mandarin.jsonl \
+VAL_DATA=/path/to/validation.mandarin.jsonl \
+bash Qwen3-ASR-Mandarin/train.sh
+```
+
+### FireRedASR2-AED-WSM: paired-transcript recognition
+
+[Training code](https://github.com/ASLP-lab/WenetSpeech-Min-Repo/tree/main/FireRedASR2-AED-WSM)
+
+Paired transcripts supervise one model; two learnable task embeddings select
+the output target, and CTC loss is disabled. Represent each recording as two
+WeNet raw JSONL examples sharing the same audio path:
+
+```json
+{"key":"example_001_transcribe_minnan","wav":"/path/to/audio/example_001.wav","txt":"你欲食啥物？","task":"transcribe_minnan","lang":"zh"}
+{"key":"example_001_translate_zh","wav":"/path/to/audio/example_001.wav","txt":"你想吃什么？","task":"translate_zh","lang":"zh"}
+```
+
+`transcribe_minnan` selects the Minnan transcript; `translate_zh` selects the
+Mandarin transcript. These are the task identifiers used by the implementation.
+
+```bash
+BASE_MODEL_DIR=/path/to/wenet-converted-FireRedASR2-AED \
+TRAIN_DATA=/path/to/train.paired.jsonl \
+VAL_DATA=/path/to/validation.paired.jsonl \
+bash FireRedASR2-AED-WSM/train.sh
+```
+
+`BASE_MODEL_DIR` contains the converted base checkpoint `wenet_firered.pt`,
+`global_cmvn`, `train_bpe1000.model`, and `units.txt`. See the
+[FireRed training recipe](https://github.com/ASLP-lab/WenetSpeech-Min-Repo/tree/main/FireRedASR2-AED-WSM)
+for base-model conversion. Each recipe directory also includes `prepare_data.py`
+for converting paired audio/transcript manifests to its training format.
 
 ## Acknowledgements