|
|
--- |
|
|
license: mit |
|
|
base_model: |
|
|
- facebook/wav2vec2-large-robust |
|
|
- aadel4/Wav2vec_Classroom |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
tags: |
|
|
- wav2vec2 |
|
|
library_name: transformers |
|
|
--- |
|
|
## Model Card: Wav2vec_Classroom_FT |
|
|
|
|
|
### Model Overview |
|
|
**Model Name:** Wav2vec_Classroom_FT |
|
|
**Version:** 1.0 |
|
|
**Developed By:** Ahmed Adel Attia (University of Maryland and Stanford University) |
|
|
**Date:** 2025 |
|
|
|
|
|
**Description:** |
|
|
Wav2vec_Classroom_FT is an automatic speech recognition (ASR) model trained for classroom speech transcription using direct fine-tuning on a small set of human-verified gold-standard transcriptions. Unlike **NCTE-WSP-ASR**, this model does not leverage weak transcriptions for intermediate training and is solely trained on high-quality annotations. |
|
|
|
|
|
This model is adapted from **[Wav2vec-Classroom](https://huggingface.co/aadel4/Wav2vec_Classroom)**, which was trained using continued pretraining (CPT) on large-scale unlabeled classroom speech data. The adaptation involves direct fine-tuning on a limited transcribed dataset. |
|
|
|
|
|
This model was originally trained using the fairseq library then ported into Huggingface. |
|
|
|
|
|
**Use Case:** |
|
|
- Speech-to-text transcription for classroom environments. |
|
|
- ASR applications requiring high precision with limited data. |
|
|
- Benchmarking ASR performance without weakly supervised pretraining. |
|
|
|
|
|
### Model Details |
|
|
**Architecture:** Wav2vec2.0-based model fine-tuned with Fairseq |
|
|
|
|
|
**Training Data:** |
|
|
- **NCTE-Gold:** 13 hours of manually transcribed classroom recordings. |
|
|
|
|
|
**Training Strategy:** |
|
|
1. **Direct Fine-tuning:** The model is fine-tuned directly on NCTE-Gold without any pretraining on weak transcripts. |
|
|
2. **Evaluation:** The model is tested on classroom ASR tasks to compare its performance with WSP-based models. |
|
|
|
|
|
### Evaluation Results |
|
|
**Word Error Rate (WER) comparison on NCTE and MPT test sets:** |
|
|
|
|
|
| Training Data | NCTE WER | MPT WER | |
|
|
|--------------|----------|---------| |
|
|
| **Baseline (TEDLIUM-trained ASR)** | 55.82 / 50.56 | 55.11 / 50.50 | |
|
|
| **NCTE-Gold only (NCTE-Baseline-ASR)** | 21.12 / 16.47 | 31.52 / 27.93 | |
|
|
| **NCTE-WSP-ASR (NCTE-Weak → NCTE-Gold)** | **16.54 / 13.51** | **25.07 / 23.70** | |
|
|
|
|
|
### Limitations |
|
|
- The model is trained on a small dataset (13 hours), which limits its ability to generalize beyond classroom speech. |
|
|
- Performance is lower than **NCTE-WSP-ASR**, which benefits from weak transcripts for pretraining. |
|
|
- Background noise, overlapping speech, and speaker variations may still impact transcription quality. |
|
|
|
|
|
### Usage Request |
|
|
If you use the NCTE-Baseline-ASR model in your research, please acknowledge this work and refer to the original paper submitted to Interspeech 2025. |
|
|
|
|
|
For inquiries or collaborations, please contact the authors of the original paper. |
