pasketti-word 🍝

pasketti-word is an automatic speech recognition (ASR) model designed specifically for child speech. pasketti-word generates orthographic (per word) transcriptions from audio files.

🐣 Existing open-source ASR models trained on adult speech perform poorly on children because child speech is fundamentally different from adult speech. Children are still learning to produce speech sounds and developing fine motor skills. Common child speech errors include:

Metathesis: "elephant" → "ephelant"
Velar fronting: "cup" → "tup"
Syllable deletion: "banana" → "nana"
Combinations of errors: "spaghetti" → "pasketti" 🍝

🔉 pasketti-word is fine-tuned on a large corpus of child speech data including 372 hours of child speech spanning >3,600 different children. This is the largest dataset of its kind for use in developing and testing open ASR models for kids. The model is designed to advance early education assessments, teaching tools, and needs screening.

📈 pasketti-word achieves an overall word error rate (WER) of 17% on held-out evaluation data. When compared to leading open ASR models (Whisper and KidWhisper) on the same evaluation set, pasketti-word demonstrates improved performance across all demographic groups, cutting the error rate by more than half for children ages 3-7 and achieving close to adult-level performance for children ages 5 and above.

In this model card:

Quickstart
Model Details
Uses
Evaluation
1. Performance by Subset
2. Bias and Limitations
Training Data

Quickstart

Prerequisites

Hardware. Inference runs through vLLM. See the vLLM installation docs for supported hardware and operating systems. This model has been tested only on Linux with NVIDIA GPUs.
Environment. Install qwen_asr with its vLLM extra:
```
pip install "qwen_asr[vllm]==0.0.6"
```
pip install transformers alone cannot load this model. The qwen3_asr architecture is not built into transformers; it is registered when qwen_asr is imported.

This release was verified against qwen_asr==0.0.6, vllm==0.14.0, transformers==4.57.6 and torch==2.9.1, on Python 3.13. qwen_asr pins an exact transformers version, so if newer releases break the install, pin these versions to reconstruct a working environment.
Data. Pass each clip to asr.transcribe(audio=...) in one of these forms:
- a local file path, an https URL, or a base64-encoded audio string, which qwen_asr reads directly, or
- a (waveform, sample_rate) tuple, where waveform is a 1-D (mono) or 2-D (multi-channel) NumPy array and sample_rate is its true sample rate in Hz. To match how the model was trained, load the audio with librosa, as in the example below.

qwen_asr converts every input to 16 kHz mono before transcription.

Inference

Pass the Hub repo id, drivendata/pasketti-word, to Qwen3ASRModel.LLM. The weights download automatically the first time the model loads.

Transcribe a single clip:

from qwen_asr import Qwen3ASRModel
import librosa

asr = Qwen3ASRModel.LLM(
    "drivendata/pasketti-word",
    gpu_memory_utilization=0.9,
    max_inference_batch_size=32,
    max_new_tokens=256,
)
wav, _ = librosa.load("clip.flac", sr=16000, mono=True)
result = asr.transcribe(audio=[(wav, 16000)], language="English")
print(result[0].text)

>> "the cat sat down"

For large datasets, load and transcribe the audio in chunks (for example, a few thousand clips at a time) rather than holding every clip in memory at once.

qwen_asr also offers a transformers backend through Qwen3ASRModel.from_pretrained(). That backend has not been evaluated with this model and is not supported.

Model Details

This model is a supervised fine-tune of Qwen3-ASR-1.7B. Qwen3-ASR pairs an audio encoder with a Qwen3 language model, which writes out the transcript one token at a time. All of the model's weights were updated during fine-tuning, using clips of up to 20 seconds of children's speech with orthographic (word-level) transcripts.

The model was trained for two epochs with a global batch size of 32 (4 per GPU across 8 A100s), using AdamW with a peak learning rate of 3e-5. The schedule used linear warmup over the first 2% of steps, followed by linear decay sized for five epochs.

The model design is based on the results of the On Top of Pasketti Challenge, in which participants competed to build child ASR models. All five prize-winning Word Track solutions fine-tuned Qwen3-ASR-1.7B, and pasketti-word adapts the 3rd-place solution, a full-parameter fine-tune.

Developed by: DrivenData, based on the 3rd-place Word Track solution by Tang Yongqwei (chuxiliyixiaosa)
References
- On Top of Pasketti Challenge
- Overview of competition-winning solutions

Uses

This model is intended to be used for automatic speech recognition in service of improving educational and developmental outcomes for children.

Based on the model's performance, it is most suitable for downstream use cases that depend on child transcripts as an input. These downstream use cases often do not require exact transcripts to effectively inform decision-making.

For use cases that require exact transcription from child speech, model outputs should be human reviewed.

Downstream Use Cases

Example downstream use cases include:

Needs screening: Identifying children who may be struggling and better targeting additional support. For example, identifying children behind in literacy or who may have a speech pathology.
Formative assessment: Monitoring student learning based on naturally collected classroom speech data.
Educator feedback: Enabling better teacher or tutor support and growth by incorporating spoken interactions.

The model could be plugged into decision-making processes as is to generate transcripts for human review, or fine-tuned to predict in-scope outcomes directly. Intended users include education researchers, education technology developers, teachers, and school administrators.

Out-of-Scope Use

The model should not be used in ways that do not serve the interests of children whose speech it processes. Out-of-scope uses include:

Surveillance or profiling of children, such as building voice or behavioral profiles for advertising, identifying individual speakers, or monitoring children without informed consent from them and their guardians.
Engagement maximization, such as designing products that use speech signals to drive compulsive use or unhealthy dependence on technology.
Fully automated high-stakes decisions, such as diagnosis or educational placement, without review by a qualified educator or clinician.

Evaluation

pasketti-word performance: 17% word error rate (WER)

The evaluation set includes 148 hours of child speech from >2,300 different children as young as 3. All models are scored on every utterance with Whisper's English normalizer.

The model is compared to existing state-of-the-art open ASR models openai/whisper-medium and aadel4/kid-whisper-medium-en-myst (Whisper fine-tuned on a small amount of child speech data).

All models were run with loop protection:

Whisper and KidWhisper are decoded greedily with the compression-ratio fallback from OpenAI's transcribe(). Retrying low-confidence predictions is not implemented.
pasketti-word implements loop protection by default. It uses qwen_asr's default decoding, which collapses repeats on every output, with a 256-token limit.

About Word Error Rate

Word Error Rate (WER) measures the minimum number of word-level substitutions (𝑆), deletions (𝐷), and insertions (𝐼) required to transform the predicted transcript into the reference transcript, divided by the total number of words in the reference (𝑁). WER indicates roughly what percent of words predicted are wrong.

$\text{WER} = \frac{\text{Substitutions} + \text{Deletions} + \text{Insertions}}{\text{Words}} = \frac{S + D + I}{N}$

A limitation of WER is that it only measures whether the predicted word is exactly correct — it does not reward similar predictions that are close to correct. As a result, WER often slightly underestimates how well a model can perform in a real-world setting.

Performance by Subset

The model was evaluated for bias in performance based on age, race, speech development pathology, sex, and source corpus. As expected, performance tends to improve for older children compared to younger children. However, performance gains are pronounced for younger children, with WER for children ages 3-4 improving by 30 percentage points compared to Whisper.

Compared to Whisper, pasketti-word decreases key disparities between white vs. non-white children as well as between children with speech pathologies vs. children without.

Performance by Age

Whisper performance is likely poor on ages 12+ because those samples are derived from particularly challenging corpuses.