README.md

---
language: en
library_name: accentscore
pipeline_tag: audio-classification
base_model: microsoft/wavlm-base-plus
license: other
license_name: research-only-mixed-third-party-terms
tags:
  - audio
  - accent-classification
  - pronunciation-assessment
  - wavlm
  - research
---

# AccentScore model card

## Summary

AccentScore 0.2.1 is a local, transcript-free English-pronunciation research model. It
combines a frozen `microsoft/wavlm-base-plus` encoder with three task-specific heads:

- a five-target pronunciation-quality regressor;
- a four-class American/British/Indian/Russian accent classifier;
- an American-vs-non-American classifier mapped to a 0–100 score.

The model accepts one 0.2–30 second audio recording and produces a structured
`PredictionResult`. It does not call an external inference service.

## Usage

Install AccentScore from its source repository, then load the pinned model bundle:

```python
from accentscore import AccentScorer

scorer = AccentScorer.from_pretrained()
result = scorer.predict("recording.wav")
print(result.detected_accent, result.scores)
```

The model heads are downloaded once and cached locally. Inference remains local. The
frozen WavLM encoder is also downloaded at the exact revision recorded in the bundle.

## Intended use

Appropriate uses include local experimentation, pronunciation-practice interfaces,
model evaluation, and research on representation transfer.

The model is not intended for employment, education access, immigration, identity
inference, grading, diagnosis, or any decision that affects a person's rights or
opportunities.

Accent labels refer only to resemblance among the supported acoustic classes. They are
not claims about a speaker's nationality, ethnicity, birthplace, native language, or
identity.

## Architecture

| Component | Configuration |
| --- | --- |
| Encoder | WavLM Base+, frozen, pinned revision |
| Input | Mono 16 kHz waveform |
| Pooling | Masked mean and population standard deviation from 13 layers |
| Quality head | Target-specific softmax layer mixtures and bounded MLP regressors |
| Accent head | Layer 7, L2 logistic regression, temperature scaling |
| American head | Layer 6, binary logistic regression, temperature scaling |
| Quality uncertainty | Split-conformal absolute residual intervals |

The four-class and binary heads are intentionally independent. American similarity is
not computed from pronunciation accuracy or from the four-class probability.

## Training data

| Component | Corpus | Split policy |
| --- | --- | --- |
| Quality | SpeechOcean762 / OpenSLR SLR101 | Official test preserved; train/validation/calibration are speaker-disjoint |
| Accent | AESRC2020 + Speech Accent Archive | Class-stratified, speaker-disjoint train/validation/test partitions |
| American similarity | AESRC2020 | American versus British/Indian/Russian, speaker-disjoint |

SpeechOcean762 is distributed under CC BY 4.0. Operators must independently verify
AESRC2020 and Speech Accent Archive terms, consent, attribution, derivative-work, and
redistribution requirements before using or distributing the accent checkpoints.
The model weights are therefore marked `license: other` and are published for research
and evaluation only; the source repository's MIT license applies to code, not to these
weights or third-party datasets.

## Evaluation

### Pronunciation quality

SpeechOcean762 official test: 2,500 utterances from 125 speakers.

| Metric | Macro result |
| --- | ---: |
| Pearson | 0.7487 |
| Spearman | 0.7486 |
| MAE | 6.109 points |
| RMSE | 9.225 points |
| R² | 0.4182 |
| 90% interval coverage | 0.8922 |

Completeness is almost constant in this corpus. Its correlations are undefined and its
near-perfect interval coverage should not be interpreted as strong modeling.

### Accent tasks

| Held-out source | Accuracy | Macro-F1 |
| --- | ---: | ---: |
| AESRC2020 | 0.9055 | 0.9047 |
| Speech Accent Archive | 0.9390 | 0.8950 |

The American-vs-non-American head reaches ROC-AUC 0.9607 on the speaker-disjoint AESRC
test split.

The Hub bundle ships machine-readable results as `model_evaluation.json`; the source
repository tracks the same receipt at `artifacts/metrics/model_evaluation.json`.

## Output interpretation

`american_similarity` equals 100 times a calibrated binary probability. It is useful
for ranking recordings under the model's training distribution, but it is not a direct
percentage similarity and has not been calibrated against human accent-strength
ratings.

The `unknown` label is emitted when no supported class reaches probability 0.4. This
does not guarantee reliable out-of-distribution detection.

Pronunciation-quality intervals are marginal conformal intervals under exchangeability
assumptions. They are not guarantees for an individual speaker or a shifted domain.

## Known limitations

- Four labels cannot represent the diversity or continuity of English pronunciation.
- The training corpora may encode speaker, microphone, location, sentence, and source
  shortcuts.
- Source-disjoint evaluation is limited to two accent corpora.
- Short, noisy, code-switched, spontaneous, or unsupported speech can be unreliable.
- Quality targets reflect one corpus's annotation conventions.
- The model does not evaluate intelligibility, communication success, or personal
  ability.

## Operational guidance

- Keep the responsible-use warning with user-facing results.
- Show probabilities and `unknown`, not only the top label.
- Allow users to ignore or contest a result.
- Validate on the target population and recording conditions before deployment.
- Monitor performance separately by source and relevant acoustic conditions.
- Do not infer protected or sensitive traits from accent outputs.

## Reproducibility

The source repository pins dependencies in `uv.lock`, the model bundle is tagged
`v0.2.1`, the encoder uses a full commit revision, and deterministic seeds and split
rules live in the training code. Runtime consumers pin the full Hugging Face model
commit rather than the mutable `main` branch.

Run static checks and the non-network test suite with:

```bash
make check
```

The exported model contract is covered by an opt-in real-encoder smoke test.