samuelr3/hear-service-models

Hear Service models

The on-device models behind Hear Service, a voice-first app for blind and partially sighted people who listen to UK talking newspapers. Everything here runs on the phone. No speech, text or audio is sent to a server to be understood or spoken.

The app ships with one voice built in (Alice, the default). On first launch it downloads everything in this repository, checks every file against manifest.json, and loads the models. The listener can't use the app until that has finished.

Part	Model	Size	Job
Voices	5 Piper VITS voices	63.5 MB each	Speak every line in the listener's chosen voice
Speech recognition	Parakeet TDT-CTC 110M, fine-tuned	178 MB	Hear names the phone's own recogniser gets wrong
Understanding	all-MiniLM-L6-v2 + 2 classifier heads	23 MB	Work out what the listener wants and how to reply

Total download: 519 MB in 15 files.

Contents

• Repository layout
• How the app uses these files
• Voices
• Speech recognition
• Understanding and chat
• Why there is no large language model
• Licences and credits
• Versions

Repository layout

manifest.json                       version, size and SHA-256 of every file
voices/
  piper-voice.json                  phoneme table and sample rate shared by all voices
  en_GB-jenny.onnx                  Jenny    Irish woman
  en_GB-deborah.onnx                Deborah  English woman
  en_GB-richard.onnx                Richard  southern English man
  en_GB-david.onnx                  David    Welsh man
  en_GB-alan.onnx                   Alan     Scottish man
speech/parakeet/
  encoder.int8.onnx                 172.4 MB
  decoder.int8.onnx                   4.0 MB
  joiner.int8.onnx                    1.4 MB
  tokens.txt                        1,025 tokens
chat/
  minilm-q8.onnx                    sentence encoder, 8-bit
  minilm-vocab.txt                  WordPiece vocabulary
  intent-head.json                  14 actions
  chat-head.json                    27 kinds of conversation
  chat-replies.json                 approved reply lines

Alice (en_GB-alice.onnx), the pronunciation list and the spelling guesser for unknown words are built into the app, so it can always speak, even before the download.

How the app uses these files

1. Fetch manifest.json from https://huggingface.co/samuelr3/hear-service-models/resolve/main/manifest.json.
2. Download each file listed, from resolve/main/<path>, into the app's own storage.
3. Check each file's size and SHA-256 against the manifest. A file that fails is downloaded again.
4. Load the models. Only then does the app accept requests.
5. On later launches, fetch the manifest again. If version has gone up, download only the files whose SHA-256 changed, check them, and switch over at a quiet moment. The old files stay in use until the new ones pass.

{
  "version": 1,
  "files": [
    { "path": "voices/en_GB-jenny.onnx", "bytes": 63516050, "sha256": "…" }
  ]
}

Voices

Six British and Irish voices, VITS models trained with Piper, medium quality, 22,050 Hz, mono. Exported to ONNX at full 32-bit precision. Parakeet and MiniLM below are 8-bit; see Precision.

Training data

Voice	Accent	Recordings	Audio	Started from	Fine-tuning epochs
Alice (built in)	Northern English woman	OpenSLR 83, speaker 06136	150 clips, 13.9 min	Cori	500
Jenny	Irish woman	Jenny (Dioco) dataset	8,568 clips, 11.8 h	Cori	60
Deborah	English woman	Recorded by Deborah, cleaned with DeepFilterNet	84 clips, 8.1 min	Cori	not recorded
Richard	Southern English man	OpenSLR 83, speaker 07505	150 clips, 13.7 min	Male base	401
David	Welsh man	OpenSLR 83, speaker 09334	150 clips, 14.5 min	Male base	401
Alan	Scottish man	OpenSLR 83, speaker 07049	150 clips, 12.5 min	Male base	401

Cori is the public-domain Piper voice en_GB-cori-medium, read from LibriVox recordings. No voice here starts from Lessac, whose recordings can't be used commercially.

Male base was trained from Cori on every man in OpenSLR 83: 10,626 clips, 14.5 hours, 9 epochs. The three men's voices start from it, because starting a man's voice from a woman's sounded rough.

OpenSLR 83 speakers were picked from five candidates per accent for the lowest background noise. The Cori checkpoint they start from had already trained for 640 epochs. Training ran on one NVIDIA A40.

Running a voice

Input	Type	Shape	Value
input	int64	[1, n]	phoneme ids from piper-voice.json, starting 1, 0, a 0 after each phoneme, ending 2
input_lengths	int64	[1]	n
scales	float32	[3]	noise 0.667, length 1.0, noise width 0.8

The output is 22,050 Hz audio as float32. The app changes speaking speed through the length scale: 1.4 slower, 1.0 normal, 0.74 faster, 0.62 fastest.

Phonemes come from the app's own British English pronunciation list (IPA) and a small spelling-to-sound model for words not on the list. eSpeak isn't used anywhere.

A voice makes one second of speech in about 0.07 seconds on an Android emulator on an M-series Mac and about 0.27 seconds on an iPhone XR. Sentences are made one at a time, and the next is prepared while the current one plays.

The voices were tried at 8-bit and 16-bit. 8-bit was 3 times slower on phone CPUs, because ONNX Runtime has no fast kernel for quantised convolutions, and 94% of a VITS voice's weights are convolutions. Full precision is the fastest.

Speech recognition

NVIDIA Parakeet TDT-CTC 110M, fine-tuned on British speech and on the names of UK talking newspapers, towns and organisations. It's exported for sherpa-onnx as an 8-bit transducer (modelType: nemo_transducer), 16 kHz input, 2 threads.

The app records each request once. The phone's own recogniser and Parakeet both listen to the same audio. The app keeps whichever version matches a known name or has more real words, so the listener doesn't have to repeat themselves.

Training data

465.2 hours, 110,712 clips:

Source	Clips	Hours
Talking newspaper archive recordings, with transcripts	61,324	415.9
OpenSLR 83 UK and Ireland English dialects	17,877	31.3
Synthetic sentences containing station, town and organisation names	31,511	18.0

The archive recordings belong to the talking newspapers, and only the trained model is published here. Training ran for 3 epochs (learning rates 1e-4, 5e-5, 2e-5, AdamW, cosine schedule, bf16) on one A40, about 100 minutes per epoch.

Results

Test	Score
Word error rate on held-out archive speech	7.3%
Talking newspaper and place names recognised	87.3% of 394
"Herne Bay" said in hard recordings	22 of 22

A second round with 40,339 extra clips of place names said alone (18.7 h) reached 42.9% on lone place names, but word error rose to 8.6%, so the first round is the one published.

Understanding and chat

all-MiniLM-L6-v2, mean-pooled and normalised, exported to ONNX and quantised to 8-bit. Inputs are ids, mask and types (int64, [1, n]), and the output v is a 384-number sentence embedding. Two logistic-regression heads read that embedding:

Head	Labels	Training phrases	Held-out phrases	Result
intent-head.json	14 actions	4,031	236	219 of 236 held out; 37 of 41 real requests on an iPhone XR, about 5 ms each
chat-head.json	27 kinds of conversation	4,728	536

Actions: change speed, follow a source, help, next, not a request, settings, pause, play a place, play a source, previous, report content, resume, search a topic, what's popular.

Conversation kinds: abilities, about Hear, afternoon, can you hear me, can't do, complaint, compliment, cost, day, evening, eyesight, favourite, feeling, goodbye, grief, hello, how are you, joke, morning, music, night, none, personal, thanks, urgent, voice praise, weather.

A head's answer is used only at 60% confidence or more. The app answers from approved lines in chat-replies.json and from the real catalogue (titles, descriptions, topics), never from generated text. Searches always ask the listener to confirm.

Why there is no large language model

Hear Service doesn't use a large language model. Two were trained and tested on the phones:

Model	Size on phone	Result
SmolLM2-135M, fine-tuned on Hear conversations	4-bit: about 75 MB	4-bit quantisation wiped out the fine-tuning (0 of 80 test answers right), and 8-bit went over the 75 MB budget
Qwen2.5-0.5B-Instruct, fine-tuned	4-bit: 318 MB	Worked, but too slow on an iPhone XR and didn't suit how Hear Service works

The classifier with approved lines answers in about 5 ms and only says lines that have been checked.

Precision

Model	Precision	Why
Voices	32-bit	94% of a voice's weights are convolutions, and 8-bit convolutions have no fast kernel on phones, so 8-bit was 3 times slower
Parakeet	8-bit weights	Its weights are mostly matrix multiplies, which have fast 8-bit kernels on ARM; 32-bit would be about 440 MB instead of 178 MB
MiniLM	8-bit weights	Same reason; 32-bit would be about 90 MB instead of 23 MB

Licences and credits

Model	Licence	Credit
Jenny	Jenny (Dioco) licence, commercial use allowed	The voice must be named Jenny and credited as "Jenny (Dioco)"
Alice, Richard, David, Alan	CC BY-SA 4.0	Crowdsourced high-quality UK and Ireland English Dialect speech data set, OpenSLR 83, Google
Deborah	Used with permission for Hear Service	Recordings by Deborah
All voices' starting point	Public domain	Cori, en_GB-cori-medium, LibriVox
Parakeet	CC BY 4.0	nvidia/parakeet-tdt_ctc-110m, NVIDIA
MiniLM encoder	Apache 2.0	sentence-transformers/all-MiniLM-L6-v2

Voices made from OpenSLR 83 are shared under CC BY-SA 4.0, so changed versions must be shared the same way.

Versions

Version	Date	Changes
1	2026-11-07	First release: Jenny, Deborah, Richard, David and Alan at full precision; Parakeet round 1; MiniLM with intent and chat heads