Moonshine Tiny for spoken chess moves

A small speech recogniser (27M parameters, about 38 MB) that turns a spoken chess move into words, made for voice-controlled blindfold chess in the browser. It is Moonshine Tiny (English) fine-tuned on synthetic recordings of chess moves, exported to ONNX for transformers.js, and runs fully offline on the CPU (WebAssembly).

Stock Moonshine Tiny writes almost none of these phrases correctly; this model gets about 97% exactly right on held-out test voices, including moves whose first word was cut off ("โ€ฆook a one" is still "rook a one").

What it writes

Lowercase words, numbers spelled out, the way a player says the move:

said output
Nf3 knight f three
exd5 e takes d five
Bh5# bishop h five checkmate
Nbd2 knight b d two
e8=Q e eight queen
O-O castle king side

It also knows a few command words used by voice-control apps (clock, opponent time, no).

The model only writes what it hears. It does not know the position, so pick the move with your own legal-move check. In the app it was made for, its words are compared with the legal moves of the current position, and a second, independent recogniser has to agree before a doubtful move is played.

Use with transformers.js

Replace yourname/moonshine-tiny-chess with this repository's id.

import { pipeline } from '@huggingface/transformers';

const transcriber = await pipeline('automatic-speech-recognition', 'yourname/moonshine-tiny-chess', {
  device: 'wasm',
  dtype: { encoder_model: 'fp32', decoder_model_merged: 'int8' },
});

// audio: Float32Array, 16 kHz mono, a few seconds at most
const { text } = await transcriber(audio); // "knight f three"

The fp32 encoder setting loads onnx/encoder_model.onnx, which in this repository is already the smaller 8-bit encoder (see below). Tested with transformers.js 4.3.2.

Files

file size
onnx/encoder_model.onnx 13.2 MB encoder, 8-bit weights except the three convolutions at the front
onnx/decoder_model_merged_int8.onnx 21.0 MB decoder (with and without key-value cache), 8-bit weights
tokenizer.json, tokenizer_config.json, special_tokens_map.json 3.9 MB Moonshine's tokenizer, unchanged
config.json, generation_config.json, preprocessor_config.json small model settings

The encoder's weights are quantised to 8 bits (dynamic, per channel) everywhere except the convolution layers that turn raw audio into features: quantising those too lost 1 to 3 clips per test set, while this version scores the same as the full-precision encoder (30.9 MB) on every set below.

Results

Test clips: three Windows voices (George, Hazel, Zira) that were never used in training, 37 phrases each, plus versions of them with the start of the audio cut off or with noise added. "Exact" means the output text is exactly the phrase said.

test set exact
clean 108 / 111
0.5 s of quiet before and 0.3 s after the speech 108 / 111
first 80 ms cut off 48 / 48
first 160 ms cut off 47 / 48
first 240 ms cut off 26 / 48
noise, 15 dB SNR 108 / 111
noise, 8 dB SNR 102 / 111

About 40โ€“75 ms per clip on a desktop CPU.

In the app, combined with a legal-move check and a second recogniser, it played no wrong move on 773 move clips (synthetic and microphone recordings) and asked a short question on the doubtful ones.

Limitations

  • English only, and only chess moves and a few commands. Other speech often comes out as something move-shaped ("I am thinking" can become a three check), so never play its output without checking it.
  • Trained on synthetic speech only. It has been tried on few real voices so far; accents and microphones far from the training mix may do worse.
  • When more than about 0.2 s of the start of a word is missing, it often loses the piece word.
  • Made for short clips (a few seconds), like a push-to-talk hold.

Training

  • Base: UsefulSensors/moonshine-tiny (English), MIT.
  • Data: 18,887 synthetic clips (6.6 h) of chess moves and commands, from 1,898 voices, rendered locally with the open TTS voices listed below. Split into training and validation by voice.
  • Augmentation, new every epoch: speed and pitch 0.85โ€“1.2, reverb, band-limiting, noise, level changes and cutting off the start of the clip.
  • 20 epochs, batch 64, learning rate 1.5e-4, bf16. Best epoch 18 (89.9% exact on held-out synthetic voices).
  • Exported to ONNX with Optimum; decoder and encoder quantised with ONNX Runtime's dynamic quantisation.

No Windows, cloud or other proprietary voice was used for training, and no recordings of real people beyond the public speech data sets the TTS voices were built from.

Training data and attribution

The training audio was generated with these voices. Their data sets are credited here as their licences ask.

TTS voices data set licence
Piper en_US-libritts_r-medium LibriTTS-R (Koizumi et al., 2023), https://www.openslr.org/141/ CC BY 4.0
Piper en_US-libritts-high LibriTTS (Zen et al., 2019), https://www.openslr.org/60/ CC BY 4.0
Piper en_GB-vctk-medium CSTR VCTK Corpus (Yamagishi, Veaux, MacDonald, University of Edinburgh), https://datashare.ed.ac.uk/handle/10283/3443 CC BY 4.0
Piper en_GB-alba-medium Alba voice data (University of Edinburgh), https://datashare.ed.ac.uk/handle/10283/3270 CC BY 4.0
Piper en_US-ljspeech-high LJ Speech (Keith Ito), https://keithito.com/LJ-Speech-Dataset/ public domain
Kokoro-82M v1.0, 28 English voices https://huggingface.co/hexgrad/Kokoro-82M Apache-2.0

The Piper and Kokoro model files came from the k2-fsa/sherpa-onnx release assets, which repackage the upstream files unchanged.

Licence

MIT. See LICENSE, which also carries the Moonshine Tiny copyright notice (Copyright (c) 2025 Useful Sensors, Inc., dba Moonshine AI).

Downloads last month
26
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for pwndl11/moonshine-tiny-chess

Quantized
(21)
this model