iot_keyword_spotting_wav2vec2

Overview

This model is a fine-tuned version of Wav2Vec2 designed for Keyword Spotting (KWS) tasks in low-latency environments. It is optimized to recognize specific directional and operational commands commonly used in smart home devices and industrial IoT sensors. By processing raw audio waveforms, it bypasses traditional spectrogram conversion, significantly reducing inference time on the edge.

Model Architecture

The model utilizes the Wav2Vec2 architecture, which consists of a multi-layer convolutional feature encoder followed by a Transformer context network.

• Feature Encoder: 7 layers of temporal convolutions that transform raw audio into latent speech representations.
• Context Network: 12 Transformer blocks that capture global dependencies across the audio sequence.
• Classification Head: A mean-pooling layer followed by a linear transformation and Softmax to output probabilities for 12 classes (e.g., "ON", "OFF", "STOP").

The model optimizes for the cross-entropy loss between predicted labels and ground truth: $$\mathcal{L}=-\sum _{c=1}^M{y}_{o,c}\log (p_{o,c})\backslash mathcal\{L\}\; =\; -\backslash sum\_\{c=1\}^\{M\}\; y\_\{o,c\}\; \backslash log(p\_\{o,c\})$$

Intended Use

• Voice User Interfaces (VUI): Enabling hands-free control for localized smart devices.
• Industrial Automation: Detecting verbal safety triggers (e.g., "STOP") in noisy factory environments.
• Edge Computing: Deployment on low-power hardware with model quantization (INT8/FP16).

Limitations

• Ambient Noise: While robust, extreme background noise (e.g., wind or heavy machinery) may degrade precision.
• Speaker Variability: Performance may vary across different accents or speech impediments not represented in the Google Speech Commands dataset.
• Sampling Rate: The model strictly requires audio input at 16kHz; other rates must be resampled prior to inference.