q1805/hubert-large-german-v1

---

🇩🇪 HuBERT Large German IPA Phoneme Model (v1 - Baseline)

This repository hosts HuBERT Large German v1, an acoustic phoneme recognition model fine-tuned to predict the International Phonetic Alphabet (IPA) for German pronunciation evaluation.

📌 Model Overview

• Base Architecture: facebook/hubert-large-ls960-ft (315M parameters).
• Task: Connectionist Temporal Classification (CTC) Phoneme Recognition for German speech.
• Target Alphabet: 70 German IPA phoneme tokens (including vowels, diphthongs, consonants, stress markers ˈ, ˌ, and German umlauts ä, ö, ü, ß).
• Objective: Serve as the baseline acoustic model for the PronunCheck pronunciation scoring system (Goodness of Pronunciation - GOP).

---

📚 Training Dataset

• Source: Multilingual LibriSpeech (MLS) - German.
• Size: approx. 20 GB of clean, studio-quality audiobook recordings (~160,000 utterances).
• Audio Characteristics: Clean studio narration, single speakers per chapter, 16 kHz sampling rate, minimal background noise.
• G2P Backend: Text labels were converted to German IPA using espeak-ng via phonemizer with German dialect flags (de).

---

⚙️ Training Setup & Hyperparameters

• Hardware: 1x NVIDIA L4 GPU (24GB VRAM) on Google Cloud Platform (GCP).
• Precision: Mixed Precision (fp16).
• Batch Size: Per-device train batch size = 4, Gradient Accumulation = 4 (Effective batch size = 16).
• Optimizer: AdamW (lr = 3e-5, linear warmup for 1,000 steps, linear decay).
• Feature Extractor: Fixed 16 kHz CNN feature extractor (freeze_feature_encoder=True).
• Training Time: ~36 hours for 3 epochs.

---

📊 Evaluation & Limitations

• Clean Speech (MLS Test Split): PER ~28.4%.
• Spontaneous Speech (Mozilla SPS Corpus 4.0): PER ~35.60% (Accuracy: 64.40%).
• Known Limitations: Because v1 was trained purely on clean audiobook speech, its accuracy degrades noticeably when exposed to fast real-world speech, room reverberation, or low-grade microphones. (See v2 model for real-world robustness).

---

Quickstart Inference Code

import torch
import librosa
from transformers import Wav2Vec2Processor, HubertForCTC

# Load model and processor directly from Hugging Face
MODEL_ID = "q1805/hubert-large-german-v1"
processor = Wav2Vec2Processor.from_pretrained(MODEL_ID)
model = HubertForCTC.from_pretrained(MODEL_ID).eval()

# Load and resample audio
audio_path = "sample_german.wav"
speech, sr = librosa.load(audio_path, sr=16000)

# Inference
inputs = processor(speech, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(inputs.input_values).logits
    predicted_ids = torch.argmax(logits, dim=-1)
    ipa_output = processor.batch_decode(predicted_ids)[0]

print("Predicted IPA:", ipa_output)