fix: allow endpoint callers to select transcription language

README.md

@@ -1,137 +1,168 @@
----
-license: apache-2.0
-datasets:
-- AImpower/MandarinStutteredSpeech
-language:
-- zh
-metrics:
-- cer
-base_model:
-- openai/whisper-large-v2
-pipeline_tag: automatic-speech-recognition
----
-# Model Card: AImpower/StutteredSpeechASR
-
-This model is a version of OpenAI's `whisper-large-v2` fine-tuned on the **AImpower/MandarinStutteredSpeech** dataset, a grassroots-collected corpus of Mandarin Chinese speech from people who stutter (PWS).
-
-## Model Details
-
-* **Base Model:** `openai/whisper-large-v2`
-* **Language:** Mandarin Chinese
-* **Fine-tuning Dataset:** [AImpower/MandarinStutteredSpeech](https://huggingface.co/datasets/AImpower/MandarinStutteredSpeech)
-* **Fine-tuning Method:** The model was fine-tuned using the LoRA adapter (AdaLora) methodology to preserve speech disfluencies in its transcriptions.
-* **Paper:** [Our Collective Voices: The Social and Technical Values of a Grassroots Chinese Stuttered Speech Dataset](https://doi.org/10.1145/3715275.3732179)
-
-## Model Description
-
-This model is specifically adapted to provide more accurate and authentic transcriptions for Mandarin-speaking PWS.  
-Standard Automatic Speech Recognition (ASR) models often exhibit "fluency bias," where they "smoothen" out or delete stuttered speech patterns like repetitions and interjections.  
-This model was fine-tuned on literal transcriptions that intentionally preserve these disfluencies.
-
-The primary goal is to create a more inclusive ASR system that recognizes and respects the natural speech patterns of PWS, reducing deletion errors and improving overall accuracy.
-
-## Intended Uses & Limitations
-
-### Intended Use
-
-This model is intended for transcribing conversational Mandarin Chinese speech from individuals who stutter. It's particularly useful for:
-* Improving accessibility in speech-to-text applications.
-* Linguistic research on stuttered speech.
-* Developing more inclusive voice-enabled technologies.
-
-### Limitations
-
-* **Language Specificity:** The model is fine-tuned exclusively on Mandarin Chinese and is not intended for other languages.
-* **Data Specificity:** Performance is optimized for speech patterns present in the AImpower/MandarinStutteredSpeech dataset. It may not perform as well on other types of atypical speech or in environments with significant background noise.
-* **Variability:** Stuttering is highly variable. While the model shows significant improvements across severity levels, accuracy may still vary between individuals and contexts.
-
----
-
-## How to Use
-
-You can use the model with the `transformers` library. Ensure you have `torch`, `transformers`, and `librosa` installed.
-
-```python
-from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
-import torch
-import librosa
-
-# Load the fine-tuned model and processor
-model_path = "AImpower/StutteredSpeechASR"
-model = AutoModelForSpeechSeq2Seq.from_pretrained(model_path)
-processor = AutoProcessor.from_pretrained(model_path)
-
-device = "cuda" if torch.cuda.is_available() else "cpu"
-model.to(device)
-
-# Load an example audio file (replace with your audio file)
-audio_input_name = "example_stuttered_speech.wav"
-waveform, sampling_rate = librosa.load(audio_input_name, sr=16000)
-
-# Process the audio and generate transcription
-input_features = processor(waveform, sampling_rate=sampling_rate, return_tensors="pt").input_features
-input_features = input_features.to(device)
-
-predicted_ids = model.generate(input_features)
-transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
-
-print(f"Transcription: {transcription}")
+---
+license: apache-2.0
+datasets:
+- AImpower/MandarinStutteredSpeech
+language:
+- zh
+metrics:
+- cer
+base_model:
+- openai/whisper-large-v2
+pipeline_tag: automatic-speech-recognition
+---
+# Model Card: AImpower/StutteredSpeechASR
+
+This model is a version of OpenAI's `whisper-large-v2` fine-tuned on the **AImpower/MandarinStutteredSpeech** dataset, a grassroots-collected corpus of Mandarin Chinese speech from people who stutter (PWS).
+
+## Model Details
+
+* **Base Model:** `openai/whisper-large-v2`
+* **Language:** Mandarin Chinese
+* **Fine-tuning Dataset:** [AImpower/MandarinStutteredSpeech](https://huggingface.co/datasets/AImpower/MandarinStutteredSpeech)
+* **Fine-tuning Method:** The model was fine-tuned using the LoRA adapter (AdaLora) methodology to preserve speech disfluencies in its transcriptions.
+* **Paper:** [Our Collective Voices: The Social and Technical Values of a Grassroots Chinese Stuttered Speech Dataset](https://doi.org/10.1145/3715275.3732179)
+
+## Model Description
+
+This model is specifically adapted to provide more accurate and authentic transcriptions for Mandarin-speaking PWS.  
+Standard Automatic Speech Recognition (ASR) models often exhibit "fluency bias," where they "smoothen" out or delete stuttered speech patterns like repetitions and interjections.  
+This model was fine-tuned on literal transcriptions that intentionally preserve these disfluencies.
+
+The primary goal is to create a more inclusive ASR system that recognizes and respects the natural speech patterns of PWS, reducing deletion errors and improving overall accuracy.
+
+## Intended Uses & Limitations
+
+### Intended Use
+
+This model is intended for transcribing conversational Mandarin Chinese speech from individuals who stutter. It's particularly useful for:
+* Improving accessibility in speech-to-text applications.
+* Linguistic research on stuttered speech.
+* Developing more inclusive voice-enabled technologies.
+
+### Limitations
+
+* **Language Specificity:** The model is fine-tuned exclusively on Mandarin Chinese and is not intended for other languages.
+* **Data Specificity:** Performance is optimized for speech patterns present in the AImpower/MandarinStutteredSpeech dataset. It may not perform as well on other types of atypical speech or in environments with significant background noise.
+* **Variability:** Stuttering is highly variable. While the model shows significant improvements across severity levels, accuracy may still vary between individuals and contexts.
+
+---
+
+## How to Use
+
+You can use the model with the `transformers` library. Ensure you have `torch`, `transformers`, and `librosa` installed.
+
+```python
+from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
+import torch
+import librosa
+
+# Load the fine-tuned model and processor
+model_path = "AImpower/StutteredSpeechASR"
+model = AutoModelForSpeechSeq2Seq.from_pretrained(model_path)
+processor = AutoProcessor.from_pretrained(model_path)
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model.to(device)
+
+# Load an example audio file (replace with your audio file)
+audio_input_name = "example_stuttered_speech.wav"
+waveform, sampling_rate = librosa.load(audio_input_name, sr=16000)
+
+# Process the audio and generate transcription
+input_features = processor(waveform, sampling_rate=sampling_rate, return_tensors="pt").input_features
+input_features = input_features.to(device)
+
+predicted_ids = model.generate(input_features)
+transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
+
+print(f"Transcription: {transcription}")
+```
+
+-----
+
+### Hosted inference endpoint: selecting a language
+
+For an endpoint deployed with `handler.py`, send a JSON request with the
+language in `parameters.language`:
+
+```json
+{
+  "inputs": "<base64-encoded audio file>",
+  "parameters": {"language": "en"}
+}
 ```
 
------
+Whisper accepts language codes, names, or language tokens, for example `"en"`,
+`"english"`, or `"<|en|>"` for English and `"zh"` for Mandarin Chinese.
+The endpoint passes the selected language to Whisper with `task="transcribe"`
+so the output stays in the source language rather than translating it.
+The same `parameters` object works with `{"inputs": {"audio": "..."}}`
+and `{"audio": "..."}` requests. Use JSON to select a language; raw binary
+requests do not carry parameters.
+
+Omitting `language` or setting it to `null` preserves the model's default
+generation behavior. The checked-in generation configuration leaves the
+language unspecified, allowing Whisper to detect it automatically; the handler
+does not force Mandarin. A language override applies only to its own request.
+Unsupported language strings are rejected by Whisper using the handler's
+existing error response format.
+
+Language selection enables cross-language benchmarking, but this model was
+fine-tuned on Mandarin and improvements in other languages have not been
+established. Redeploy the endpoint after merging handler changes to use this option.
 
 ## Training Data
-
-The model was fine-tuned on the **[AImpower/MandarinStutteredSpeech](https://huggingface.co/datasets/AImpower/MandarinStutteredSpeech)** dataset.  
-This dataset was created through a community-led, grassroots effort with StammerTalk, an online community for Chinese-speaking PWS.
-
-  * **Size:** The dataset contains nearly 50 hours of speech from 72 adults who stutter.
-  * **Content:** It includes both unscripted, spontaneous conversations between two PWS and the dictation of 200 voice commands.
-  * **Transcription:** The training was performed on verbatim (literal) transcriptions that include disfluencies such as word repetitions and interjections, which was a deliberate choice by the community to ensure their speech was represented authentically.
-
-## Training Procedure
-
-  * **Data Split:** A three-fold cross-validation approach was used, with data split by participant to ensure robustness. Each fold had a roughly 65:10:25 split for train/dev/test sets, with a balanced representation of mild, moderate, and severe stuttering levels. This model card represents the best-performing fold.
-  * **Hyperparameters:**
-      * **Epochs:** 3
-      * **Learning Rate:** 0.001
-      * **Optimizer:** AdamW
-      * **Batch Size:** 16
-      * **Fine-tuning Method:** AdaLora
-  * **GPU:** Four NVIDIA A100 80G GPUs
------
-
-## Evaluation Results
-
-The fine-tuned model demonstrates a substantial improvement in transcription accuracy across all stuttering severity levels compared to the baseline `whisper-large-v2` model.  
-The key metric used is Character Error Rate (CER), evaluated on literal transcriptions to measure the model's ability to preserve disfluencies.
-
-| Stuttering Severity | Baseline Whisper CER | Fine-tuned Model CER |
-| :------------------ | :------------------- | :------------------- |
-| Mild                | 16.34%               | **5.80%**            |
-| Moderate            | 21.72%               | **9.03%**            |
-| Severe              | 49.24%               | **20.46%**           |
-
-*(Results from Figure 3 of the paper)*
-
-Notably, the model achieved a significant reduction in **deletion errors (DEL)**, especially for severe speech (from 26.56% to 2.29%), indicating that it is much more effective at preserving repeated words and phrases instead of omitting them.
-
-## Citation
-
-If you use this model, please cite the original paper:
-
-```bibtex
-@inproceedings{li2025collective,
-  author = {Li, Jingjin and Li, Qisheng and Gong, Rong and Wang, Lezhi and Wu, Shaomei},
-  title = {Our Collective Voices: The Social and Technical Values of a Grassroots Chinese Stuttered Speech Dataset},
-  year = {2025},
-  isbn = {9798400714825},
-  publisher = {Association for Computing Machinery},
-  address = {New York, NY, USA},
-  url = {https://doi.org/10.1145/3715275.3732179},
-  booktitle = {The 2025 ACM Conference on Fairness, Accountability, and Transparency},
-  pages = {2768–2783},
-  location = {Athens, Greece},
-  series = {FAccT '25}
-}
-```
+
+The model was fine-tuned on the **[AImpower/MandarinStutteredSpeech](https://huggingface.co/datasets/AImpower/MandarinStutteredSpeech)** dataset.  
+This dataset was created through a community-led, grassroots effort with StammerTalk, an online community for Chinese-speaking PWS.
+
+  * **Size:** The dataset contains nearly 50 hours of speech from 72 adults who stutter.
+  * **Content:** It includes both unscripted, spontaneous conversations between two PWS and the dictation of 200 voice commands.
+  * **Transcription:** The training was performed on verbatim (literal) transcriptions that include disfluencies such as word repetitions and interjections, which was a deliberate choice by the community to ensure their speech was represented authentically.
+
+## Training Procedure
+
+  * **Data Split:** A three-fold cross-validation approach was used, with data split by participant to ensure robustness. Each fold had a roughly 65:10:25 split for train/dev/test sets, with a balanced representation of mild, moderate, and severe stuttering levels. This model card represents the best-performing fold.
+  * **Hyperparameters:**
+      * **Epochs:** 3
+      * **Learning Rate:** 0.001
+      * **Optimizer:** AdamW
+      * **Batch Size:** 16
+      * **Fine-tuning Method:** AdaLora
+  * **GPU:** Four NVIDIA A100 80G GPUs
+-----
+
+## Evaluation Results
+
+The fine-tuned model demonstrates a substantial improvement in transcription accuracy across all stuttering severity levels compared to the baseline `whisper-large-v2` model.  
+The key metric used is Character Error Rate (CER), evaluated on literal transcriptions to measure the model's ability to preserve disfluencies.
+
+| Stuttering Severity | Baseline Whisper CER | Fine-tuned Model CER |
+| :------------------ | :------------------- | :------------------- |
+| Mild                | 16.34%               | **5.80%**            |
+| Moderate            | 21.72%               | **9.03%**            |
+| Severe              | 49.24%               | **20.46%**           |
+
+*(Results from Figure 3 of the paper)*
+
+Notably, the model achieved a significant reduction in **deletion errors (DEL)**, especially for severe speech (from 26.56% to 2.29%), indicating that it is much more effective at preserving repeated words and phrases instead of omitting them.
+
+## Citation
+
+If you use this model, please cite the original paper:
+
+```bibtex
+@inproceedings{li2025collective,
+  author = {Li, Jingjin and Li, Qisheng and Gong, Rong and Wang, Lezhi and Wu, Shaomei},
+  title = {Our Collective Voices: The Social and Technical Values of a Grassroots Chinese Stuttered Speech Dataset},
+  year = {2025},
+  isbn = {9798400714825},
+  publisher = {Association for Computing Machinery},
+  address = {New York, NY, USA},
+  url = {https://doi.org/10.1145/3715275.3732179},
+  booktitle = {The 2025 ACM Conference on Fairness, Accountability, and Transparency},
+  pages = {2768–2783},
+  location = {Athens, Greece},
+  series = {FAccT '25}
+}
+```

handler.py

@@ -1,197 +1,218 @@
-"""
-Custom Inference Handler for StutteredSpeechASR Model
-Handles audio input and returns transcriptions for stuttered speech.
-"""
-
-import torch
-import librosa
-import numpy as np
-import base64
-import io
+"""
+Custom Inference Handler for StutteredSpeechASR Model
+Handles audio input and returns transcriptions for stuttered speech.
+"""
+
+import torch
+import librosa
+import numpy as np
+import base64
+import io
 import logging
-from typing import Dict, Any
-from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
-
-# Configure logging
-logging.basicConfig(level=logging.INFO)
-logger = logging.getLogger(__name__)
-
-
-class EndpointHandler:
-    """
-    Custom handler for StutteredSpeechASR inference endpoint.
-    
-    This handler processes audio inputs and returns transcriptions
-    using the fine-tuned Whisper model for stuttered Mandarin speech.
-    """
-    
-    def __init__(self, path: str = ""):
-        """
-        Initialize the handler by loading the model and processor.
-        
-        Args:
-            path: Path to the model directory (provided by Inference Endpoints)
-        """
-        logger.info("Initializing StutteredSpeechASR handler...")
-        
-        # Determine device and dtype
-        self.device = "cuda" if torch.cuda.is_available() else "cpu"
-        self.torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
-        
-        logger.info(f"Using device: {self.device}")
-        logger.info(f"Using dtype: {self.torch_dtype}")
-        
-        # Load model and processor
-        try:
-            self.model = AutoModelForSpeechSeq2Seq.from_pretrained(
-                path,
-                torch_dtype=self.torch_dtype
-            )
-            self.processor = AutoProcessor.from_pretrained(path)
-            self.model.to(self.device)
-            self.model.eval()  # Set to evaluation mode
-            
-            logger.info("Model and processor loaded successfully!")
-        except Exception as e:
-            logger.error(f"Error loading model: {e}")
-            raise
-    
-    def _load_audio_from_bytes(self, audio_bytes: bytes) -> np.ndarray:
-        """
-        Load audio from bytes and resample to 16kHz.
-        
-        Args:
-            audio_bytes: Raw audio bytes
-            
-        Returns:
-            Audio waveform as numpy array
-        """
-        try:
-            # Load audio from bytes using librosa
-            audio_buffer = io.BytesIO(audio_bytes)
-            waveform, _ = librosa.load(audio_buffer, sr=16000, mono=True)
-            return waveform
-        except Exception as e:
-            logger.error(f"Error loading audio from bytes: {e}")
-            raise ValueError(f"Failed to load audio: {e}")
-    
-    def _load_audio_from_base64(self, base64_string: str) -> np.ndarray:
-        """
-        Load audio from base64-encoded string.
-        
-        Args:
-            base64_string: Base64-encoded audio data
-            
-        Returns:
-            Audio waveform as numpy array
-        """
-        try:
-            # Decode base64 string
-            audio_bytes = base64.b64decode(base64_string)
-            return self._load_audio_from_bytes(audio_bytes)
-        except Exception as e:
-            logger.error(f"Error decoding base64 audio: {e}")
-            raise ValueError(f"Failed to decode base64 audio: {e}")
-    
-    def __call__(self, data: Dict[str, Any]) -> Dict[str, Any]:
-        """
-        Process incoming requests and return transcriptions.
-        
-        Expected input formats:
-        1. {"inputs": "base64_encoded_audio_string"}
-        2. {"inputs": {"audio": "base64_encoded_audio_string"}}
+from copy import deepcopy
+from typing import Dict, Any
+from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
+
+# Configure logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+class EndpointHandler:
+    """
+    Custom handler for StutteredSpeechASR inference endpoint.
+    
+    This handler processes audio inputs and returns transcriptions
+    using the fine-tuned Whisper model for stuttered Mandarin speech.
+    """
+    
+    def __init__(self, path: str = ""):
+        """
+        Initialize the handler by loading the model and processor.
+        
+        Args:
+            path: Path to the model directory (provided by Inference Endpoints)
+        """
+        logger.info("Initializing StutteredSpeechASR handler...")
+        
+        # Determine device and dtype
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        self.torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
+        
+        logger.info(f"Using device: {self.device}")
+        logger.info(f"Using dtype: {self.torch_dtype}")
+        
+        # Load model and processor
+        try:
+            self.model = AutoModelForSpeechSeq2Seq.from_pretrained(
+                path,
+                torch_dtype=self.torch_dtype
+            )
+            self.processor = AutoProcessor.from_pretrained(path)
+            self.model.to(self.device)
+            self.model.eval()  # Set to evaluation mode
+            
+            logger.info("Model and processor loaded successfully!")
+        except Exception as e:
+            logger.error(f"Error loading model: {e}")
+            raise
+    
+    def _load_audio_from_bytes(self, audio_bytes: bytes) -> np.ndarray:
+        """
+        Load audio from bytes and resample to 16kHz.
+        
+        Args:
+            audio_bytes: Raw audio bytes
+            
+        Returns:
+            Audio waveform as numpy array
+        """
+        try:
+            # Load audio from bytes using librosa
+            audio_buffer = io.BytesIO(audio_bytes)
+            waveform, _ = librosa.load(audio_buffer, sr=16000, mono=True)
+            return waveform
+        except Exception as e:
+            logger.error(f"Error loading audio from bytes: {e}")
+            raise ValueError(f"Failed to load audio: {e}")
+    
+    def _load_audio_from_base64(self, base64_string: str) -> np.ndarray:
+        """
+        Load audio from base64-encoded string.
+        
+        Args:
+            base64_string: Base64-encoded audio data
+            
+        Returns:
+            Audio waveform as numpy array
+        """
+        try:
+            # Decode base64 string
+            audio_bytes = base64.b64decode(base64_string)
+            return self._load_audio_from_bytes(audio_bytes)
+        except Exception as e:
+            logger.error(f"Error decoding base64 audio: {e}")
+            raise ValueError(f"Failed to decode base64 audio: {e}")
+    
+    def __call__(self, data: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        Process incoming requests and return transcriptions.
+        
+        Expected input formats:
+        1. {"inputs": "base64_encoded_audio_string"}
+        2. {"inputs": {"audio": "base64_encoded_audio_string"}}
         3. Binary audio data in request body
-        
-        Args:
-            data: Input data dictionary
-            
-        Returns:
-            Dictionary containing transcription and metadata
-        """
-        try:
+
+        Dictionary requests may include {"parameters": {"language": "en"}}.
+        Omit language (or use null) to keep the model's default behavior.
+        
+        Args:
+            data: Input data dictionary
+            
+        Returns:
+            Dictionary containing transcription and metadata
+        """
+        try:
             logger.info("Processing inference request...")
-            
-            # Extract audio data from various input formats
-            waveform = None
-            
+
+            generate_kwargs = {}
             if isinstance(data, dict):
-                # Format 1: {"inputs": "base64_string"}
-                if "inputs" in data:
-                    inputs = data["inputs"]
-                    
-                    if isinstance(inputs, str):
-                        # Base64-encoded audio
-                        waveform = self._load_audio_from_base64(inputs)
-                    
-                    elif isinstance(inputs, dict):
-                        # Format 2: {"inputs": {"audio": "base64_string"}}
-                        if "audio" in inputs:
-                            waveform = self._load_audio_from_base64(inputs["audio"])
-                        else:
-                            raise ValueError("Missing 'audio' field in inputs dictionary")
-                    
-                    elif isinstance(inputs, bytes):
-                        # Binary audio data
-                        waveform = self._load_audio_from_bytes(inputs)
-                    
-                    else:
-                        raise ValueError(f"Unsupported input type: {type(inputs)}")
-                
-                # Direct audio field
-                elif "audio" in data:
-                    audio_data = data["audio"]
-                    if isinstance(audio_data, str):
-                        waveform = self._load_audio_from_base64(audio_data)
-                    elif isinstance(audio_data, bytes):
-                        waveform = self._load_audio_from_bytes(audio_data)
-                
-                else:
-                    raise ValueError("No valid audio data found in request. Expected 'inputs' or 'audio' field.")
-            
-            elif isinstance(data, (bytes, bytearray)):
-                # Format 3: Direct binary data
-                waveform = self._load_audio_from_bytes(bytes(data))
-            
-            else:
-                raise ValueError(f"Unsupported data type: {type(data)}")
-            
-            if waveform is None:
-                raise ValueError("Failed to extract audio from request")
-            
-            logger.info(f"Audio loaded: {len(waveform)} samples at 16kHz")
-            
-            # Process audio with the processor
-            input_features = self.processor(
-                waveform,
-                sampling_rate=16000,
-                return_tensors="pt"
-            ).input_features
-            
-            # Move to device
-            input_features = input_features.to(self.device, dtype=self.torch_dtype)
-            
-            # Run inference with forced Mandarin Chinese language
+                parameters = data.get("parameters", {})
+                if not isinstance(parameters, dict):
+                    raise ValueError("'parameters' must be a dictionary")
+                language = parameters.get("language")
+                if language is not None:
+                    if not isinstance(language, str) or not language.strip():
+                        raise ValueError("'language' must be a non-empty string or null")
+                    # Keep language overrides local to this request, including on
+                    # Transformers versions that modify the generation config.
+                    generate_kwargs = {
+                        "generation_config": deepcopy(self.model.generation_config),
+                        "language": language.strip(),
+                        "task": "transcribe",
+                    }
+            
+            # Extract audio data from various input formats
+            waveform = None
+            
+            if isinstance(data, dict):
+                # Format 1: {"inputs": "base64_string"}
+                if "inputs" in data:
+                    inputs = data["inputs"]
+                    
+                    if isinstance(inputs, str):
+                        # Base64-encoded audio
+                        waveform = self._load_audio_from_base64(inputs)
+                    
+                    elif isinstance(inputs, dict):
+                        # Format 2: {"inputs": {"audio": "base64_string"}}
+                        if "audio" in inputs:
+                            waveform = self._load_audio_from_base64(inputs["audio"])
+                        else:
+                            raise ValueError("Missing 'audio' field in inputs dictionary")
+                    
+                    elif isinstance(inputs, bytes):
+                        # Binary audio data
+                        waveform = self._load_audio_from_bytes(inputs)
+                    
+                    else:
+                        raise ValueError(f"Unsupported input type: {type(inputs)}")
+                
+                # Direct audio field
+                elif "audio" in data:
+                    audio_data = data["audio"]
+                    if isinstance(audio_data, str):
+                        waveform = self._load_audio_from_base64(audio_data)
+                    elif isinstance(audio_data, bytes):
+                        waveform = self._load_audio_from_bytes(audio_data)
+                
+                else:
+                    raise ValueError("No valid audio data found in request. Expected 'inputs' or 'audio' field.")
+            
+            elif isinstance(data, (bytes, bytearray)):
+                # Format 3: Direct binary data
+                waveform = self._load_audio_from_bytes(bytes(data))
+            
+            else:
+                raise ValueError(f"Unsupported data type: {type(data)}")
+            
+            if waveform is None:
+                raise ValueError("Failed to extract audio from request")
+            
+            logger.info(f"Audio loaded: {len(waveform)} samples at 16kHz")
+            
+            # Process audio with the processor
+            input_features = self.processor(
+                waveform,
+                sampling_rate=16000,
+                return_tensors="pt"
+            ).input_features
+            
+            # Move to device
+            input_features = input_features.to(self.device, dtype=self.torch_dtype)
+            
+            # Use the requested language, or the model's defaults when omitted.
             with torch.no_grad():
-                predicted_ids = self.model.generate(input_features)
-            
-            # Decode transcription
-            transcription = self.processor.batch_decode(
-                predicted_ids,
-                skip_special_tokens=True
-            )[0]
-            
-            logger.info(f"Transcription complete: {transcription[:100]}...")
-            
-            # Return result
-            return {
-                "transcription": transcription.strip(),
-                "audio_duration_seconds": float(len(waveform) / 16000),
-                "model": "AImpower/StutteredSpeechASR"
-            }
-        
-        except Exception as e:
-            logger.error(f"Error during inference: {e}", exc_info=True)
-            return {
-                "error": str(e),
-                "transcription": None
-            }
+                predicted_ids = self.model.generate(input_features, **generate_kwargs)
+            
+            # Decode transcription
+            transcription = self.processor.batch_decode(
+                predicted_ids,
+                skip_special_tokens=True
+            )[0]
+            
+            logger.info(f"Transcription complete: {transcription[:100]}...")
+            
+            # Return result
+            return {
+                "transcription": transcription.strip(),
+                "audio_duration_seconds": float(len(waveform) / 16000),
+                "model": "AImpower/StutteredSpeechASR"
+            }
+        
+        except Exception as e:
+            logger.error(f"Error during inference: {e}", exc_info=True)
+            return {
+                "error": str(e),
+                "transcription": None
+            }

tests/test_handler.py

@@ -0,0 +1,83 @@
+"""Handler regression tests; no model weights or endpoint required."""
+
+import unittest
+from types import SimpleNamespace
+from unittest.mock import Mock
+
+from handler import EndpointHandler
+
+
+class LanguageTests(unittest.TestCase):
+    def setUp(self):
+        self.handler = EndpointHandler.__new__(EndpointHandler)
+        self.handler.device = "cpu"
+        self.handler.torch_dtype = None
+        self.handler.model = Mock()
+        self.handler.model.generation_config = SimpleNamespace(language=None)
+        self.features = Mock()
+        self.features.to.return_value = self.features
+        self.handler.processor = Mock(return_value=SimpleNamespace(input_features=self.features))
+        self.handler.processor.batch_decode.return_value = [" test transcription "]
+        self.handler._load_audio_from_base64 = Mock(return_value=[0.0] * 16000)
+        self.handler._load_audio_from_bytes = Mock(return_value=[0.0] * 16000)
+
+    def assert_success(self, request):
+        self.assertEqual(self.handler(request), {
+            "transcription": "test transcription",
+            "audio_duration_seconds": 1.0,
+            "model": "AImpower/StutteredSpeechASR",
+        })
+
+    def test_languages_and_json_audio_formats(self):
+        for language in ("en", "english", "<|en|>", "zh", " en "):
+            for audio in ({"inputs": "audio"}, {"inputs": {"audio": "audio"}},
+                          {"audio": "audio"}, {"inputs": b"audio"}):
+                with self.subTest(language=language, audio=audio):
+                    self.assert_success({**audio, "parameters": {"language": language}})
+                    args, kwargs = self.handler.model.generate.call_args
+                    self.assertEqual(args, (self.features,))
+                    self.assertEqual(kwargs["language"], language.strip())
+                    self.assertEqual(kwargs["task"], "transcribe")
+                    self.assertIsNot(kwargs["generation_config"], self.handler.model.generation_config)
+
+    def test_default_requests_keep_existing_generation_call(self):
+        for request in ({"inputs": "audio"}, {"inputs": "audio", "parameters": {}},
+                        {"inputs": "audio", "parameters": {"language": None}},
+                        b"audio", bytearray(b"audio")):
+            with self.subTest(request=request):
+                self.assert_success(request)
+                self.handler.model.generate.assert_called_with(self.features)
+
+    def test_language_does_not_leak_between_requests(self):
+        def generate(features, **kwargs):
+            if "generation_config" in kwargs:
+                kwargs["generation_config"].language = kwargs["language"]
+            return [[1]]
+        self.handler.model.generate.side_effect = generate
+        for language in ("en", "zh"):
+            self.assert_success({"inputs": "audio", "parameters": {"language": language}})
+            self.assertIsNone(self.handler.model.generation_config.language)
+        self.assert_success({"inputs": "audio"})
+        self.handler.model.generate.assert_called_with(self.features)
+
+    def test_invalid_parameter_shapes_return_existing_error_format(self):
+        for parameters in (None, [], "en", {"language": ""}, {"language": " "},
+                           {"language": 123}, {"language": ["en"]}):
+            with self.subTest(parameters=parameters):
+                result = self.handler({"inputs": "audio", "parameters": parameters})
+                self.assertIsNone(result["transcription"])
+                self.assertIn("error", result)
+        self.handler.model.generate.assert_not_called()
+
+    def test_unsupported_language_error_is_returned(self):
+        self.handler.model.generate.side_effect = ValueError("Unsupported language: invalid")
+        result = self.handler({"inputs": "audio", "parameters": {"language": "invalid"}})
+        self.assertEqual(result, {"error": "Unsupported language: invalid", "transcription": None})
+
+    def test_unrelated_parameters_are_not_forwarded(self):
+        self.assert_success({"inputs": "audio", "parameters": {"max_new_tokens": 1}})
+        self.handler.model.generate.assert_called_with(self.features)
+
+
+if __name__ == "__main__":
+    unittest.main()