docs/API_CONTRACT_SPEC.md

Read-Along AI: API & Backend Contract Specification

1. Architecture Objective

This application utilizes a decoupled architecture. The Gradio frontend must remain completely agnostic to the underlying inference engine.

The application uses a Dual-Mode Hybrid Architecture. The Gradio UI events must pass through an abstraction layer (wrapper functions) in app.py that routes to either Modal endpoints (Turbo Mode) or local HF Space inference (Off the Grid Mode).

2. The Abstraction Layer & Routing

The Gradio UI events should call wrapper functions in app.py, never Modal endpoints directly. A UI toggle determines whether these wrappers route payloads to modal_inference.py or local_inference.py.

• def transcribe_audio(audio_filepath: str) -> str:
  • Input: The local file path to the .wav file generated by the Gradio microphone component.
  • Output: A clean, lower-cased string of the transcribed text.
• def synthesize_speech(target_text: str) -> str:
  • Input: The string of text to be spoken.
  • Output: The local file path to the generated .wav file to be played by the Gradio UI.
• def ask_minicpm_judge(target_text: str, transcript: str) -> bool:
  • Input: The displayed target sentence and ASR transcript.
  • Output: True when the transcript is an acceptable phonetic match, otherwise False.

3. Modal Endpoint Contracts (Phase 1 Backend)

Codex should write the Modal stub functions that the wrappers above will call. We will use Modal's @app.function() decorator for direct RPC calls rather than setting up web webhooks to reduce latency.

Endpoint A: Speech-to-Text (Cohere Transcribe)

• Modal Function Name: run_cohere_asr
• Payload In: audio_bytes (bytes) - The Gradio wrapper must read the .wav file into bytes before passing it to Modal to avoid file-path resolution errors across the cloud boundary.
• Payload Out: dict
  • Schema: {"text": "the dog ran fast", "status": "success"}

Endpoint B: Text-to-Speech (OpenBMB VoxCPM)

• Modal Function Name: run_voxcpm_tts
• Payload In: text (str) - The target sentence or word.
• Payload Out: bytes - The raw audio buffer of the generated speech. The Gradio wrapper is responsible for catching these bytes, writing them to a temporary .wav file, and passing the path back to the UI.

Endpoint C: Phonetic Evaluator (Fine-Tuned MiniCPM)

• Modal Function Name: run_minicpm_evaluator
• Model: kingkw1/minicpm-phonetic-evaluator
• Payload In: target sentence or target text (str), transcript (str)
• Payload Out: str
  • Schema: "True" or "False"
• Behavior: The endpoint loads the fine-tuned MiniCPM model with trust_remote_code=True, formats the prompt using the same instruction/input/output structure used during training, and returns a binary verdict for whether the ASR transcript is a valid phonetic match for the target sentence.

3.5 Local Endpoint Contracts (Off the Grid Backend)

Codex should write equivalent functions in local_inference.py to mirror the inputs/outputs above.

• ASR: Implement faster-whisper using the tiny.en model.
• TTS / Audio Help: Default Off the Grid audio help should use committed curriculum WAVs in data/curriculum_audio/ and prewarm word clips by slicing the matching label/timing files. Live local VoxCPM remains an optional fallback only when LOCAL_LIVE_TTS=1 is set.
• Evaluator: Use llama-cpp-python to load the minicpm-phonetic-evaluator-Q4_K_M.gguf file.

4. Error Handling & Fallbacks

Young users cannot parse stack traces.

• If the Modal ASR endpoint times out or fails, the transcribe_audio wrapper must catch the exception and return a specific string: "[ASR_ERROR]". The Gradio UI must handle this silently by asking the user to "Try pressing record again!"
• If the TTS endpoint fails, synthesize_speech must return None, and the UI should gracefully fail open (no audio plays, but the UI does not freeze).
• If the MiniCPM evaluator fails, the wrapper should fail closed and return False so incorrect readings are not accidentally marked as successful.
• Word-click assistance should not block the UI. The current app prewarms word clips from committed curriculum label timings in Off the Grid Mode, pre-generates from sentence TTS in Turbo Mode where possible, and falls back to browser speech synthesis when cached audio is unavailable.

5. Testing Strategy

As explicitly defined in Phase 1 (Verification Checkpoint 2), the backend API contract must be strictly isolated for testing:

• Backend and app contract tests live under tests/.
• Local unit tests should verify the Gradio-facing contracts without calling deployed services by default.
• Modal endpoint checks should live as explicit integration tests and run only when the developer opts in, since they call deployed infrastructure.