Serve NCAIR1/Hausa-ASR + opus-mt-ha-en over REST

Gradio SDK, but the REST routes are the interface: the Blocks UI is mounted
onto our own FastAPI so /asr and /translate stay at the root, where the
ScriptFlow backend already expects them.

/asr returns word-level timings in the shape Chirp 3 produced, so the
backend's existing cue splitter consumes the output unchanged.

Dockerfile

@@ -0,0 +1,26 @@
+# Hugging Face Space (Docker SDK), free CPU tier: 2 vCPU / 16GB RAM.
+FROM python:3.11-slim
+
+# ffmpeg decodes whatever audio the backend posts; transformers' ffmpeg_read
+# shells out to it rather than depending on a Python codec stack.
+RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg \
+    && rm -rf /var/lib/apt/lists/*
+
+# Spaces run as uid 1000; HF caches must be writable or model download fails.
+RUN useradd -m -u 1000 user
+USER user
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH \
+    HF_HOME=/home/user/.cache/huggingface \
+    PYTHONUNBUFFERED=1
+
+WORKDIR $HOME/app
+
+COPY --chown=user requirements.txt .
+RUN pip install --no-cache-dir --user -r requirements.txt
+
+COPY --chown=user app.py .
+
+# Spaces route external traffic to 7860.
+EXPOSE 7860
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,8 +1,8 @@
 ---
-title: Scriptflow Hausa
-emoji: 🌍
-colorFrom: blue
-colorTo: indigo
+title: ScriptFlow Hausa Inference
+emoji: 🎬
+colorFrom: indigo
+colorTo: purple
 sdk: gradio
 sdk_version: 6.24.0
 python_version: '3.12'
@@ -12,4 +12,57 @@ license: other
 short_description: Hausa ASR + MT for ScriptFlow
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# ScriptFlow inference service
+
+Hausa ASR + Hausa→English MT, served over HTTP so the Render backend does not
+have to hold ~2GB of model weights in a 512MB process.
+
+- **ASR** — [`NCAIR1/Hausa-ASR`](https://huggingface.co/NCAIR1/Hausa-ASR) (Whisper-small fine-tune)
+- **MT** — [`Helsinki-NLP/opus-mt-ha-en`](https://huggingface.co/Helsinki-NLP/opus-mt-ha-en)
+
+## Deploying
+
+1. Create a Space: **New Space → Gradio → Blank**, hardware **CPU basic (free)**.
+   Gradio rather than Docker because Docker Spaces need PRO. It costs nothing
+   here: Gradio is FastAPI underneath, so `/asr` and `/translate` sit at the
+   root exactly as they would have, with a status UI at `/ui`. `Dockerfile` is
+   kept for anyone who does have PRO — switch `sdk:` back to `docker` to use it.
+2. Push the contents of this directory to it. `packages.txt` installs ffmpeg.
+3. On the model page for `NCAIR1/Hausa-ASR`, **accept the licence** — the model
+   is gated (`gated: auto`), and without acceptance the download 403s.
+4. In **Space → Settings → Secrets**, set:
+   - `HF_TOKEN` — a read token from the account that accepted the licence
+   - `SERVICE_TOKEN` — any random string; the backend must send the same value
+
+The first request downloads weights and can take several minutes. `GET /`
+answers immediately throughout and reports load state, so you can watch it come
+up without holding a request open.
+
+## API
+
+`POST /asr` — raw audio bytes as the body, `X-Service-Token` header.
+
+```json
+{
+  "text": "...",
+  "duration": 41.2,
+  "wordLevel": true,
+  "words": [{"word": "sannu", "start": 0.4, "end": 0.9, "speaker": null}]
+}
+```
+
+Times are relative to the audio posted; the backend adds each chunk's offset.
+`end` may be `null` — that is meaningful, and the backend infers a real end from
+the following word rather than inventing a duration here.
+
+`POST /translate` — `{"texts": [...]}`, returns `{"translations": [...]}` with
+one entry per input, same order.
+
+## Notes
+
+- Free Spaces sleep after inactivity; the first call after a sleep pays the
+  cold start again.
+- CPU inference on Whisper-small runs roughly 1–3× realtime, so a 20-minute
+  chunk is minutes of compute, not seconds.
+- `NCAIR1/Hausa-ASR` is licensed with a 1000 active end-user cap for
+  non-commercial use. Check that against how ScriptFlow ships.

app.py

@@ -0,0 +1,252 @@
+"""
+ScriptFlow inference service — Hausa ASR + Hausa->English MT on a free HF Space.
+
+Exists so the Render backend stays a thin 512MB web process: Whisper-small plus
+an MT model needs ~2GB resident, which the free Render plan cannot hold. A free
+Space (2 vCPU / 16GB) can, and both are reached over plain HTTP.
+
+  GET  /          -> {"status", "asr", "mt"}   (never blocks on model load)
+  POST /asr       -> {"text", "duration", "words": [{word,start,end,speaker}]}
+  POST /translate -> {"translations": [...]}
+
+/asr returns word-level timings in exactly the shape Chirp 3 produced, so the
+backend's cue splitter consumes it unchanged.
+"""
+
+import os
+import time
+import threading
+
+import numpy as np
+from fastapi import FastAPI, Request, HTTPException, Header
+from pydantic import BaseModel
+
+ASR_MODEL = os.environ.get("ASR_MODEL", "NCAIR1/Hausa-ASR")
+MT_MODEL = os.environ.get("MT_MODEL", "Helsinki-NLP/opus-mt-ha-en")
+# NCAIR1/Hausa-ASR is gated: the token's account must have accepted the licence
+# on the model page, or from_pretrained 403s.
+HF_TOKEN = os.environ.get("HF_TOKEN") or None
+# Free Spaces are world-reachable. A shared secret keeps strangers off the CPU
+# budget; unset means open, which is fine for local testing only.
+SERVICE_TOKEN = os.environ.get("SERVICE_TOKEN") or None
+
+SAMPLE_RATE = 16000
+# Whisper's context is 30s. The pipeline slides this window with overlap so a
+# word straddling a boundary is still decoded once, correctly.
+CHUNK_LENGTH_S = float(os.environ.get("CHUNK_LENGTH_S", "30"))
+STRIDE_LENGTH_S = float(os.environ.get("STRIDE_LENGTH_S", "5"))
+MT_BATCH = int(os.environ.get("MT_BATCH", "16"))
+
+# Gradio owns the process on a Gradio-SDK Space, but it is a FastAPI app
+# underneath — so the REST routes below are the real interface and the little UI
+# mounted at /ui is only there to give the Space a face (and to make it obvious
+# at a glance whether the models have finished loading).
+app = FastAPI()
+
+# --------------------------------------------------------------- MODEL LOAD ---
+# Loading takes minutes on a cold Space (weights download + CPU init). Doing it
+# at import time would make the Space fail its health check and restart-loop, so
+# models load lazily behind a lock and / reports progress instead.
+_lock = threading.Lock()
+_models: dict = {"asr": None, "mt": None}
+_errors: dict = {"asr": None, "mt": None}
+
+
+def _load(kind: str):
+    if _models[kind] is not None:
+        return _models[kind]
+    with _lock:
+        if _models[kind] is not None:
+            return _models[kind]
+        from transformers import pipeline
+        t0 = time.time()
+        try:
+            if kind == "asr":
+                obj = pipeline(
+                    "automatic-speech-recognition",
+                    model=ASR_MODEL,
+                    token=HF_TOKEN,
+                    chunk_length_s=CHUNK_LENGTH_S,
+                    stride_length_s=STRIDE_LENGTH_S,
+                    device=-1,
+                )
+            else:
+                obj = pipeline("translation", model=MT_MODEL, token=HF_TOKEN, device=-1)
+        except Exception as e:
+            _errors[kind] = f"{type(e).__name__}: {e}"
+            raise
+        print(f"loaded {kind} in {time.time() - t0:.1f}s", flush=True)
+        _models[kind] = obj
+        return obj
+
+
+def _auth(token):
+    if SERVICE_TOKEN and token != SERVICE_TOKEN:
+        raise HTTPException(status_code=401, detail="bad service token")
+
+
+@app.get("/")
+def health():
+    return {
+        "status": "ok",
+        "asr": {"model": ASR_MODEL, "loaded": _models["asr"] is not None,
+                "error": _errors["asr"]},
+        "mt": {"model": MT_MODEL, "loaded": _models["mt"] is not None,
+               "error": _errors["mt"]},
+    }
+
+
+# ----------------------------------------------------------------------- ASR ---
+def _decode(raw: bytes) -> np.ndarray:
+    """
+    Any container -> float32 mono @16k. The backend already sends 16k mono WAV,
+    but decoding defensively costs nothing and keeps the service reusable.
+    """
+    from transformers.pipelines.audio_utils import ffmpeg_read
+    return ffmpeg_read(raw, SAMPLE_RATE)
+
+
+def _words_from_chunks(chunks, audio_secs: float):
+    """
+    Whisper chunks -> the backend's word contract.
+
+    With return_timestamps="word" each chunk is already one word. With the
+    phrase-level fallback a chunk is several words, so its span is divided
+    across them by character length — an approximation, but one that keeps cue
+    boundaries near the speech instead of collapsing them onto a single instant.
+    """
+    words = []
+    for ch in chunks or []:
+        text = (ch.get("text") or "").strip()
+        if not text:
+            continue
+        ts = ch.get("timestamp") or (None, None)
+        start = ts[0]
+        end = ts[1] if len(ts) > 1 else None
+        if start is None:
+            continue
+        start = float(start)
+        if audio_secs:
+            start = min(max(start, 0.0), audio_secs)
+        if end is not None:
+            end = float(end)
+            end = min(max(end, start), audio_secs) if audio_secs else max(end, start)
+
+        parts = text.split()
+        if len(parts) <= 1:
+            # A None end is meaningful downstream: the backend infers a real end
+            # from the next word rather than inventing a duration here.
+            words.append({"word": text, "start": start, "end": end, "speaker": None})
+            continue
+
+        span = (end - start) if end is not None else None
+        total = sum(len(p) for p in parts) or 1
+        cursor = start
+        for p in parts:
+            if span is None:
+                words.append({"word": p, "start": cursor, "end": None, "speaker": None})
+                cursor += 0.3
+            else:
+                width = span * (len(p) / total)
+                words.append({"word": p, "start": cursor, "end": cursor + width,
+                              "speaker": None})
+                cursor += width
+    return words
+
+
+@app.post("/asr")
+async def asr(request: Request, x_service_token: str = Header(default=None)):
+    _auth(x_service_token)
+    raw = await request.body()
+    if not raw:
+        raise HTTPException(status_code=400, detail="empty audio body")
+
+    audio = _decode(raw)
+    audio_secs = len(audio) / float(SAMPLE_RATE)
+    pipe = _load("asr")
+
+    # language/task are forced because a fine-tune sometimes ships a generation
+    # config still defaulting to English transcription, which silently produces
+    # garbage on Hausa audio.
+    gen = {"language": "ha", "task": "transcribe"}
+
+    t0 = time.time()
+    word_level = True
+    try:
+        out = pipe(audio.copy(), return_timestamps="word", generate_kwargs=gen)
+    except Exception as e:
+        # Word timings need alignment_heads in the generation config. Fine-tunes
+        # frequently drop them; phrase-level timings still make usable cues.
+        print(f"word timestamps unavailable ({type(e).__name__}: {e}) — phrase level",
+              flush=True)
+        word_level = False
+        out = pipe(audio.copy(), return_timestamps=True, generate_kwargs=gen)
+
+    words = _words_from_chunks(out.get("chunks"), audio_secs)
+    print(f"asr {audio_secs:.1f}s audio -> {len(words)} words in {time.time() - t0:.1f}s "
+          f"({'word' if word_level else 'phrase'} timings)", flush=True)
+
+    return {
+        "text": (out.get("text") or "").strip(),
+        "duration": audio_secs,
+        "wordLevel": word_level,
+        "words": words,
+    }
+
+
+# --------------------------------------------------------------- TRANSLATION ---
+class TranslateIn(BaseModel):
+    texts: list[str]
+
+
+@app.post("/translate")
+def translate(body: TranslateIn, x_service_token: str = Header(default=None)):
+    _auth(x_service_token)
+    texts = body.texts or []
+    if not texts:
+        return {"translations": []}
+
+    pipe = _load("mt")
+    # Blank inputs are held out and reinserted: MarianMT will happily emit a
+    # hallucinated sentence for an empty string.
+    idx = [i for i, t in enumerate(texts) if (t or "").strip()]
+    out = [""] * len(texts)
+    if idx:
+        res = pipe([texts[i] for i in idx], batch_size=MT_BATCH, truncation=True)
+        for i, r in zip(idx, res):
+            out[i] = (r.get("translation_text") or "").strip()
+    return {"translations": out}
+
+
+# ------------------------------------------------------------------- LAUNCH ---
+# A Gradio-SDK Space runs `python app.py` and expects something listening on
+# 7860. Mounting the Blocks onto our own FastAPI (rather than calling
+# demo.launch()) keeps /asr and /translate at the root, where the backend
+# already expects them, and puts the UI at /ui.
+def _ui_check(_):
+    import json as _json
+    return _json.dumps(health(), indent=2)
+
+
+def _build_ui():
+    import gradio as gr
+    with gr.Blocks(title="ScriptFlow Hausa inference") as demo:
+        gr.Markdown(
+            "## ScriptFlow inference service\n"
+            "Hausa ASR + Hausa→English MT. The REST API is the real interface:\n"
+            "`POST /asr` (raw audio body) and `POST /translate` "
+            "(`{\"texts\": [...]}`), both with an `X-Service-Token` header.\n\n"
+            "Press the button to see whether the models have loaded — the first "
+            "call after a cold start downloads ~1GB and takes several minutes."
+        )
+        out = gr.Code(label="GET /", language="json")
+        gr.Button("Check status").click(_ui_check, inputs=[gr.State(None)], outputs=out)
+    return demo
+
+
+if __name__ == "__main__":
+    import uvicorn
+    import gradio as gr
+
+    application = gr.mount_gradio_app(app, _build_ui(), path="/ui")
+    uvicorn.run(application, host="0.0.0.0", port=int(os.environ.get("PORT", 7860)))

packages.txt

@@ -0,0 +1 @@
+ffmpeg

requirements.txt

@@ -0,0 +1,8 @@
+--extra-index-url https://download.pytorch.org/whl/cpu
+
+gradio
+torch
+transformers>=4.44,<5
+sentencepiece
+sacremoses
+numpy