Update documentation

README.md

@@ -1,47 +1,114 @@
 ---
-title: ASL TTS Test
+title: Sign2Voice
 emoji: 🗣️
-colorFrom: blue
-colorTo: purple
+colorFrom: green
+colorTo: yellow
 sdk: gradio
 python_version: "3.12"
 app_file: app.py
 pinned: false
 ---
 
-# ASL TTS Test
+A local-first AI stack that translates sign language, intent, and expression
+into natural speech.
 
-MVP Gradio pour tester la pipeline ASL video/camera -> intent JSON -> llama.cpp -> Qwen3-TTS.
+Sign2Voice is a Gradio demo built for the Hugging Face Build Small Hackathon.
+It takes an uploaded or webcam ASL clip, extracts sign candidates and facial
+expression signals, converts the result into a compact intent JSON, then speaks
+the sentence with a local small-model voice stack.
+
+## Scope Note
+
+We would have loved to push Sign2Voice further toward open-ended ASL
+conversation during the hackathon. The main constraint was not the app shell or
+the voice stack; it was data. Public ASL resources are still uneven for this
+task: many usable datasets focus on isolated signs, dictionary retrieval, or
+fingerspelling, while fluent ASL needs signer diversity, real-world lighting and
+camera angles, temporal boundaries, facial grammar, body movement, and careful
+expert annotation.
+
+That is why this submission is intentionally evidence-first. It exposes top sign
+candidates, confidence thresholds, segment diagnostics, and emotion metadata
+instead of pretending that a small hackathon model can solve full ASL
+translation end to end. Microsoft Research describes sign-language modeling as
+being far behind spoken-language modeling largely because of a lack of
+appropriate training data, and recent SLR survey work calls data acquisition and
+annotation the main bottleneck for systems that work on fluent signing.
+
+References:
+
+- [ASL Citizen, Microsoft Research](https://www.microsoft.com/en-us/research/project/asl-citizen/)
+- [Trends and Challenges for Sign Language Recognition with Machine Learning, ESANN 2023](https://www.esann.org/sites/default/files/proceedings/2023/ES2023-7.pdf)
+
+## Hackathon Fit
+
+The project is aimed at the Backyard AI track: a practical communication tool
+for people who need fast sign-to-speech support without sending every clip to a
+cloud API.
+
+Build Small constraints covered by this repo:
+
+- Gradio app, ready for a Hugging Face Space.
+- Small-model stack under the 32B parameter limit.
+- Local-first runtime: ASL detection, llama.cpp text generation, and Qwen3-TTS
+  speech generation run in the app process.
+- Step-by-step demo flow that shows intermediate evidence instead of hiding
+  uncertainty.
+
+Badges that fit this build:
+
+- Off the Grid / Local-first: no cloud inference API is required at runtime.
+- Llama Champion: the intent-to-speech text step uses `llama.cpp`.
+- Off-Brand / Custom UI: the Space uses custom Gradio styling.
+- Field Notes: claim this only after publishing the build write-up.
+
+Badges not claimed:
+
+- Well-Tuned: this repo uses published models; it does not publish a new
+  fine-tuned model.
+- Sharing is Caring: no public agent trace is included yet.
 
 ## Pipeline
 
 ```text
-Video upload / camera capture
--> ASL sequential frame sampling
--> MediaPipe landmarks, if installed
--> TFLite ASL classifier over sliding temporal windows, if data/models/asl/model.tflite is present
--> gloss sequence aggregation
--> DeepFace emotion aggregation, if installed
--> llama.cpp subtitle + voice instruction
+Video upload or camera capture
+-> Sequential ASL frame sampling
+-> MediaPipe landmarks, when installed
+-> WLASL2000 I3D or TFLite ASL classifier
+-> Ordered gloss sequence with confidence diagnostics
+-> DeepFace emotion aggregation, when installed
+-> llama.cpp subtitle and voice instruction
 -> Qwen3-TTS audio
 ```
 
-The app is organized so each brick can fail independently with diagnostics instead of blocking
+Each brick can fail independently and return diagnostics instead of blocking
 the whole interface at startup.
 
-The demo screen is intentionally step-by-step:
+The demo screen is intentionally staged:
 
 ```text
-1 Analyze ASL -> debug overlay + intent JSON
-2 Generate subtitle -> llama.cpp output
-3 Generate speech -> Qwen3-TTS audio
+1. Analyze ASL -> debug overlay + intent JSON
+2. Generate subtitle -> llama.cpp output
+3. Generate speech -> Qwen3-TTS audio
 ```
 
-When the ASL classifier file is missing, the app reports `model_missing` and does not invent
-ASL words. An empty-by-default manual gloss override exists only under advanced debug controls
-for testing downstream LLM/TTS behavior.
+When no ASL classifier is available, Sign2Voice reports `model_missing` and
+does not invent ASL words. The manual gloss override is empty by default and
+lives under advanced debug controls for downstream LLM/TTS testing only.
+
+## Run Locally
+
+Install the default CPU-friendly dependency set:
+
+```bash
+pip install -r requirements.txt
+```
 
-## Local checks
+Start the Gradio app:
+
+```bash
+python3 app.py
+```
 
 Run unit tests:
 
@@ -58,16 +125,17 @@ python3 scripts/test_tts_brick.py
 python3 scripts/test_full_pipeline.py
 ```
 
-`scripts/test_asl_brick.py` creates a tiny temporary demo clip when no video path is supplied.
-It also writes a debug overlay video path. To test the transparent fallback:
+`scripts/test_asl_brick.py` creates a tiny temporary clip when no video path is
+supplied. It also writes a debug overlay video path. To test the transparent
+fallback:
 
 ```bash
 python3 scripts/test_asl_brick.py --gloss-override "I LOVE YOU"
 ```
 
-## ASL model files
+## ASL Model Files
 
-The ASL classifier assets are stored under:
+The ASL classifier assets live under:
 
 ```text
 data/models/asl/model.tflite
@@ -75,21 +143,21 @@ data/models/asl/train.csv
 data/models/asl/sign_to_prediction_index_map.json
 ```
 
-They come from:
+They come from
+[jamesjbustos/sign-language-recognition](https://github.com/jamesjbustos/sign-language-recognition).
 
-```text
-https://github.com/jamesjbustos/sign-language-recognition
-```
+Without these files, the ASL brick still samples frames and emits
+`model_missing` diagnostics. The TFLite model recognizes the isolated signs in
+`sign_to_prediction_index_map.json`; it is not a full sentence or fingerspelling
+recognizer. Predictions below `ASL_CONFIDENCE_THRESHOLD`, defaulting to `0.70`,
+are reported as `low_confidence` and are not forwarded as detected glosses.
 
-Without these files, the ASL brick still samples frames and emits `model_missing` diagnostics.
-This model recognizes the isolated signs listed in `sign_to_prediction_index_map.json`; it is not
-a full sentence or fingerspelling recognizer. Predictions below `ASL_CONFIDENCE_THRESHOLD`
-defaulting to `0.70` are reported as `low_confidence` and are not forwarded as detected glosses.
+Uploaded videos use a phrase-prototype mode: frames are read in temporal order,
+landmarks are extracted once, and the ASL model runs over sliding windows.
+Accepted window predictions are collapsed into an ordered gloss sequence before
+`llama.cpp` rewrites them as natural speech.
 
-Uploaded videos use a phrase-prototype mode: frames are read in temporal order, landmarks are
-extracted once, then the ASL model runs over sliding windows. Accepted window predictions are
-collapsed into an ordered gloss sequence before llama.cpp rewrites them as natural speech. Tune it
-with:
+Tune it with:
 
 ```text
 ASL_UPLOAD_TARGET_FPS=12
@@ -99,15 +167,17 @@ ASL_SEQUENCE_STRIDE=15
 ASL_CONFIDENCE_THRESHOLD=0.70
 ```
 
-This is still not full continuous ASL translation, but it lets recorded phrase clips become
-`hello where water`-style gloss sequences instead of one global class.
+This is still not full continuous ASL translation, but it lets recorded phrase
+clips become `hello where water`-style gloss sequences instead of one global
+class.
 
-An experimental WLASL2000 I3D backend is also available for broader vocabulary coverage. It uses
-`raghuhasan/asl2000-i3d` from Hugging Face, downloads the I3D architecture helper if needed, and
-falls back to the TFLite detector when `ASL_DETECTOR_BACKEND=auto` cannot initialize it.
+An experimental WLASL2000 I3D backend is also available for broader vocabulary
+coverage. It uses `raghuhasan/asl2000-i3d` from Hugging Face, downloads the I3D
+architecture helper if needed, and falls back to the TFLite detector when
+`ASL_DETECTOR_BACKEND=auto` cannot initialize it.
 
 ```text
-ASL_DETECTOR_BACKEND=auto       # default: try WLASL2000, then fallback to TFLite
+ASL_DETECTOR_BACKEND=auto       # default: WLASL2000, then TFLite
 ASL_DETECTOR_BACKEND=tflite     # lightweight TFLite-only backend
 ASL_DETECTOR_BACKEND=wlasl_i3d  # WLASL2000 I3D only
 WLASL_I3D_CONFIDENCE_THRESHOLD=0.20
@@ -116,15 +186,16 @@ WLASL_I3D_SEQUENCE_STRIDE=32
 WLASL_I3D_FRAME_SIZE=224
 ```
 
-The WLASL backend is heavier and more experimental. The model card reports 2,000 classes with
-32.48% top-1, 57.31% top-5, and 66.31% top-10 accuracy, so the UI exposes top candidates and
-segment diagnostics instead of hiding uncertainty.
+The WLASL backend is heavier and more experimental. Its model card reports
+2,000 classes with 32.48% top-1, 57.31% top-5, and 66.31% top-10 accuracy, so
+the UI exposes top candidates and segment diagnostics instead of hiding
+uncertainty.
 
-Live camera debug prioritizes speed over long temporal batching. It starts predicting after
-`LIVE_ASL_MIN_FRAMES=4`, keeps a rolling buffer of `LIVE_ASL_MAX_FRAMES=12`, and runs ASL
-prediction every `LIVE_ASL_PREDICT_EVERY=1` frame. DeepFace emotion is heavier, so it runs every
-`LIVE_EMOTION_EVERY=45` frames by default. The overlay and status panel still show the current top
-candidates even when the accepted gloss is empty because the confidence is below threshold.
+Live camera debug prioritizes speed over long temporal batching. It starts
+predicting after `LIVE_ASL_MIN_FRAMES=4`, keeps a rolling buffer of
+`LIVE_ASL_MAX_FRAMES=12`, and runs ASL prediction every
+`LIVE_ASL_PREDICT_EVERY=1` frame. DeepFace emotion is heavier, so it runs every
+`LIVE_EMOTION_EVERY=45` frames by default.
 
 Good first signs to test because they are in the model vocabulary:
 
@@ -132,7 +203,7 @@ Good first signs to test because they are in the model vocabulary:
 hello, where, who, why, yes, no, thankyou, please, water, happy, sad
 ```
 
-Reference clips/GIFs from the upstream demo list:
+Reference clips and GIFs from the upstream demo list:
 
 ```text
 hello    https://media.giphy.com/media/3o7TKNKOfKlIhbD3gY/giphy.gif
@@ -146,20 +217,21 @@ please   https://lifeprint.com/asl101/gifs-animated/pleasecloseup.gif
 water    https://lifeprint.com/asl101/gifs/w/water-2.gif
 ```
 
-## GPU dependencies
+## GPU Dependencies
 
-`flash-attn` is only useful on a CUDA GPU Space with compatible PyTorch/CUDA versions.
-Keep the default `requirements.txt` for CPU Spaces. If the Space is moved to a compatible
-GPU runtime, install the GPU dependency set instead:
+`flash-attn` is only useful on a CUDA GPU Space with compatible PyTorch/CUDA
+versions. Keep the default `requirements.txt` for CPU Spaces. If the Space is
+moved to a compatible GPU runtime, install the GPU dependency set instead:
 
 ```bash
 pip install -r requirements-gpu.txt --no-build-isolation
 ```
 
-## Full ASL dependencies
+## Full ASL Dependencies
 
-The default requirements keep the Space build lighter. For a full local ASL/emotion runtime with
-MediaPipe landmarks and DeepFace emotion detection, install:
+The default requirements keep the Space build lighter. For a full local
+ASL/emotion runtime with MediaPipe landmarks and DeepFace emotion detection,
+install:
 
 ```bash
 pip install -r requirements-asl-full.txt

app.py

@@ -71,34 +71,34 @@ def build_video_input(label: str) -> gr.Video:
     )
 
 
-with gr.Blocks(title="SignSpeak Local") as demo:
+with gr.Blocks(title="Sign2Voice") as demo:
     gr.HTML(
         """
         <main id="hero">
           <div class="brand-lockup">
             <span class="brand-mark" aria-hidden="true"></span>
             <div>
-              <p class="eyebrow">Local ASL speech console</p>
-              <h1>SignSpeak Local</h1>
+              <p class="eyebrow">Local-first sign-to-speech console</p>
+              <h1>Sign2Voice</h1>
             </div>
           </div>
           <div class="hero-grid">
             <div>
               <p class="hero-copy">
-                Translate camera or uploaded ASL clips into expressive speech with a
-                controllable ASL, llama.cpp, and Qwen3-TTS pipeline.
+                Translate camera or uploaded signing clips into natural speech with
+                visible intent, expression, and confidence diagnostics.
               </p>
               <div class="pipeline-rail" aria-label="Pipeline stages">
                 <span>Capture</span>
-                <span>ASL</span>
+                <span>Signs</span>
                 <span>Intent</span>
                 <span>Voice</span>
               </div>
             </div>
             <div class="system-strip" aria-label="System capabilities">
               <span>Camera ready</span>
-              <span>Offline LLM</span>
-              <span>Expressive TTS</span>
+              <span>llama.cpp local</span>
+              <span>Expressive voice</span>
             </div>
           </div>
         </main>
@@ -223,7 +223,7 @@ with gr.Blocks(title="SignSpeak Local") as demo:
     gr.HTML(
         """
         <p class="footer-note">
-          Hackathon target badges: Llama Champion, Off-Brand, Off the Grid.
+          Build Small badges targeted: Off the Grid, Llama Champion, Off-Brand.
         </p>
         """
     )

scripts/test_llm_brick.py

@@ -11,7 +11,7 @@ from signspeak.pipeline import DEFAULT_INTENT, json_text
 
 def main() -> None:
     parser = argparse.ArgumentParser(description="Run only the llama.cpp intent-to-text brick.")
-    parser.add_argument("--intent", help="Path to an intent JSON file. Uses default mock intent if omitted.")
+    parser.add_argument("--intent", help="Path to an intent JSON file. Uses the sample intent if omitted.")
     args = parser.parse_args()
 
     intent_json = Path(args.intent).read_text(encoding="utf-8") if args.intent else json_text(DEFAULT_INTENT)

signspeak/__init__.py

@@ -1,2 +1 @@
-"""SignSpeak local ASL-to-speech pipeline package."""
-
+"""Sign2Voice local sign-to-speech pipeline package."""

signspeak/pipeline.py

@@ -19,7 +19,7 @@ DEFAULT_INTENT = {
         "confidence": 0.83,
     },
     "communication_intent": "friendly_greeting",
-    "pipeline_stage": "mock_asl_intent_for_llama_cpp_test",
+    "pipeline_stage": "sample_asl_intent_for_llama_cpp_check",
 }
 
 DEFAULT_VIDEO_PATH = Path(__file__).resolve().parents[1] / "data" / "examples" / "videoplayback.mp4"
@@ -52,7 +52,7 @@ def apply_gloss_override(result: dict[str, Any], gloss_override: str | None) ->
     asl["top_prediction"] = " ".join(glosses)
     asl["status"] = f"{asl.get('status', 'unknown')}_with_manual_override"
     intent["detected_glosses"] = glosses
-    intent["communication_intent"] = "manual_gloss_override_for_demo"
+    intent["communication_intent"] = "manual_gloss_override_for_pipeline_check"
     intent.setdefault("diagnostics", {})["manual_gloss_override"] = True
     intent["diagnostics"]["override_reason"] = "ASL classifier is missing or uncertain; user supplied visible glosses."
     return result
@@ -88,7 +88,7 @@ def create_synthetic_demo_video() -> Path:
     except Exception as exc:
         raise RuntimeError("OpenCV is required to create the fallback demo video.") from exc
 
-    output_path = Path(tempfile.gettempdir()) / "signspeak_demo_input.mp4"
+    output_path = Path(tempfile.gettempdir()) / "sign2voice_demo_input.mp4"
     if output_path.exists():
         return output_path
 
@@ -111,7 +111,7 @@ def create_synthetic_demo_video() -> Path:
             cv2.circle(frame, (width - center_x, 144), 18, (129, 140, 248), -1)
             cv2.putText(
                 frame,
-                "SignSpeak demo",
+                "Sign2Voice demo",
                 (36, 214),
                 cv2.FONT_HERSHEY_SIMPLEX,
                 0.62,