Spaces:
Running
Running
Commit History
-
Update Hugging Face Space short description
Oliver Nitsche +1: Claude Sonnet 4.6Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ac1c63b -
Fix critical alignment bug: pass full frame to warpAffine, not crop
Oliver Nitsche +1: Claude Sonnet 4.6YuNet returns landmark coordinates in the FULL FRAME pixel space (e.g. right eye at x=320, y=140 in a 640×480 image). The previous code cropped the face bounding box first: _embed(frame_bgr[y:y+h, x:x+w], lm) ...then passed those full-frame landmarks to _align, which applied warpAffine to the CROP using coordinates from the full frame. The transform therefore mapped to completely wrong pixels in the crop, producing a scrambled 112×112 tile that bore no relation to the face. Enrollment and recognition each produced different scrambled tiles, making cosine similarity effectively random — hence "always same person" (random embeddings cluster near a fixed enrolled vector). Fix: pass the full frame to _embed/_align so the landmark coordinates and the image coordinate space match: _embed(frame_bgr, lm) # full frame, full-frame landmarks Also improved the _align fallback (M=None) to derive a bbox from the landmark extents rather than plain-resizing the full frame. SCHEMA_VERSION bu…12fca32 -
Update app description
Oliver Nitsche +1: Claude Sonnet 4.6Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>dbdff5f -
Tighten recognition: raise threshold to 0.50, add 2-hit debounce
Oliver Nitsche +1: Claude Sonnet 4.6Two fixes for "always recognises the same person": 1. Threshold 0.35 → 0.50 MobileFaceNet embeddings for different people typically score 0.20–0.45; the same person scores 0.55–0.85. The old threshold of 0.35 sits right in the overlap zone, so random faces exceeded it. 0.50 sits safely above the inter-person range while remaining reachable for the same person. 2. 2-hit recognition debounce (RECOGNITION_DEBOUNCE = 2) The same name must appear in two consecutive 0.5-second frames before GREETING is triggered. A single ambiguous frame (slight pose change, partial occlusion) can no longer cause a false positive. The counter resets on every mismatch, NoFaceDetected, wake-up, or state change. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>8dc6771 -
Hold GREETING state for at least 4 s so UI always shows recognition
Oliver Nitsche +1: Claude Sonnet 4.6When TTS is unavailable speak() returns in milliseconds, causing the GREETING state to be entered and exited before the UI's 1-second poll could catch it — the user saw the robot go straight to sleep with no feedback at all. Fix: record the time before speak(); after it returns, sleep the remaining time up to a 4-second minimum. The UI is guaranteed to see at least three poll cycles with state="greeting" and the face snapshot, regardless of whether TTS is working. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>83afe82 -
Update app description in pyproject.toml
Oliver Nitsche +1: Claude Sonnet 4.6Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c29ef09 -
Make TTS robust: fallback to espeak, catch play_sound exceptions
Oliver Nitsche +1: Claude Sonnet 4.6Previously speak() silently returned when espeak-ng was missing (only a log warning) and left play_sound() exceptions uncaught, which would crash the state machine and leave the robot mute. Changes: - Auto-detect espeak-ng or espeak via shutil.which (cached per process); logs clearly at startup which engine is found or that none is available. - Wrap play_sound() in its own try/except so an audio driver error no longer propagates and crashes the GREETING/ENROLLING state. - Switch sleep calculation from words_per_second to words_per_minute to match the espeak -s flag unit; use 2.0 WPS (conservative at 140 WPM) + 1 s buffer so short phrases like "Hi Oliver!" are never cut off. - Wrap the entire body in a broad except so any unexpected error is logged rather than silently swallowed or propagated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>be17ec0 -
Replace Haar cascade with YuNet deep-learning face detector
Oliver Nitsche +1: Claude Sonnet 4.6The Haar cascade produced frequent false positives (walls, patterns) and had no confidence score to filter them. Replaced with YuNet (cv2.FaceDetectorYN), a small deep-learning detector built into OpenCV >= 4.8: - Confidence threshold (0.75) eliminates non-face detections - Returns 5-point facial landmarks (eyes, nose, mouth corners) directly, so the separate haarcascade_eye step is gone - 5-point similarity transform gives much more accurate alignment than the previous two-eye-centre approach - Model is tiny (~337 KB) and downloaded once from opencv_zoo Canonical 5-point InsightFace landmarks used for the warpAffine so MobileFaceNet embeddings are as consistent as possible across frames. SCHEMA_VERSION bumped to 4 → stale DB entries from the Haar pipeline are auto-discarded and the user is prompted to re-enroll. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>f7c60d9 -
Show face snapshot in UI for greeting and enrolling states
Oliver Nitsche +1: Claude Sonnet 4.6- face_db.py: add get_face_jpeg(frame_bgr) — detects the face, adds 40% padding, encodes as JPEG and returns bytes (None if no face found). - main.py: store snapshot in _shared whenever a face is detected (both known and unknown); clear it when returning to sleep. New GET /snapshot endpoint returns the JPEG, 204 when none is available. - index.html: <img id="face-snapshot"> in greeting section, <img id="enroll-snapshot"> in enroll section. - main.js: loadSnapshot() helper (cache-busts URL, hides img on error); called on state transition to greeting / enrolling. - style.css: circular 160×160 portrait style for both snapshot imgs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>f4e577f -
Fix face recognition: BGR→RGB, robust zip extraction, schema versioning
Oliver Nitsche +1: Claude Sonnet 4.6Two bugs made recognition impossible: 1. BGR→RGB: MobileFaceNet is trained on RGB images. _embed() was feeding raw OpenCV BGR arrays, producing systematically wrong embeddings. Fixed by adding cv2.cvtColor(BGR2RGB) before normalisation. 2. Zip entry path: w600k_mbf.onnx lives inside a buffalo_sc/ subdirectory in the release zip, not at the root. The hardcoded entry name caused extraction to fail (KeyError), so the model was never written to disk. Fixed by searching the zip namelist for any entry ending in the filename. Additional improvements: - SCHEMA_VERSION (=3) in face_db.py: load() auto-clears the DB when the embedding pipeline changes, so stale BGR embeddings from older code are discarded automatically on the first run after upgrade. - Threshold raised back to 0.35 (correct RGB embeddings are more consistent). - Detection more lenient: minNeighbors 4→3, minSize 60→40. - INFO/DEBUG logging of similarity scores for diagnosability. - POST /clear_db endpoint + "…6d06d8a -
Add GREETING state so recognised faces are shown in GUI and spoken
Oliver Nitsche +1: Claude Sonnet 4.6Previously, a matched face triggered speak() → goto_sleep() inline inside the ACTIVE state, so the GUI state jumped straight to "sleeping" and the user saw nothing. Changes: - New State.GREETING: entered as soon as a face is matched; sets _shared["recognized_name"] so the GUI shows the name immediately, before speak() is even called. If TTS is unavailable the name is still visible in the control panel. - /status now returns recognized_name alongside state. - GUI: new green greeting-section div shows "Welcome back, <name>!" when state is "greeting"; hidden in all other states. - STATE_LABELS updated with greeting entry. - Greeting section styled in style.css (green card). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ee7b214 -
Fix: always speak name after enrollment, even when face save fails
Oliver Nitsche +1: Claude Sonnet 4.6The enrollment loop was catching only ValueError, but add_face can also raise OSError (from save()), onnxruntime errors, or numpy exceptions. Any of those caused the exception to propagate past the loop, skipping the speak("Nice to meet you, {name}!") call entirely. Broadened the catch to Exception so that enrollment frame failures are logged at DEBUG level but never prevent the robot from greeting the user. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>897c12b -
Fix face recognition: align faces and enroll multiple frames
Oliver Nitsche +1: Claude Sonnet 4.6Two root causes for recognition failures: 1. MobileFaceNet expects faces aligned to a canonical eye position (InsightFace 112×112 standard). Plain resize produced embeddings that varied too much across frames for the same person. Added _align() which uses the OpenCV eye cascade to compute a similarity transform mapping detected eye centres to the standard positions, with plain resize as fallback when eyes aren't detectable. 2. Only one frame was enrolled, so any change in lighting or angle on the next encounter could push similarity below threshold. ENROLLING state now collects up to 5 frames while the user types their name; all frames with a detectable face are enrolled. Also lowered the matching threshold from 0.35 → 0.25 to account for residual variation when the eye cascade falls back to plain resize. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>6026dde -
Fix model extraction path for buffalo_sc v0.7 zip
Oliver Nitsche +1: Claude Sonnet 4.6The InsightFace buffalo_sc.zip release changed its internal layout — files are now at the zip root (w600k_mbf.onnx) rather than under a buffalo_sc/ subdirectory. The mismatched path caused a KeyError on first run, crashing the app with exit code 1. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>8ed4512 -
Align motion API with AGENTS.md recommendations
Oliver Nitsche +1: Claude Sonnet 4.6- Use goto_target() directly for the WAKING look (recommended primary API), with body_yaw=None to leave body position unchanged after wake_up() - Replace set_target(head=_look_direction(...)) with look_at_world(duration=0) in the ACTIVE scan loop; duration=0 calls set_target() internally - Remove hand-rolled _look_direction() helper that duplicated SDK rotation logic - Drop scipy dependency (only used by _look_direction) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1f086bc
-
Switch to local ONNX face recognition (no compilation, no cloud)
Oliver Nitsche +1: Claude Sonnet 4.6Replaces AWS Rekognition with onnxruntime + OpenCV Haar cascade + InsightFace MobileFaceNet (w600k_mbf.onnx, ~17 MB downloaded once on first run). Both onnxruntime and opencv-python-headless ship pre-built ARM64 wheels so pip install is fast with no compilation step. Face embeddings are stored locally in face_db.json as before. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>2fda523 -
Replace face-recognition/dlib with AWS Rekognition
Oliver Nitsche +1: Claude Sonnet 4.6Eliminates the ~15-minute dlib compilation on Raspberry Pi. boto3 and Pillow (pure Python, pre-built wheels) replace the face-recognition package. Face embeddings are now stored in an AWS Rekognition collection instead of a local face_db.json file; existing enrolled faces will need to be re-registered. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>77c6ffa -
Add reachy-mini keyword tag
Oliver Nitsche +1: Claude Sonnet 4.6Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>715f41e -
Initial commit
Oliver Nitscheb5a13fe