DEVLOG.md

# DEVLOG — totes-emosh

## 2026-06-07 chore: rename Fingerprint → EmotionMap; attribution footer; verdict bands

**Capability:** facial-expression-toy-app (static-only)
**Uses:** —
**Approach:** Rename across current-facing materials at Gordon's
call — "Fingerprint" doesn't make sense in context, defined term is
now **EmotionMap**. Bulk replace in `app.py`, `app/tiles.py` (PDF
title + temp-file prefix), `README.md`, `.codewright.yaml`, plus all
lesson docs (`formative-activity.html`, `expert-insight.{html,md}`,
`key-takeaways.{html,md}`, `W3L4-deliverable.md`,
`teaching/.../README.md`). Word template script renamed
`scratch/build_emotionmap_template.py`; Word file regenerated as
`Lesson4-EmotionMap-template.docx`; old `.docx` deleted.

Also (earlier same day): added the standard attribution footer to
the app (`gr.HTML` block under the download button) and to the
single-page PDF (line below the privacy reminder) — *Created by Dr.
Gordon Wright — A LittleMonkeyLab caper. Part of the Goldsmiths MSc
in Psychology, Week 3 Part 4.* Verdict bands made far more visible:
full-width coloured strips (green AGREES / orange DISAGREES), white
bold 15pt title + 11pt detail, taller tile (TILE_H 396, VERDICT_H
44). Empty tiles get a matching grey "awaiting your attempt" band so
the grid stays aligned. Deleted dead `app/description.py` (no
longer imported anywhere).

Smoke tests: `app.py` imports clean, both PDF modes render against
the six bundled sample faces (`lesson4-emotionmap*-{face,wireframe}.pdf`),
the attribution line and EmotionMap title both stamp correctly.
Historical DEVLOG / `toy-app-plan.md` entries deliberately left
intact for the historical record.

---

## 2026-06-05 feat: six-emotion replication grid + wireframe (face-free) mode + single-page A4 PDF artefact

**Capability:** facial-expression-toy-app (static-only)
**Uses:** mediapipe (478-point face mesh), Pillow (compositing), matplotlib (PDF)
**Approach:** Reshaped the toy app around the *replication challenge*
framing. New `app/tiles.py` owns:

- A six-tile grid (2 rows × 3 cols) keyed by `BASIC_EMOTIONS = [happy,
  sad, fear, disgust, anger, surprise]`. Neutral dropped — it isn't
  part of the replication ask.
- A composite per-tile renderer (`render_tile`, `render_filled_tile`,
  `render_empty_tile`) that produces a single 280×370 PIL image
  containing a header strip (`1. HAPPY`), the face (or wireframe)
  thumbnail, a 7-emotion classifier bar strip, and a coloured verdict
  line (`agrees: Happiness (0.99)` / `sees: Fear (0.51) — not Anger`).
- `render_wireframe()` — anonymised landmark mesh, drawn from
  hand-coded subsets of MediaPipe FACEMESH connections (face oval,
  lips outer + inner, both eyes, both brows, nose bridge) plus dots
  for every one of the 478 landmarks. The result is recognisably
  face-shaped — you can read brows pulled down on the angry attempt,
  a smile on the happy attempt — without any identifying photographic
  content.
- `export_single_page_pdf()` — one A4 portrait page with a title row,
  name strip, 2×3 tile grid, status footer, and privacy reminder.
  Replaces the two-page radar / two-mode export from earlier today.

State model: `app/session.py` now exposes
`empty_session() -> {emotion: Capture|None}` over the six emotions and
a `Capture` dataclass with `intended`, `landmarks`, `bbox`, and
`image_size` (the last three are needed so the wireframe can be
re-drawn at the exact crop of the face). `session_status()` reports
"N / 6 captured · classifier agreed on M / 6".

UI rewrite in `app.py`: gone are the logo row, multi-line header,
two-paragraph privacy banner, separate Static / Dynamic / Fingerprint
tabs, gr.Gallery, and the radar plot. Replaced by:

- A one-line `<h>` header with a small `<details>` privacy disclosure.
- A single compact input row (image upload/webcam + intended-emotion
  dropdown + wireframe toggle + submit + clear-all).
- A six-tile grid where every tile re-renders on every action.
- Per-tile **Retry** buttons that clear just that slot.
- A name field + single **Download single-page Fingerprint PDF**
  button at the bottom.

Toggling Wireframe re-renders every visible tile in real time, so
students can preview either format before they download.

`app/fingerprint.py` and `scratch/smoke_fingerprint.py` deleted —
radar and two-PDF export are gone. `app/app_utils.preprocess_image_and_predict()`
now returns the bbox and landmarks alongside the face/heatmap/probs/
blendshapes so the tiles module can reconstruct the wireframe from a
stored Capture.

Word doc Fingerprint template regenerated as a paper fallback — a
2×3 replication grid table mirroring the in-app layout, plus the
Privacy and Reflection sections. Lesson copy updated:
`formative-activity.html` now frames the activity as a
six-emotion replication challenge with the wireframe-vs-face format
choice; `expert-insight.md` and `key-takeaways.md` rewritten around
the per-slot agree/disagree pattern.

**Related:** online-teaching-for-bence

Reframing note (same day, post-shipping): Gordon shared the original
lecture brief, which puts the activity at slides 23-24 as the
experiential capstone after a deception-focused arc (Spy the Lie /
Duping Delight → Ekman micro-expressions → TSA SPOT boondoggle →
Porter & ten Brinke 2008 on posed emotion → FACS survived, the
deception theory didn't). The six-tile replication grid lands as a
single-person Porter & ten Brinke replication: pose each emotion in
turn; happy will be easiest to fake; the classifier disagreement
pattern *is* the data. No app changes needed for that reframing —
the existing per-tile agree/disagree verdict already exposes the
Porter & ten Brinke effect. Lesson copy in
`online-teaching-for-bence` updated accordingly:
`expert-insight.md` rewritten in Gordon's voice around the deception
arc, `formative-activity.html` now leads with the two stated LOs,
`key-takeaways.md` rewritten to match.

Smoke tests: `scratch/smoke_tiles.py` builds synthetic state, renders
both face and wireframe tile grids, and exports both PDF modes.
`scratch/smoke_real_face.py` runs the six bundled sample images
through the real classifier + landmarker; all six produced a face,
landmarks (478 each), bbox, and a top-class read — five correct
(Happy 0.99, Sad 0.99, Fear 0.99, Disgust 0.99, Surprise 0.76),
Anger misread as Fear 0.51 (which is itself the variability story
the lesson is now built around). Sample tile renders at
`scratch/tile-samples/` confirm the face mode and wireframe mode
both communicate the pose cleanly.

`PYTHONUNBUFFERED=1 uv run python app.py` reaches Gradio's
"Running on local URL" within a few seconds, no warnings, no
deprecations.

---

## 2026-06-05 chore: stripped to static-only; basic-emotions scope; video moves to a separate app

**Capability:** facial-expression-toy-app (static-only)
**Uses:** gradio, torch, mediapipe (tasks API), grad-cam
**Approach:** Activity scope re-cut at Gordon's call: too many trials,
basic emotions in the static path are enough on their own. Dynamic
Faces tab removed from `app.py`. `submit_video_and_add`,
`clear_dynamic_info`, and all video-tab widgets gone. `LABEL_PRESETS`
trimmed from 17 entries (warm-up × 7 + AU × 6 + condition × 3 + custom)
to 8 (basic emotions × 7 + custom). Static tab renamed to "Faces" and
the preset dropdown relabelled "Emotion you posed".

Dead code purged: deleted `app/face_utils.py` (`display_info` overlay
was video-only), `app/plot.py` (`statistics_plot` line graph was
video-only), `scratch/smoke_video_peaks.py`. `app/app_utils.py` now
exports only `preprocess_image_and_predict`. `app/model.py` no longer
loads the LSTM weights or class; `LSTMPyTorch` removed from
`app/model_architectures.py`; `config.toml` `[model_dynamic]` section
and stale `FRAME_DOWNSAMPLING` removed. Deleted the bundled LSTM
checkpoint (`FER_dinamic_LSTM_IEMOCAP.pt`, ~11 MB) and the leftover
demo render artefacts (`result_face.mp4`, `result_hm.mp4`).

Word-doc Fingerprint template regenerated against the slimmed scope:
A1/A2 (basic-emotion stills + radar) and B1/B2 (basic-emotion bar
charts + radar) — AU-build and crash-emotion sections gone. Privacy
checklist and reflection prompts retained, prompts retargeted to the
basic-emotions experience.

Bumped `config.toml` `APP_VERSION` to 0.3.0.

**Related:** online-teaching-for-bence

Smoke tests: `import` of `app.py` via the spec loader builds the
Blocks cleanly. `PYTHONUNBUFFERED=1 uv run python app.py` reaches
"Running on local URL: http://127.0.0.1:7861" with no warnings, no
deprecation notices. The Fingerprint PDF export still works against
the new label scheme — radar uses session-level means so it doesn't
care about label structure.

The video element will live in a separate app (TBC). The HF Space
push should now be of this slimmer canonical version.

---

## 2026-06-05 feat: R3 + R4 + R5 landed; Word doc Fingerprint template generated

**Capability:** facial-expression-toy-app
**Uses:** matplotlib (polar + PdfPages), python-docx
**Approach:** New `app/fingerprint.py` owns the session-level analytics
and export. R3: `session_radar(captures)` builds a polar plot of mean
classifier confidence per emotion across the whole session (Static
captures + video peaks combined), with one coloured marker per emotion
and a filled polygon. Empty sessions render a blank grid so the panel
doesn't pop in/out. R4: `export_fingerprint_pdf(captures, mode)` uses
`matplotlib.backends.backend_pdf.PdfPages` to assemble the Fingerprint
in either Format A (cover with radar, summary bar, then a 4-up face
grid per page) or Format B (cover with radar, summary bar, a labelled
table of top emotions, then per-capture probability bars 4-up). Returns
a NamedTemporaryFile path so `gr.File` can serve it as a download. R5:
`PRIVACY_BANNER` `gr.Markdown` block sits above the tab strip with a
new `.privacy-banner` CSS rule (warm cream block, amber left border).
New "Fingerprint" tab shows the live radar and a format-toggle radio +
"Download Fingerprint PDF" button; the radar updates from every event
that mutates the session (`submit`, `submit_dynamic`, remove-last,
clear-session). `pyproject.toml` and `requirements.txt` gain
`matplotlib>=3.8.0` and `python-docx>=1.1.0`.

Also generated the Word-doc Fingerprint template at
`Lesson4-Fingerprint-template.docx` (~39 KB) via
`scratch/build_fingerprint_template.py`. Single document carries both
Format A (warm-up stills, AU-build attempts, crash-emotion captures,
radar) and Format B (warm-up bars, AU table, per-condition time-series
tables, radar) panels with delete-the-other-format instructions; a
shared Privacy Confirm checklist and Reflection section close the
document. Mirrors §6 of `formative-activity.html` verbatim.

**Related:** online-teaching-for-bence

Smoke tests: `scratch/smoke_fingerprint.py` builds 7 synthetic
captures, renders the radar (`Figure`, 1 axis), exports Format A
(~75 KB PDF) and Format B (~79 KB PDF), and verifies the empty-session
paths produce valid PDFs too (~43 KB / 44 KB). Full app launch:
`PYTHONUNBUFFERED=1 uv run python app.py` reaches "Running on local
URL: http://127.0.0.1:7861" cleanly, no warnings, no deprecations.
UI flow (radar update on submit, format toggle, PDF download) needs a
browser eyeball — can't verify click wiring from CLI.

R-list now complete: R0 ✅ R1 ✅ R2 ✅ R3 ✅ R4 ✅ R5 ✅. Next step is
pushing back to the HF Space as the new canonical version once Gordon
has eyeballed the UI flow in a browser.

---

## 2026-06-03 feat: R2 — video sweep + per-emotion peak frames feed the shared session

**Capability:** facial-expression-toy-app
**Uses:** mediapipe, torch (LSTM), cv2, gradio
**Approach:** Extended `preprocess_video_and_predict` to keep a
`frame_records` buffer of `(frame_idx, face_crop, prob_vector,
blendshapes)` for each analysed frame (one every `FRAME_DOWNSAMPLING`
face-detected frames). Post-loop, `_extract_peak_captures` derives 7
`Capture` instances by per-emotion argmax across the LSTM probability
vectors. Returns the existing 4-tuple (videos + plot) extended to a
5-tuple with the peak captures list. `app.py` lifted `session_captures`
from the Static tab scope to the demo level so both tabs share one
session; new `submit_video_and_add` handler runs the video pipeline,
merges the 7 peaks into the same `gr.State` that R1's Static submit
writes to, and surfaces a status markdown on the Dynamic tab telling
the student what was added. `Capture.heatmap` made `Optional` —
peak captures don't carry a heatmap (would require 7 extra Grad-CAM
passes; pedagogically not required).
**Related:** online-teaching-for-bence

Smoke test on the bundled `videos/BillClinton.mp4`: 7 peak captures
extracted. Pedagogically useful artefact: `peak-from-video:happiness`
lands on a frame where the actual top class is *Anger* 0.48 — the
clip never registers strongly happy, so the "happiest" frame is
still angry. That mismatch between the label (what we extracted
for) and the top emotion (what the classifier called it) is the
variability story made directly visible in the gallery caption.

`scratch/smoke_video_peaks.py` runs the pipeline end-to-end on the
bundled clip; Gradio app boots in ~18s with no warnings.

---

## 2026-06-03 feat: R1 — multi-capture session memory landed

**Capability:** facial-expression-toy-app
**Uses:** gradio (gr.State, gr.Gallery), mediapipe, torch
**Approach:** New `app/session.py` carrying a `Capture` dataclass
(label, face crop, heatmap, emotion probabilities, blendshapes) plus
`LABEL_PRESETS` (17 entries: 7 warm-up emotions, 6 hard AUs from the
FACS Build-an-AU section, 3 crash-emotion conditions, and a custom
free-text option). `app_utils.preprocess_image_and_predict` extended
to a 4-tuple return that surfaces the blendshape dict R0 plumbed.
`app.py` Static tab rewritten: per-tab `gr.State` accumulates
`Capture` instances across submissions; a label-preset dropdown +
visibility-toggled custom textbox set the label before each submission;
a `gr.Gallery` renders the session as a grid of cropped-face thumbnails
captioned with the resolved label and the top emotion; Remove-last and
Clear-session buttons act on the state. Latest-capture readout (face +
heatmap + 7-emotion label) preserved alongside the gallery so each
submission gives immediate feedback. Also moved `gr.Blocks(css=)` to
`launch(css_paths=[...])` (Gradio 6.0 deprecation).
**Related:** online-teaching-for-bence

Smoke tests in `scratch/`: session ops correct, Capture renders the
right caption (`warm-up:happy · Happiness (0.99)`), blendshapes line
up with AU 12 (`mouthSmileLeft/Right` ~0.92 on Happy.png). Gradio app
boots in ~18s with no deprecation warnings, server reaches "Running
on" state. UI flow (gallery rendering, click handlers, visibility
toggles) needs manual browser eyeball — verifying click wiring from
the CLI isn't feasible.

Env hygiene: `numexpr` (transitive of the now-removed py-feat) was
hard-failing pandas import on numpy 2.x. Uninstalled — pandas degrades
gracefully without it. Worth a clean venv rebuild from `requirements.txt`
at some point but not blocking.

---

## 2026-06-03 feat: R0 — face detection switched to MediaPipe FaceLandmarker tasks API

**Capability:** facial-expression-toy-app
**Uses:** mediapipe (tasks API), torch, pytorch_grad_cam
**Approach:** Implemented R0 (Option δ). New `app/face_landmarker.py`
module wraps MediaPipe FaceLandmarker as a lazy singleton, exposing
`detect(pil_image) -> (landmarks, blendshapes)` and
`bbox_from_landmarks(...)`. Bundle downloads on first call via the
existing `load_model()` helper, driven by a new `[model_landmarker]`
section in `config.toml`. `app/app_utils.py` rewritten to call the new
module instead of `mp.solutions.face_mesh` (broken on Apple Silicon);
return signatures preserved so `app.py` is unchanged. Dead code
(`get_box`, `norm_coordinates`) removed from `app/face_utils.py`;
`display_info` retained for the dynamic-video overlay. Smoke test
post-R0 confirms end-to-end pipeline runs cleanly and classifier
output matches pre-R0 results within rounding (Disgust 0.987,
Fear 0.992, Happiness 0.986, Sadness 0.990, Surprise 0.763).
Blendshapes are extracted on every detection but not yet surfaced to
the UI — they land with R1's session memory rewrite.
**Related:** online-teaching-for-bence

Also cleaned: broken py-feat install removed from the venv. uv.lock
committed for reproducible builds.

---

## 2026-06-03 audit: pipeline walked, smoke-tested upstream, evaluated AU detection options

**Capability:** facial-expression-toy-app
**Uses:** mediapipe (tasks API), torch, pytorch_grad_cam (rejected: py-feat)
**Approach:** Walked the existing pipeline (`app.py` + `app/` package, six
modules) and ran three smoke tests under `scratch/`. Confirmed the
ResNet50 + Grad-CAM static path works on bundled samples — top class
hits the labelled emotion on every well-defined sample (Disgust 0.92,
Fear 0.97, Happiness 0.995, Sadness 0.99, Surprise 0.84; Anger
top-2 with Fear; "contempt" maps to Anger because the model has no
contempt class). MediaPipe legacy solutions API
(`mp.solutions.face_mesh` in `app_utils.py`) is broken on Apple Silicon
mediapipe>=0.10 — only the tasks API ships; blocks local dev as-is.
Py-feat 0.6.2 rejected as an AU-detection route: imports fail against
torchvision 0.27 (uses removed `torchvision.io.read_video`), confirmed
in both the main venv and an isolated venv. Pivoted to MediaPipe Face
Landmarker (tasks API), which downloads its own bundle and runs on
Apple Silicon: bilateral ARKit blendshape output maps cleanly onto the
lesson's target AUs (`browInnerUp` = AU 1, `cheekSquintLeft/Right` =
AU 6 Duchenne, `mouthDimpleLeft/Right` = AU 14 dimpler, etc.). Sample
sanity-check: Fear shows asymmetric `browOuterUp` 0.49/0.29; contempt
shows unilateral `mouthDimpleLeft` 0.10 alongside asymmetric smile.
**Related:** online-teaching-for-bence

**Recommendation pending user approval: Option δ.** Replace
`mp.solutions.face_mesh` with the tasks API Face Landmarker; reuse the
same model for both face cropping (478 landmarks → bbox, replacing
`get_box`) and blendshape output (AU-equivalent feedback). Keep
upstream ResNet50/LSTM for emotion classification. Three wins: fixes
local dev, gives the FACS Build-an-AU section the per-AU feedback it
needs, no new heavy dependency. Adds R0 ahead of R1 in the R-plan at
`~/Code/teaching/online-teaching-for-bence/toy-app-plan.md`:
"Replace legacy `mp.solutions.face_mesh` with the tasks API Face
Landmarker — required to run on Apple Silicon, unlocks blendshape
output." Awaiting user approval before implementation.

Smoke tests live under `scratch/` (gitignored content, scripts kept
local; not committed — they're throwaway evidence for this decision).

---

## 2026-06-03 chore: rename repo from all-a-bit-emotional to totes-emosh

**Capability:** facial-expression-toy-app
**Uses:** —
**Approach:** Renamed the local codewright fork from `all-a-bit-emotional`
to `totes-emosh` at Gordon's request. Updated `.codewright.yaml`,
`pyproject.toml`, README, DEVLOG, registry, and capability index.
Upstream HuggingFace Space slug at LittleMonkeyLab/All_a_bit_emotional
is unchanged.
**Related:** online-teaching-for-bence

---

## 2026-06-03 init: standalone fork of LittleMonkeyLab/All_a_bit_emotional HF Space

**Capability:** new: facial-expression-toy-app
**Uses:** gradio, torch, mediapipe, grad-cam, hf-hub (upstream Space)
**Approach:** Downloaded the live HuggingFace Space via `hf download` and
re-anchored it as a codewright-managed repo at
`~/Code/research/totes-emosh/` (originally named `all-a-bit-emotional`,
renamed same day). No code changes — clean fork of
the deployed Space as the baseline for the toy-app rework described in
`~/Code/teaching/online-teaching-for-bence/toy-app-plan.md` (R1–R5).
Added `pyproject.toml` as the codewright-canonical dependency manifest;
preserved upstream `requirements.txt` for HuggingFace Space deployment.
**Related:** online-teaching-for-bence

Standalone exists so the rework can be code-reviewed and version-controlled
before pushing back upstream as the new canonical Space. Original Space
remains live and unchanged at
https://huggingface.co/spaces/LittleMonkeyLab/All_a_bit_emotional.
Next session: walk the existing `app/` package and plan the R1 (multi-capture
session memory) integration point.

---