Spaces:
Running
Running
De-brand: drop Claude mentions, add source link, fix HF host suffix
Browse files- README.md +12 -8
- assets/index-Cz6a28xH.js +0 -0
- index.html +38 -9
README.md
CHANGED
|
@@ -11,11 +11,15 @@ short_description: Calibrated in-browser facial expression reader
|
|
| 11 |
|
| 12 |
# Live Expression Reader
|
| 13 |
|
| 14 |
-
Real-time facial-expression analysis with **calibrated confidence**, **literature-grounded cognitive states** (tired / focused / bored / stressed / engaged / confused / calm), and **
|
| 15 |
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
>
|
| 18 |
-
> **Source:** <https://huggingface.co/spaces/arjun10g/live-expression-reader>
|
| 19 |
|
| 20 |
---
|
| 21 |
|
|
@@ -29,13 +33,13 @@ Real-time facial-expression analysis with **calibrated confidence**, **literatur
|
|
| 29 |
- **Stability sparkline** — top-1 confidence over the last ~6 s.
|
| 30 |
- **Cognitive states** computed from PERCLOS (Wierwille 1994), blink rate (Stern 1984), expression entropy (D'Mello 2010), AU4 + AU7 (D'Mello & Graesser 2010), affect-lability V-flux (Kuppens 2010 / 2013), and Russell circumplex (1980). Hidden until 60 s of data is in the window.
|
| 31 |
- **Personal classifier** — opt-in: hold each emotion for 3 s; a diagonal-Gaussian Naive Bayes fits to your samples and runs alongside the model.
|
| 32 |
-
- **Why? / Ask / Summarize / Discuss recording** —
|
| 33 |
- **Recording** + **JSON export** — clip up to 10 minutes of session data for later analysis or research.
|
| 34 |
|
| 35 |
## Privacy
|
| 36 |
|
| 37 |
- Webcam frames are processed locally. **No frames ever leave the device.**
|
| 38 |
-
- The "Why?" feature sends only numerical features (calibrated probabilities, blendshape activations, valence / arousal) to
|
| 39 |
- Personal templates and saved recordings stay in your browser tab. Persistence to `localStorage` is opt-in and off by default; "Clear stored data" wipes it instantly.
|
| 40 |
- No analytics, no tracking, no cookies.
|
| 41 |
|
|
@@ -54,7 +58,7 @@ Open <http://127.0.0.1:5173/>.
|
|
| 54 |
Two pieces:
|
| 55 |
|
| 56 |
1. **Frontend** → Hugging Face Spaces (Static SDK).
|
| 57 |
-
2. **
|
| 58 |
|
| 59 |
### 1. Cloudflare Worker (one-time)
|
| 60 |
|
|
@@ -89,7 +93,7 @@ git commit -m "Deploy"
|
|
| 89 |
git push
|
| 90 |
```
|
| 91 |
|
| 92 |
-
The Space
|
| 93 |
|
| 94 |
## Stack
|
| 95 |
|
|
@@ -98,7 +102,7 @@ The Space will auto-build (instant for static) and serve at `https://<your-usern
|
|
| 98 |
| Face landmarks + 52 ARKit blendshapes | [`@mediapipe/tasks-vision`](https://www.npmjs.com/package/@mediapipe/tasks-vision) | Apache-2.0 |
|
| 99 |
| Emotion classifier | [HSEmotion `enet_b0_8_va_mtl`](https://github.com/HSE-asavchenko/face-emotion-recognition) | Apache-2.0 |
|
| 100 |
| ONNX inference | [`onnxruntime-web`](https://www.npmjs.com/package/onnxruntime-web) | MIT |
|
| 101 |
-
| AI explanations |
|
| 102 |
| Build + UI | Vite, TypeScript, Tailwind CDN, Inter / JetBrains Mono | MIT / Apache-2.0 |
|
| 103 |
|
| 104 |
## Research backing the cognitive-states layer
|
|
|
|
| 11 |
|
| 12 |
# Live Expression Reader
|
| 13 |
|
| 14 |
+
Real-time facial-expression analysis with **calibrated confidence**, **literature-grounded cognitive states** (tired / focused / bored / stressed / engaged / confused / calm), and **AI-powered explanations** grounded in the Facial Action Coding System. Runs entirely in your browser — webcam frames never leave your device.
|
| 15 |
|
| 16 |
+
[](https://arjun10g-live-expression-reader.static.hf.space)
|
| 17 |
+
[](https://huggingface.co/spaces/arjun10g/live-expression-reader/tree/main)
|
| 18 |
+
[](https://www.apache.org/licenses/LICENSE-2.0)
|
| 19 |
+
|
| 20 |
+
> **Live demo:** <https://arjun10g-live-expression-reader.static.hf.space>
|
| 21 |
>
|
| 22 |
+
> **Source:** <https://huggingface.co/spaces/arjun10g/live-expression-reader/tree/main>
|
| 23 |
|
| 24 |
---
|
| 25 |
|
|
|
|
| 33 |
- **Stability sparkline** — top-1 confidence over the last ~6 s.
|
| 34 |
- **Cognitive states** computed from PERCLOS (Wierwille 1994), blink rate (Stern 1984), expression entropy (D'Mello 2010), AU4 + AU7 (D'Mello & Graesser 2010), affect-lability V-flux (Kuppens 2010 / 2013), and Russell circumplex (1980). Hidden until 60 s of data is in the window.
|
| 35 |
- **Personal classifier** — opt-in: hold each emotion for 3 s; a diagonal-Gaussian Naive Bayes fits to your samples and runs alongside the model.
|
| 36 |
+
- **Why? / Ask / Summarize / Discuss recording** — an LLM reads the muscles + V/A + cognitive states + personal templates and explains them in plain language. Multi-turn chat with conversation history.
|
| 37 |
- **Recording** + **JSON export** — clip up to 10 minutes of session data for later analysis or research.
|
| 38 |
|
| 39 |
## Privacy
|
| 40 |
|
| 41 |
- Webcam frames are processed locally. **No frames ever leave the device.**
|
| 42 |
+
- The "Why?" feature sends only numerical features (calibrated probabilities, blendshape activations, valence / arousal) to the LLM, and only when you click.
|
| 43 |
- Personal templates and saved recordings stay in your browser tab. Persistence to `localStorage` is opt-in and off by default; "Clear stored data" wipes it instantly.
|
| 44 |
- No analytics, no tracking, no cookies.
|
| 45 |
|
|
|
|
| 58 |
Two pieces:
|
| 59 |
|
| 60 |
1. **Frontend** → Hugging Face Spaces (Static SDK).
|
| 61 |
+
2. **LLM proxy** → Cloudflare Worker (free tier). Holds the API key as a server-side secret so the browser bundle never sees it.
|
| 62 |
|
| 63 |
### 1. Cloudflare Worker (one-time)
|
| 64 |
|
|
|
|
| 93 |
git push
|
| 94 |
```
|
| 95 |
|
| 96 |
+
The Space serves at `https://<your-username>-<space-name>.static.hf.space`.
|
| 97 |
|
| 98 |
## Stack
|
| 99 |
|
|
|
|
| 102 |
| Face landmarks + 52 ARKit blendshapes | [`@mediapipe/tasks-vision`](https://www.npmjs.com/package/@mediapipe/tasks-vision) | Apache-2.0 |
|
| 103 |
| Emotion classifier | [HSEmotion `enet_b0_8_va_mtl`](https://github.com/HSE-asavchenko/face-emotion-recognition) | Apache-2.0 |
|
| 104 |
| ONNX inference | [`onnxruntime-web`](https://www.npmjs.com/package/onnxruntime-web) | MIT |
|
| 105 |
+
| AI explanations | LLM (proxied via Cloudflare Worker) | API |
|
| 106 |
| Build + UI | Vite, TypeScript, Tailwind CDN, Inter / JetBrains Mono | MIT / Apache-2.0 |
|
| 107 |
|
| 108 |
## Research backing the cognitive-states layer
|
assets/index-Cz6a28xH.js
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
index.html
CHANGED
|
@@ -6,7 +6,7 @@
|
|
| 6 |
<title>Live Expression Reader</title>
|
| 7 |
<meta
|
| 8 |
name="description"
|
| 9 |
-
content="Real-time facial expression analysis with calibrated confidence, cognitive states, and FACS-grounded
|
| 10 |
/>
|
| 11 |
<script src="https://cdn.tailwindcss.com"></script>
|
| 12 |
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
@@ -29,7 +29,7 @@
|
|
| 29 |
color: rgb(229, 229, 229);
|
| 30 |
}
|
| 31 |
</style>
|
| 32 |
-
<script type="module" crossorigin src="./assets/index-
|
| 33 |
</head>
|
| 34 |
<body
|
| 35 |
class="bg-gradient-to-b from-neutral-950 via-neutral-950 to-neutral-900 text-neutral-100 min-h-screen"
|
|
@@ -321,13 +321,22 @@
|
|
| 321 |
</section>
|
| 322 |
|
| 323 |
<footer
|
| 324 |
-
class="mt-10 pt-4 border-t border-neutral-900 text-xs text-neutral-500 flex flex-wrap gap-x-3 gap-y-1"
|
| 325 |
>
|
| 326 |
<span>Apache-2.0 / MIT throughout</span>
|
| 327 |
<span aria-hidden="true">·</span>
|
| 328 |
-
<span>HSEmotion + MediaPipe +
|
| 329 |
<span aria-hidden="true">·</span>
|
| 330 |
<span>Calibrated, in-browser, no tracking</span>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 331 |
</footer>
|
| 332 |
</main>
|
| 333 |
|
|
@@ -445,11 +454,11 @@
|
|
| 445 |
</h3>
|
| 446 |
<ul class="space-y-1.5 text-neutral-300">
|
| 447 |
<li>
|
| 448 |
-
<kbd>Why?</kbd> —
|
| 449 |
the slider window (drag the slider to set 2–30 s).
|
| 450 |
</li>
|
| 451 |
<li>
|
| 452 |
-
<kbd>Summarize session</kbd> —
|
| 453 |
session since you last calibrated.
|
| 454 |
</li>
|
| 455 |
<li>
|
|
@@ -471,8 +480,9 @@
|
|
| 471 |
<ul class="space-y-1.5 text-neutral-300">
|
| 472 |
<li>
|
| 473 |
<kbd>Record</kbd> — captures a clip of session data (max
|
| 474 |
-
10 minutes). After stopping, click
|
| 475 |
-
to have the AI analyze the
|
|
|
|
| 476 |
</li>
|
| 477 |
<li>
|
| 478 |
<kbd>Export</kbd> — downloads the session timeline as a JSON
|
|
@@ -523,11 +533,30 @@
|
|
| 523 |
<p class="text-emerald-100/90 text-xs leading-relaxed">
|
| 524 |
Webcam frames stay on this device. Only numerical features
|
| 525 |
(blendshapes, calibrated probabilities, valence / arousal)
|
| 526 |
-
are sent to
|
| 527 |
Summarize. Personal templates and saved recordings live only
|
| 528 |
in this browser tab; the storage opt-in is off by default.
|
| 529 |
</p>
|
| 530 |
</section>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 531 |
</div>
|
| 532 |
|
| 533 |
<div class="mt-6 flex justify-end">
|
|
|
|
| 6 |
<title>Live Expression Reader</title>
|
| 7 |
<meta
|
| 8 |
name="description"
|
| 9 |
+
content="Real-time facial expression analysis with calibrated confidence, cognitive states, and FACS-grounded explanations. Runs entirely in your browser."
|
| 10 |
/>
|
| 11 |
<script src="https://cdn.tailwindcss.com"></script>
|
| 12 |
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
|
|
|
| 29 |
color: rgb(229, 229, 229);
|
| 30 |
}
|
| 31 |
</style>
|
| 32 |
+
<script type="module" crossorigin src="./assets/index-Cz6a28xH.js"></script>
|
| 33 |
</head>
|
| 34 |
<body
|
| 35 |
class="bg-gradient-to-b from-neutral-950 via-neutral-950 to-neutral-900 text-neutral-100 min-h-screen"
|
|
|
|
| 321 |
</section>
|
| 322 |
|
| 323 |
<footer
|
| 324 |
+
class="mt-10 pt-4 border-t border-neutral-900 text-xs text-neutral-500 flex flex-wrap gap-x-3 gap-y-1 items-center"
|
| 325 |
>
|
| 326 |
<span>Apache-2.0 / MIT throughout</span>
|
| 327 |
<span aria-hidden="true">·</span>
|
| 328 |
+
<span>HSEmotion + MediaPipe + LLM (proxied)</span>
|
| 329 |
<span aria-hidden="true">·</span>
|
| 330 |
<span>Calibrated, in-browser, no tracking</span>
|
| 331 |
+
<span aria-hidden="true">·</span>
|
| 332 |
+
<a
|
| 333 |
+
href="https://huggingface.co/spaces/arjun10g/live-expression-reader/tree/main"
|
| 334 |
+
target="_blank"
|
| 335 |
+
rel="noopener"
|
| 336 |
+
class="inline-flex items-center gap-1 text-neutral-400 hover:text-indigo-300 underline decoration-dotted underline-offset-2 transition-colors"
|
| 337 |
+
>
|
| 338 |
+
View source ↗
|
| 339 |
+
</a>
|
| 340 |
</footer>
|
| 341 |
</main>
|
| 342 |
|
|
|
|
| 454 |
</h3>
|
| 455 |
<ul class="space-y-1.5 text-neutral-300">
|
| 456 |
<li>
|
| 457 |
+
<kbd>Why?</kbd> — the AI summarizes what's happening over
|
| 458 |
the slider window (drag the slider to set 2–30 s).
|
| 459 |
</li>
|
| 460 |
<li>
|
| 461 |
+
<kbd>Summarize session</kbd> — the AI summarizes the entire
|
| 462 |
session since you last calibrated.
|
| 463 |
</li>
|
| 464 |
<li>
|
|
|
|
| 480 |
<ul class="space-y-1.5 text-neutral-300">
|
| 481 |
<li>
|
| 482 |
<kbd>Record</kbd> — captures a clip of session data (max
|
| 483 |
+
10 minutes). After stopping, click
|
| 484 |
+
<kbd>Discuss recording</kbd> to have the AI analyze the
|
| 485 |
+
whole clip.
|
| 486 |
</li>
|
| 487 |
<li>
|
| 488 |
<kbd>Export</kbd> — downloads the session timeline as a JSON
|
|
|
|
| 533 |
<p class="text-emerald-100/90 text-xs leading-relaxed">
|
| 534 |
Webcam frames stay on this device. Only numerical features
|
| 535 |
(blendshapes, calibrated probabilities, valence / arousal)
|
| 536 |
+
are sent to the AI, and only when you click Why? / Ask /
|
| 537 |
Summarize. Personal templates and saved recordings live only
|
| 538 |
in this browser tab; the storage opt-in is off by default.
|
| 539 |
</p>
|
| 540 |
</section>
|
| 541 |
+
|
| 542 |
+
<section
|
| 543 |
+
class="rounded-lg bg-neutral-950/60 border border-neutral-800 p-3"
|
| 544 |
+
>
|
| 545 |
+
<h3 class="font-semibold text-neutral-200 mb-1.5">
|
| 546 |
+
Source & license
|
| 547 |
+
</h3>
|
| 548 |
+
<p class="text-neutral-400 text-xs leading-relaxed">
|
| 549 |
+
Apache-2.0. Built on HSEmotion, MediaPipe, onnxruntime-web,
|
| 550 |
+
Vite, and TypeScript.
|
| 551 |
+
<a
|
| 552 |
+
href="https://huggingface.co/spaces/arjun10g/live-expression-reader/tree/main"
|
| 553 |
+
target="_blank"
|
| 554 |
+
rel="noopener"
|
| 555 |
+
class="text-indigo-300 hover:text-indigo-200 underline decoration-dotted underline-offset-2"
|
| 556 |
+
>View the source on Hugging Face ↗</a
|
| 557 |
+
>.
|
| 558 |
+
</p>
|
| 559 |
+
</section>
|
| 560 |
</div>
|
| 561 |
|
| 562 |
<div class="mt-6 flex justify-end">
|