Image Classification
Py-Feat
facial-expression-analysis
action-units
emotion-recognition
gaze-estimation
face-landmarks
head-pose
blendshapes
multitask
Instructions to use py-feat/face_multitask_v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Py-Feat
How to use py-feat/face_multitask_v2 with Py-Feat:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -21,10 +21,10 @@ A single multi-task convolutional model for facial behavior analysis, used by
|
|
| 21 |
[py-feat](https://github.com/cosanlab/py-feat)'s `Detectorv2`. From one face crop
|
| 22 |
it jointly predicts **action units, categorical emotion, valence/arousal,
|
| 23 |
eye gaze, a 478-point face mesh, 6-DoF head pose, and 52 MediaPipe/ARKit
|
| 24 |
-
blendshapes**
|
| 25 |
|
| 26 |
- **Backbone:** ConvNeXt-V2 Tiny (FCMAE + IN-22k/IN-1k pretrained)
|
| 27 |
-
- **Heads:**
|
| 28 |
gaze heads + landmark, pose, and **blendshape** regression heads
|
| 29 |
- **Params:** ~30M · **Input:** 224×224 RGB (from a 256×256 face crop)
|
| 30 |
- **File:** `face_multitask_v2.safetensors` (safetensors; `ModelV2Config` JSON in the file metadata)
|
|
@@ -54,12 +54,8 @@ blendshapes** (the v2.5 model; replaces v2.4).
|
|
| 54 |
| Gaze | MPIIGaze (leave-subject-out) | mean angular err | 7.05° |
|
| 55 |
| Gaze | Gaze360 (held-out split) | mean angular err | 12.89° |
|
| 56 |
|
| 57 |
-
Notes: **
|
| 58 |
-
|
| 59 |
-
Aff-Wild2 V/A (0.82/0.78 → 0.85/0.80). **Gaze numbers are now leave-subject-out
|
| 60 |
-
held-out** (honest generalization); the lower v2.4 figures (3.92°/6.81°) came from a
|
| 61 |
-
leaky evaluation that included training subjects, so they are not comparable — the
|
| 62 |
-
v2.5 numbers are the real ones. Numbers are from the deployed checkpoint
|
| 63 |
(`v25c_release_ep14`), weight-verified against the published `.safetensors`.
|
| 64 |
|
| 65 |
## Usage
|
|
|
|
| 21 |
[py-feat](https://github.com/cosanlab/py-feat)'s `Detectorv2`. From one face crop
|
| 22 |
it jointly predicts **action units, categorical emotion, valence/arousal,
|
| 23 |
eye gaze, a 478-point face mesh, 6-DoF head pose, and 52 MediaPipe/ARKit
|
| 24 |
+
blendshapes**.
|
| 25 |
|
| 26 |
- **Backbone:** ConvNeXt-V2 Tiny (FCMAE + IN-22k/IN-1k pretrained)
|
| 27 |
+
- **Heads:** AU graph (AFG/FGG/SC) + unified-feature emotion/V-A and
|
| 28 |
gaze heads + landmark, pose, and **blendshape** regression heads
|
| 29 |
- **Params:** ~30M · **Input:** 224×224 RGB (from a 256×256 face crop)
|
| 30 |
- **File:** `face_multitask_v2.safetensors` (safetensors; `ModelV2Config` JSON in the file metadata)
|
|
|
|
| 54 |
| Gaze | MPIIGaze (leave-subject-out) | mean angular err | 7.05° |
|
| 55 |
| Gaze | Gaze360 (held-out split) | mean angular err | 12.89° |
|
| 56 |
|
| 57 |
+
Notes: **Gaze numbers are now leave-subject-out
|
| 58 |
+
held-out** (honest generalization); Numbers are from the deployed checkpoint
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
(`v25c_release_ep14`), weight-verified against the published `.safetensors`.
|
| 60 |
|
| 61 |
## Usage
|