Document which face path won, and the stochastic DPE eval
Browse files
README.md
CHANGED
|
@@ -98,8 +98,25 @@ closeup, is held out as an unseen view, and the whole `C2` session is held out a
|
|
| 98 |
session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside
|
| 99 |
the training split, so no evaluation view leaks into the face conditioning.
|
| 100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
## Limitations
|
| 102 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
- One subject per avatar. These are personalised avatars, not a generalisable model.
|
| 104 |
- Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention.
|
| 105 |
- Densification is disabled during training, and there is no oral-cavity geometry, so teeth
|
|
|
|
| 98 |
session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside
|
| 99 |
the training split, so no evaluation view leaks into the face conditioning.
|
| 100 |
|
| 101 |
+
## Which face path won
|
| 102 |
+
|
| 103 |
+
On P1, with identical data, schedule and architecture, `P1_smplx` **beat** `P1_dpe`. That is
|
| 104 |
+
why the other released avatars use the SMPL-X path. `P1_dpe` is the paper-faithful
|
| 105 |
+
formulation, and the right choice when you have no trustworthy face fit, but it is not the
|
| 106 |
+
stronger one on this data.
|
| 107 |
+
|
| 108 |
+
Read that carefully, though: whole-image and even head-crop metrics could not separate the
|
| 109 |
+
two at all, differing only in the fourth decimal. A face ablation moves roughly 1% of the
|
| 110 |
+
pixels, so whole-image PSNR is dominated by torso and clothing. Only a mouth region defined
|
| 111 |
+
from the jaw-driven SMPL-X vertices distinguished them (PSNR +0.58, SSIM +0.015, LPIPS
|
| 112 |
+
-10%). It is one subject, so treat it as a direction, not a settled result.
|
| 113 |
+
|
| 114 |
## Limitations
|
| 115 |
|
| 116 |
+
- **`P1_dpe` evaluation is not deterministic.** The per-frame face code is a random convex
|
| 117 |
+
combination of the two camera codes, redrawn on every sample, and that path runs at eval
|
| 118 |
+
time as well as during training. Restrict the evaluation to one camera per frame if you
|
| 119 |
+
need reproducible numbers.
|
| 120 |
- One subject per avatar. These are personalised avatars, not a generalisable model.
|
| 121 |
- Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention.
|
| 122 |
- Densification is disabled during training, and there is no oral-cavity geometry, so teeth
|