initialneil commited on
Commit
cd98315
·
verified ·
1 Parent(s): 033b370

Document which face path won, and the stochastic DPE eval

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -98,8 +98,25 @@ closeup, is held out as an unseen view, and the whole `C2` session is held out a
98
  session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside
99
  the training split, so no evaluation view leaks into the face conditioning.
100
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101
  ## Limitations
102
 
 
 
 
 
103
  - One subject per avatar. These are personalised avatars, not a generalisable model.
104
  - Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention.
105
  - Densification is disabled during training, and there is no oral-cavity geometry, so teeth
 
98
  session. The DPE codes were extracted from cam07 and cam30, both frontal tele views inside
99
  the training split, so no evaluation view leaks into the face conditioning.
100
 
101
+ ## Which face path won
102
+
103
+ On P1, with identical data, schedule and architecture, `P1_smplx` **beat** `P1_dpe`. That is
104
+ why the other released avatars use the SMPL-X path. `P1_dpe` is the paper-faithful
105
+ formulation, and the right choice when you have no trustworthy face fit, but it is not the
106
+ stronger one on this data.
107
+
108
+ Read that carefully, though: whole-image and even head-crop metrics could not separate the
109
+ two at all, differing only in the fourth decimal. A face ablation moves roughly 1% of the
110
+ pixels, so whole-image PSNR is dominated by torso and clothing. Only a mouth region defined
111
+ from the jaw-driven SMPL-X vertices distinguished them (PSNR +0.58, SSIM +0.015, LPIPS
112
+ -10%). It is one subject, so treat it as a direction, not a settled result.
113
+
114
  ## Limitations
115
 
116
+ - **`P1_dpe` evaluation is not deterministic.** The per-frame face code is a random convex
117
+ combination of the two camera codes, redrawn on every sample, and that path runs at eval
118
+ time as well as during training. Restrict the evaluation to one camera per frame if you
119
+ need reproducible numbers.
120
  - One subject per avatar. These are personalised avatars, not a generalisable model.
121
  - Driving signal must be a registered SMPL-X sequence in the DREAMS-AVATAR convention.
122
  - Densification is disabled during training, and there is no oral-cavity geometry, so teeth