Hugging Face's logo

fodorad
/
personalitylinmult-fi-models

Other
PyTorch
ONNX
personality
big-five
multimodal
affective-computing
lightning
linmult
Model card Files
xet
 Community
personalitylinmult-fi-models
File size: 4,745 Bytes
826470e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
---
license: other
license_name: chalearn-fi-derived
license_link: https://chalearnlap.cvc.uab.cat/dataset/24/description/
pretty_name: PersonalityLinMulT — FI Big Five champion weights
library_name: pytorch
pipeline_tag: other
tags:
- personality
- big-five
- multimodal
- affective-computing
- lightning
- linmult
---

# PersonalityLinMulT — FI Big Five champion weights

<!--
  This is the source of truth for the Hugging Face model card at
  https://huggingface.co/fodorad/personalitylinmult-fi-models
  Keep the two in sync (see `make push-hf-model-card`). The published card adds
  a YAML frontmatter block (license/tags/pipeline) that HF renders; this docs
  copy omits it.
-->

Trained checkpoints for the Big Five apparent-personality regression task from
[**PersonalityLinMulT**](https://github.com/fodorad/PersonalityLinMulT),
trained on the First Impressions V2 (FI) dataset. Every checkpoint here is a
**champion**: the current best-performing run for its feature combination,
promoted deliberately (never automatically) via the project's MLflow
Model Registry, and mirrored here so a model can be loaded with nothing but
`pip install personalitylinmult` — no MLflow, no repo clone.

## Usage

```python
from personalitylinmult import PersonalityModel

model = PersonalityModel.from_pretrained("wavlm_best-ccc")
scores = model.predict({"wavlm": wavlm_features})  # {trait: score in [0, 1]}
```

`model.feature_names` lists exactly which features a given checkpoint needs;
`model.traits` lists the five Big Five traits in output order
(`openness`, `conscientiousness`, `extraversion`, `agreeableness`,
`emotional_stability`).

### ONNX Runtime (optional, `pip install personalitylinmult[onnx]`)

```python
from personalitylinmult.onnx import PersonalityModelONNX

model = PersonalityModelONNX.from_pretrained("wavlm_best-ccc")
scores = model.predict_from_audio("clip.wav")  # raw audio/video -> predictions,
                                                # no torch/transformers/exordium
```

A separate class, not a flag on `PersonalityModel` — installing the plain
package never pulls in `onnxruntime`. WavLM's own ONNX export
(`wavlm-base-plus.onnx`, shared across every WavLM-based champion) downloads
on first use. See `docs/experiments.md` (Blocks 5-6) for the accuracy/speed
benchmark and the small, measured prediction drift from `ffmpeg`-based
resampling (~0.001-0.004 per trait).

## Model id naming

`{features joined by "_", multiword feature names use "-"}_best-{metric}`,
e.g.:

- `wavlm_best-ccc` — single-stream **LinT**, WavLM audio only.
- `wavlm-emotion2vec_best-ccc` — cross-modal **LinMulT**, audio fusion
  (WavLM + emotion2vec).
- `avt_best-ccc` — cross-modal **LinMulT**, all 7 features (audio + visual +
  text).

The architecture (LinT vs. LinMulT) is never part of the id: a checkpoint is
self-describing, and `PersonalityModel.from_pretrained(...)` reads it off the
downloaded checkpoint automatically.

`_best-{metric}` names which validation metric the run was selected on —
`ccc` (Lin's concordance correlation) is preferred over `mae`/`loss` for this
task, since elementwise losses collapse prediction variance toward the
training mean (see `docs/experiments.md` for the full comparison and the
`std_ratio` metric that catches this).

## Current champions

### `wavlm_best-ccc`

Single-stream **LinT**, WavLM audio only. Trained with `ccc` loss at
`batch_size=256` — the batch size that resolves CCC's per-batch statistical
noise problem (see `docs/experiments.md`, Block 3, for the full diagnosis).

| Metric | Value |
| --- | --- |
| test `mean(1 - MAE)` | 0.8918 |
| test `mean CCC` | 0.5604 |
| test `mean Pearson r` | 0.5609 |
| test `mean std_ratio` | 1.0013 |

- **MLflow experiment**: `fi_lint_wavlm-ccc-bs256`
- **MLflow run_id**: `efb88223fdd04753bb8cefc46a5b51f7`
- **Registered as**: `fi_lint_wavlm-ccc-bs256` v1, `@champion`
- **Git SHA**: `0f60e83f35061e8ee26af61e5a58ad1950986446`
- **Reproduce**: `make train-fi-wavlm ARGS="--set train.loss=ccc --set data.batch_size=256"`

Every other promoted champion (e.g. an AVT full-multimodal model, once one
finishes training and is evaluated) will be added here the same way —
promoted via `make promote-champion` and published via
`make push-champion-model`, never automatically.

## License

These weights are derived from training on the ChaLearn First Impressions V2
dataset and are released under the same terms as the dataset itself — see the
[official ChaLearn LAP release](https://chalearnlap.cvc.uab.cat/dataset/24/description/).
These are apparent-personality *perception* models: they predict how a panel
of annotators rated a person from a short clip, not any ground truth about the
person. Use accordingly.