jptaylor commited on
Commit
243f5a9
·
verified ·
1 Parent(s): 9f7ac70

Release rc2

Browse files
README.md CHANGED
@@ -12,11 +12,11 @@ tags:
12
  - onnx
13
  ---
14
 
15
- # Pipecat Voice Reader (rc1)
16
 
17
- > **⚠️ Experimental.** This is a first release candidate, published for trying out and feedback. Its outputs,
18
- > names and calibration may change between releases, and its readings are cues for an LLM to weigh, not verdicts
19
- > to act on alone. Pin a revision, and read the limitations below before relying on it.
20
 
21
  A small CPU model that hears how a user *sounds*: tone of voice, sarcasm in the delivery, energy, laughter and
22
  speech rate. It is the voice half of a pair: it is made to run beside Pipecat's
@@ -28,6 +28,24 @@ writes the reading as RCS perception at the head of the user's message, after th
28
  ([emotion happy=0.62, neutral=0.30],[sarcasm value=0.08],[tone frustrated=0.71, neutral=0.22],[arousal value=0.34],[laughter value=0.02],[speech_rate value=3.1],[sarcasm_voice value=0.63]) Oh, great.
29
  ```
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ## Made to pair with the Tone Reader
32
 
33
  The two readers split the work, and each declares its own signals so the LLM sees both side by side:
@@ -41,7 +59,7 @@ The two readers split the work, and each declares its own signals so the LLM see
41
  The voice reader never writes `emotion` or `sarcasm`. Its `sarcasm_voice` is a weak cue on its own, and its
42
  declared description tells the LLM to weigh it with the Tone Reader's `sarcasm`: the case it earns its place in is
43
  words that read sincere, said in a voice that isn't. For example, from our test clips, "He's the greatest, I'd
44
- say." said sarcastically reads as happy and 2 % sarcastic from the words, and 15 % (above the 6.8 % threshold)
45
  from the voice.
46
 
47
  It does run without the Tone Reader: the LLM then gets `tone` instead of `emotion`, and no `sarcasm` from the
@@ -53,15 +71,16 @@ This release carries two sizes of the same model, one folder each, read by the s
53
 
54
  | Folder | `variant` | Params | int8 ONNX | Read time, p50 (M1 Max, 4 / 2 / 1 threads) | 1 thread, x86 server | Choose it for |
55
  |---|---|---|---|---|---|---|
56
- | `voice-reader-base/` | `"base"` (default) | 26M | 28.6 MB | 33 / 56 / 100 ms | ~100–150 ms | the best tone and sadness reading; adds the `sad` output |
57
- | `voice-reader-tiny/` | `"tiny"` | 12.5M | 13.3 MB | 16 / 26 / 47 ms | ~45 ms | one-vCPU deployments: sarcasm about as well, half the CPU |
58
 
59
  The read starts when the VAD says the user stopped and runs while the speech-to-text finalises, so it rarely
60
  delays the transcription; the cost to watch is CPU at the end of each turn, beside the VAD, the turn detector and
61
  the reply. Give `base` two threads where the machine allows.
62
 
63
  Each folder has `model.int8.onnx` (the default), `model.onnx` (fp32), `config.json` (calibration, threshold,
64
- output names) and `MANIFEST.json` (SHA-256 of every file, checked by the runtime on download).
 
65
 
66
  ## Use
67
 
@@ -75,7 +94,7 @@ pipeline = Pipeline([
75
  transport.input(),
76
  stt,
77
  PipecatToneReader(), # the words: emotion, sarcasm
78
- VoiceReaderProcessor(params=VoiceReaderParams(revision="rc1", variant="base", threads=2)), # the voice
79
  user_aggregator,
80
  llm,
81
  tts,
@@ -93,16 +112,23 @@ this repository at the pinned revision the first time.
93
  | Signal | Published to the LLM | Meaning |
94
  |---|---|---|
95
  | `tone` | yes, top two | how the voice sounds: positive, neutral, sad, frustrated (temperature-calibrated) |
96
- | `sarcasm_voice` | yes | the chance the delivery alone sounds sarcastic, Platt-calibrated on real conversation; below the release threshold (0.068) it is published under 0.5, as a guess; dropped under laughter |
97
  | `arousal` | yes | vocal energy, 0–1 |
98
  | `laughter` | yes | 0–1 |
99
  | `speech_rate` | yes | syllables per second |
100
  | `upset` | no, raw reading only | how clearly the speaker is upset, 0–1 |
101
- | `sad` (`base` only) | no, raw reading only | how likely listeners would hear sadness, 0–1 |
102
 
103
  Nothing is written for a turn with under 0.8 s of speech. The raw reading, including `upset` and `sad`, reaches
104
  the application through the processor's `on_reading` event.
105
 
 
 
 
 
 
 
 
106
  ## Evaluation
107
 
108
  Labels on real speech come from two audio LLM judges (Gemini 3.1 Pro preview and OpenAI gpt-audio-1.5) hearing
@@ -111,28 +137,33 @@ speakers never appear in training.
111
 
112
  | | `base` | `tiny` |
113
  |---|---|---|
114
- | **Sarcasm**, everyday conversation (4,366 turns, 203 sarcastic; Open Yap, otoSpeech, HarperValley), AUROC | 0.917 | 0.904 |
115
- | — at the release threshold: F1 / recall / false alarms on sincere turns | 0.575 / 0.50 / 1.3 % | 0.560 / 0.48 / 1.3 % |
116
- | — mean AUROC over four test sets (two conversational, Emolia, MUStARD++) | 0.791 | 0.788 |
117
- | — MUStARD++ (TV, audio only, evaluation only), AUROC | 0.656 | 0.654 |
118
- | **Tone** macro-F1: conversation / Emolia / CREMA-D (acted) / EmoNet-Voice (synthetic) | 0.706 / 0.661 / 0.887 / 0.563 | 0.712 / 0.627 / 0.872 / 0.457 |
119
- | **Sadness**: tone recall for sad, mean of four sets | 0.53 | 0.38 |
120
- | — `sad` output, AUROC: conversation (any judge heard sadness) / Emolia | 0.762 / 0.929 | – |
121
- | **Upset**, mean AUROC (three sets) | 0.867 | 0.856 |
 
122
  | **Arousal**, CCC against the training target | 0.89 | 0.88 |
123
- | **Laughter**, AUROC / F1 | 0.994 / 0.72 | 0.991 / 0.70 |
124
- | int8 vs fp32: tone agreement / sarcasm p95 \|Δp\| | 0.983 / 0.011 | 0.977 / 0.010 |
125
 
126
  ## Limitations
127
 
128
  - **Sarcasm from the voice alone is a weak cue.** At the threshold it catches about half of the sarcasm both
129
- judges heard, at about 1 false alarm in 75 sincere turns. Weigh it with the words, never alone. On synthetic
130
- (TTS) sarcasm it is near chance (AUROC 0.60 on our 20 demo pairs).
131
  - **Sadness in conversation is rare** (2–6 % of turns), so even a good detector's per-turn flags are mostly
132
- wrong (12–25 % precision at a 5 % false-alarm rate). Treat `sad` as a graded cue or average it over a
133
  conversation. It is not a measure of depression or of anyone's mental health, and a single turn cannot show
134
  either.
135
- - **Acted, theatrical sadness** is under-read: 23 of 34 CREMA-D sad clips.
 
 
 
 
136
  - English only; trained on conversational and podcast speech, 8 kHz telephone audio included.
137
  - Tone labels are LLM judgments of the audio, not human ratings.
138
 
@@ -142,14 +173,16 @@ speakers never appear in training.
142
  [`laion/whisper-tiny-base-emotion-voice-burst`](https://huggingface.co/laion/whisper-tiny-base-emotion-voice-burst)
143
  (CC BY 4.0), base and tiny encoders respectively, cut to an 8 s window, with masked statistics pooling and one
144
  small head per output.
145
- - **Data** (about 119k training windows): Open Yap 1K and otoSpeech conversations, HarperValleyBank calls
146
- (CC BY 4.0), the English YODAS part of Emilia from `laion/Emolia` (CC BY 4.0), and CREMA-D (ODbL / DbCL).
147
- Augmented with gain, noise, telephone band-limiting and codecs; never speed or pitch, since rate and pitch are
148
- the signal.
149
- - **Labels**: the two judges' verdicts on sarcasm, tone and upset; where they disagreed on tone, the mean of two
150
- audio emotion models (emotion2vec+ large and the LAION model) at half weight; the `sad` head on the share of
151
- judges who heard sadness.
152
- - **Tiny**: distilled from the base model's predictions at half weight, with base-labelled Emolia clips added.
 
 
153
  - **Calibration** on real conversational dev turns: temperature for tone, Platt for sarcasm and laughter; the
154
  sarcasm threshold keeps false alarms at 2 % of sincere dev turns.
155
 
 
12
  - onnx
13
  ---
14
 
15
+ # Pipecat Voice Reader (rc2)
16
 
17
+ > **⚠️ Experimental.** This is a release candidate, published for trying out and feedback. Its outputs, names and
18
+ > calibration may change between releases, and its readings are cues for an LLM to weigh, not verdicts to act on
19
+ > alone. Pin a revision, and read the limitations below before relying on it.
20
 
21
  A small CPU model that hears how a user *sounds*: tone of voice, sarcasm in the delivery, energy, laughter and
22
  speech rate. It is the voice half of a pair: it is made to run beside Pipecat's
 
28
  ([emotion happy=0.62, neutral=0.30],[sarcasm value=0.08],[tone frustrated=0.71, neutral=0.22],[arousal value=0.34],[laughter value=0.02],[speech_rate value=3.1],[sarcasm_voice value=0.63]) Oh, great.
29
  ```
30
 
31
+ ## What changed since rc1
32
+
33
+ - **Licence-clean targets.** Every training target in rc2 is a judge's verdict or vote share, a cue from LAION's
34
+ CC BY 4.0 model, or a measurement of the audio. rc1 trained its tone head, where the two judges disagreed, on the
35
+ mean of two third-party audio emotion models; rc2 uses their vote share instead and has no such model in its
36
+ targets, so its lineage is Whisper (MIT), LAION's affect model (CC BY 4.0), the corpora under Training and the
37
+ judges. One thread remains: the pseudo-labelled windows below carry the rc1 base model's readings, so rc1's
38
+ lineage reaches rc2 second-hand through those labels.
39
+ - **More judged conversation.** A second round of about 24,000 conversational turns rated by both judges went into
40
+ training, and its held-out turns are a new test set (below).
41
+ - **Pseudo-labelled speech.** 159,000 windows read by the rc1 base model (Emolia clips no judge has rated, and
42
+ conversation turns) joined the training set. The sarcasm and conversation-tone gains come from them.
43
+ - **`tiny` now has the `sad` output** (int8 14.1 MB, from 13.3).
44
+ - Sarcasm in conversation AUROC 0.917 → 0.921 (base) and 0.904 → 0.908 (tiny); conversation tone macro-F1
45
+ 0.706 → 0.754 and 0.712 → 0.739; acted sadness read 23 → 30 of 34 CREMA-D clips (base). Tone on EmoNet-Voice's
46
+ synthetic clips slips for base (0.563 → 0.510).
47
+ - The tone head leans neutral a little more on borderline turns than rc1 did: see `tone_bias` under Outputs.
48
+
49
  ## Made to pair with the Tone Reader
50
 
51
  The two readers split the work, and each declares its own signals so the LLM sees both side by side:
 
59
  The voice reader never writes `emotion` or `sarcasm`. Its `sarcasm_voice` is a weak cue on its own, and its
60
  declared description tells the LLM to weigh it with the Tone Reader's `sarcasm`: the case it earns its place in is
61
  words that read sincere, said in a voice that isn't. For example, from our test clips, "He's the greatest, I'd
62
+ say." said sarcastically reads as happy and 2 % sarcastic from the words, and 10 % (above the 6.1 % threshold)
63
  from the voice.
64
 
65
  It does run without the Tone Reader: the LLM then gets `tone` instead of `emotion`, and no `sarcasm` from the
 
71
 
72
  | Folder | `variant` | Params | int8 ONNX | Read time, p50 (M1 Max, 4 / 2 / 1 threads) | 1 thread, x86 server | Choose it for |
73
  |---|---|---|---|---|---|---|
74
+ | `voice-reader-base/` | `"base"` (default) | 26M | 28.6 MB | 28 / 49 / 92 ms | ~90–160 ms | the best tone and sadness reading |
75
+ | `voice-reader-tiny/` | `"tiny"` | 12.5M | 14.1 MB | 13 / 23 / 42 ms | ~45–70 ms | one-vCPU deployments: sarcasm about as well, half the CPU |
76
 
77
  The read starts when the VAD says the user stopped and runs while the speech-to-text finalises, so it rarely
78
  delays the transcription; the cost to watch is CPU at the end of each turn, beside the VAD, the turn detector and
79
  the reply. Give `base` two threads where the machine allows.
80
 
81
  Each folder has `model.int8.onnx` (the default), `model.onnx` (fp32), `config.json` (calibration, threshold,
82
+ output names) and `MANIFEST.json` (SHA-256 of every file, checked by the runtime on download). The `config.json` at
83
+ the root is the release index: the variants, their folders and files. The runtime reads it first on every load.
84
 
85
  ## Use
86
 
 
94
  transport.input(),
95
  stt,
96
  PipecatToneReader(), # the words: emotion, sarcasm
97
+ VoiceReaderProcessor(params=VoiceReaderParams(revision="rc2", variant="base", threads=2)), # the voice
98
  user_aggregator,
99
  llm,
100
  tts,
 
112
  | Signal | Published to the LLM | Meaning |
113
  |---|---|---|
114
  | `tone` | yes, top two | how the voice sounds: positive, neutral, sad, frustrated (temperature-calibrated) |
115
+ | `sarcasm_voice` | yes | the chance the delivery alone sounds sarcastic, Platt-calibrated on real conversation; below the release threshold (0.061 for `base`, 0.063 for `tiny`) it is published under 0.5, as a guess; dropped under laughter |
116
  | `arousal` | yes | vocal energy, 0–1 |
117
  | `laughter` | yes | 0–1 |
118
  | `speech_rate` | yes | syllables per second |
119
  | `upset` | no, raw reading only | how clearly the speaker is upset, 0–1 |
120
+ | `sad` | no, raw reading only | how likely listeners would hear sadness, 0–1 |
121
 
122
  Nothing is written for a turn with under 0.8 s of speech. The raw reading, including `upset` and `sad`, reaches
123
  the application through the processor's `on_reading` event.
124
 
125
+ **`tone_bias`.** `tone` is calibrated on real conversation, where most turns are neutral, so on a borderline turn
126
+ it leans neutral: a sad or frustrated delivery a listener would notice can come out as neutral 0.45, sad 0.40.
127
+ `VoiceReaderParams(tone_bias={"sad": 0.5, "frustrated": 0.5})` adds to those tones' logits before the softmax
128
+ (+0.5 multiplies a tone's odds by about 1.65), so they are published more readily; our test harness runs with
129
+ exactly that. The graded `sad` and `upset` readings are not affected by it. Leave it unset to read the calibrated
130
+ probabilities.
131
+
132
  ## Evaluation
133
 
134
  Labels on real speech come from two audio LLM judges (Gemini 3.1 Pro preview and OpenAI gpt-audio-1.5) hearing
 
137
 
138
  | | `base` | `tiny` |
139
  |---|---|---|
140
+ | **Sarcasm**, everyday conversation (3,945 turns both judges rated, 203 sarcastic; Open Yap, otoSpeech, HarperValley), AUROC | 0.921 | 0.908 |
141
+ | — at the release threshold: F1 / recall / false alarms on sincere turns | 0.571 / 0.47 / 1.0 % | 0.548 / 0.45 / 1.1 % |
142
+ | — second conversational test, held out from the rc2 judging round (5,037 turns, 272 sarcastic), AUROC | 0.862 | 0.853 |
143
+ | — mean AUROC over four test sets (two conversational, Emolia, MUStARD++) | 0.798 | 0.797 |
144
+ | — MUStARD++ (TV, audio only, evaluation only), AUROC | 0.669 | 0.662 |
145
+ | **Tone** macro-F1: conversation / Emolia / CREMA-D (acted) / EmoNet-Voice (synthetic) | 0.754 / 0.667 / 0.902 / 0.510 | 0.739 / 0.647 / 0.873 / 0.477 |
146
+ | **Sadness**: tone recall for sad, mean of four sets | 0.54 | 0.45 |
147
+ | — `sad` output, AUROC: conversation (any judge heard sadness) / Emolia | 0.755 / 0.928 | 0.740 / 0.923 |
148
+ | **Upset**, mean AUROC (three sets) | 0.869 | 0.857 |
149
  | **Arousal**, CCC against the training target | 0.89 | 0.88 |
150
+ | **Laughter**, AUROC / F1 | 0.993 / 0.64 | 0.991 / 0.64 |
151
+ | int8 vs fp32: tone agreement / sarcasm p95 \|Δp\| | 0.983 / 0.008 | 0.983 / 0.009 |
152
 
153
  ## Limitations
154
 
155
  - **Sarcasm from the voice alone is a weak cue.** At the threshold it catches about half of the sarcasm both
156
+ judges heard, at about 1 false alarm in 100 sincere turns. Weigh it with the words, never alone. On synthetic
157
+ (TTS) sarcasm it is near chance (AUROC 0.57 on our 20 demo pairs).
158
  - **Sadness in conversation is rare** (2–6 % of turns), so even a good detector's per-turn flags are mostly
159
+ wrong (12–24 % precision at a 5 % false-alarm rate). Treat `sad` as a graded cue or average it over a
160
  conversation. It is not a measure of depression or of anyone's mental health, and a single turn cannot show
161
  either.
162
+ - **Borderline tones lean neutral.** Calibrated on conversation, the tone head publishes neutral on turns a
163
+ listener might call sad or frustrated; `tone_bias` above is the knob. Acted sadness is read well (30 of 34
164
+ CREMA-D sad clips by `base`, 27 by `tiny`; the misses read as frustrated).
165
+ - **Laughter** ranks well but its published value is conservative: F1 0.64 at the calibrated 0.5 (rc1: 0.72), at
166
+ the same AUROC.
167
  - English only; trained on conversational and podcast speech, 8 kHz telephone audio included.
168
  - Tone labels are LLM judgments of the audio, not human ratings.
169
 
 
173
  [`laion/whisper-tiny-base-emotion-voice-burst`](https://huggingface.co/laion/whisper-tiny-base-emotion-voice-burst)
174
  (CC BY 4.0), base and tiny encoders respectively, cut to an 8 s window, with masked statistics pooling and one
175
  small head per output.
176
+ - **Data** (about 290k training windows): about 131k judged windows from Open Yap 1K and otoSpeech conversations
177
+ (two judging rounds), HarperValleyBank calls (CC BY 4.0), the English YODAS part of Emilia from `laion/Emolia`
178
+ (CC BY 4.0) and CREMA-D (ODbL / DbCL); plus 159k windows pseudo-labelled by the rc1 base model, 112,996 Emolia
179
+ clips no judge has rated and 46,021 conversation turns. Augmented with gain, noise, telephone band-limiting and
180
+ codecs; never speed or pitch, since rate and pitch are the signal.
181
+ - **Labels**: the two judges' verdicts on sarcasm, tone and upset; where they disagreed on tone, their vote share
182
+ over the four tones; the `sad` head on the share of judges who heard sadness. No audio emotion model's output is
183
+ a direct target in rc2. The pseudo-labelled windows carry rc1's readings, and rc1 did use two such models
184
+ (emotion2vec+ large and the LAION model) for its disagreed turns.
185
+ - **Tiny**: distilled from the rc2 base model's predictions at half weight, on the same data.
186
  - **Calibration** on real conversational dev turns: temperature for tone, Platt for sarcasm and laughter; the
187
  sarcasm threshold keeps false alarms at 2 % of sincere dev turns.
188
 
config.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "voice-reader",
3
+ "release": "rc2",
4
+ "staged": "2026-11-04T17:14:31+00:00",
5
+ "window_secs": 8.0,
6
+ "sample_rate": 16000,
7
+ "tones": [
8
+ "positive",
9
+ "neutral",
10
+ "sad",
11
+ "frustrated"
12
+ ],
13
+ "outputs": [
14
+ "tone",
15
+ "sarcasm",
16
+ "laughter",
17
+ "arousal",
18
+ "speech_rate",
19
+ "upset",
20
+ "sad"
21
+ ],
22
+ "variants": {
23
+ "base": {
24
+ "folder": "voice-reader-base",
25
+ "run": "v4-base-all-sad",
26
+ "init": "laion-base",
27
+ "sarcasm_tau": 0.060534259700932624,
28
+ "files": {
29
+ "model.onnx": {
30
+ "sha256": "8a6aa25f4a0ce43d7e1289a15c0dfa894a8848ee1fbba682c38869782d19ded9",
31
+ "bytes": 109728612
32
+ },
33
+ "model.int8.onnx": {
34
+ "sha256": "8dfd1f1e271d4d64b0d001220ffb7ebd28935cef630377ac37fb9ce939de1218",
35
+ "bytes": 28567980
36
+ },
37
+ "config.json": {
38
+ "sha256": "e76664925738674626c87ae92d475a5dcf373d74ce87737ff61b869bef644a84",
39
+ "bytes": 661
40
+ }
41
+ }
42
+ },
43
+ "tiny": {
44
+ "folder": "voice-reader-tiny",
45
+ "run": "student-tiny-v4-sad",
46
+ "init": "laion-tiny",
47
+ "sarcasm_tau": 0.06329724596107583,
48
+ "files": {
49
+ "model.onnx": {
50
+ "sha256": "ad95fc8ae3370f98b91630bc4a12b71aec43d0d652f09b37dfed085e5238a833",
51
+ "bytes": 53366574
52
+ },
53
+ "model.int8.onnx": {
54
+ "sha256": "31d06f9a42656a3144997efa1f085abe29bbcbf20722e5711ba34d86f430dcbf",
55
+ "bytes": 14140364
56
+ },
57
+ "config.json": {
58
+ "sha256": "a32094d77c53f0fda50cd5328d4f4fc937535ffe1bfaefba760cb1771b1fb60d",
59
+ "bytes": 665
60
+ }
61
+ }
62
+ }
63
+ }
64
+ }
voice-reader-base/MANIFEST.json CHANGED
@@ -1,46 +1,46 @@
1
  {
2
- "release": "rc1",
3
  "folder": "voice-reader-base",
4
- "run": "v3-base-sad",
5
- "staged": "2026-11-02T20:40:58+00:00",
6
  "files": {
7
  "model.onnx": {
8
- "sha256": "99975890aa7f1b0648af82a480233fdf641cee04bfee84370d0bed9c39d63313",
9
  "bytes": 109728612
10
  },
11
  "model.int8.onnx": {
12
- "sha256": "5c25605ea019521853c92347b48ced6f7706c14f21183f5de9da2a607c9fae95",
13
  "bytes": 28567980
14
  },
15
  "config.json": {
16
- "sha256": "72ed694b3ca24b368baa0d68e47fa90ccf271bf0d74895bff2a20aefb53cd165",
17
- "bytes": 658
18
  }
19
  },
20
  "parity_int8": {
21
  "windows": 2000,
22
- "tone_argmax_agreement": 0.983,
23
- "tone_max_abs_dp": 0.14630481600761414,
24
- "sarcasm_p95_abs_dp": 0.011146726086735725,
25
- "sarcasm_max_abs_dp": 0.053686708211898804,
26
- "laughter_p95_abs_dp": 0.0013878615573048592,
27
- "arousal_p95_abs_d": 0.026246830821037292,
28
- "speech_rate_p95_rel_d": 0.04056451842188835,
29
  "auroc_sarcasm": {
30
- "reference": 0.8036348918022429,
31
- "other": 0.8078700047385879
32
  },
33
  "auroc_sarcasm_both": {
34
- "reference": 0.8481948120698782,
35
- "other": 0.8499735309687665
36
  },
37
  "auroc_sarcasm_voice_only": {
38
- "reference": 0.9116690614837782,
39
- "other": 0.9187022612115253
40
  },
41
  "tone_macro_f1": {
42
- "reference": 0.5146829335401363,
43
- "other": 0.49449348337582855,
44
  "rows": 1546
45
  }
46
  }
 
1
  {
2
+ "release": "rc2",
3
  "folder": "voice-reader-base",
4
+ "run": "v4-base-all-sad",
5
+ "staged": "2026-11-04T17:14:31+00:00",
6
  "files": {
7
  "model.onnx": {
8
+ "sha256": "8a6aa25f4a0ce43d7e1289a15c0dfa894a8848ee1fbba682c38869782d19ded9",
9
  "bytes": 109728612
10
  },
11
  "model.int8.onnx": {
12
+ "sha256": "8dfd1f1e271d4d64b0d001220ffb7ebd28935cef630377ac37fb9ce939de1218",
13
  "bytes": 28567980
14
  },
15
  "config.json": {
16
+ "sha256": "e76664925738674626c87ae92d475a5dcf373d74ce87737ff61b869bef644a84",
17
+ "bytes": 661
18
  }
19
  },
20
  "parity_int8": {
21
  "windows": 2000,
22
+ "tone_argmax_agreement": 0.9825,
23
+ "tone_max_abs_dp": 0.11225080490112305,
24
+ "sarcasm_p95_abs_dp": 0.007848226465284824,
25
+ "sarcasm_max_abs_dp": 0.05139094591140747,
26
+ "laughter_p95_abs_dp": 0.0011351238936185837,
27
+ "arousal_p95_abs_d": 0.01641533151268959,
28
+ "speech_rate_p95_rel_d": 0.024630198255181313,
29
  "auroc_sarcasm": {
30
+ "reference": 0.8032301374190491,
31
+ "other": 0.8031314168377823
32
  },
33
  "auroc_sarcasm_both": {
34
+ "reference": 0.8373954473266279,
35
+ "other": 0.8395341450502911
36
  },
37
  "auroc_sarcasm_voice_only": {
38
+ "reference": 0.8842925206080314,
39
+ "other": 0.8799818498071542
40
  },
41
  "tone_macro_f1": {
42
+ "reference": 0.5149164346694854,
43
+ "other": 0.5091877352207718,
44
  "rows": 1546
45
  }
46
  }
voice-reader-base/config.json CHANGED
@@ -15,20 +15,20 @@
15
  "upset",
16
  "sad"
17
  ],
18
- "sarcasm_tau": 0.06803736669700389,
19
  "calibration": {
20
  "sarcasm": {
21
- "a": 1.678951463802178,
22
- "b": 0.12539543584903914
23
  },
24
  "laughter": {
25
- "a": 0.8057154083813632,
26
- "b": -0.09714794868282932
27
  },
28
- "tone_temperature": 0.8488899224103036
29
  },
30
- "run": "v3-base-sad",
31
  "init": "laion-base",
32
- "best_epoch": 7,
33
  "note": "sarcasm_tau is the dev 2 % FPR threshold on the Platt-calibrated probability; tone temperature and laughter Platt curve fitted on dev"
34
  }
 
15
  "upset",
16
  "sad"
17
  ],
18
+ "sarcasm_tau": 0.060534259700932624,
19
  "calibration": {
20
  "sarcasm": {
21
+ "a": 1.7739810811041519,
22
+ "b": 0.311429803621114
23
  },
24
  "laughter": {
25
+ "a": 0.8804135276963087,
26
+ "b": 0.22502338628323926
27
  },
28
+ "tone_temperature": 0.9473796186181745
29
  },
30
+ "run": "v4-base-all-sad",
31
  "init": "laion-base",
32
+ "best_epoch": 8,
33
  "note": "sarcasm_tau is the dev 2 % FPR threshold on the Platt-calibrated probability; tone temperature and laughter Platt curve fitted on dev"
34
  }
voice-reader-base/model.int8.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5c25605ea019521853c92347b48ced6f7706c14f21183f5de9da2a607c9fae95
3
  size 28567980
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dfd1f1e271d4d64b0d001220ffb7ebd28935cef630377ac37fb9ce939de1218
3
  size 28567980
voice-reader-base/model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:99975890aa7f1b0648af82a480233fdf641cee04bfee84370d0bed9c39d63313
3
  size 109728612
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a6aa25f4a0ce43d7e1289a15c0dfa894a8848ee1fbba682c38869782d19ded9
3
  size 109728612
voice-reader-tiny/MANIFEST.json CHANGED
@@ -1,46 +1,46 @@
1
  {
2
- "release": "rc1",
3
  "folder": "voice-reader-tiny",
4
- "run": "student-tiny-v3",
5
- "staged": "2026-11-02T20:40:59+00:00",
6
  "files": {
7
  "model.onnx": {
8
- "sha256": "873b5e5991f607483331ca7078b175dadaee3a1ec2046507ae65f008db64cad0",
9
- "bytes": 50202707
10
  },
11
  "model.int8.onnx": {
12
- "sha256": "1823e7d7e861586f3981ffa470150a5c3653f5d017df97a49e606ccc4e6c6c42",
13
- "bytes": 13332966
14
  },
15
  "config.json": {
16
- "sha256": "79582d3816d963a092dd4d71da61d4444b3127659f846076223d20f240fc821d",
17
- "bytes": 655
18
  }
19
  },
20
  "parity_int8": {
21
  "windows": 2000,
22
- "tone_argmax_agreement": 0.977,
23
- "tone_max_abs_dp": 0.2388080656528473,
24
- "sarcasm_p95_abs_dp": 0.010020742192864418,
25
- "sarcasm_max_abs_dp": 0.04528002440929413,
26
- "laughter_p95_abs_dp": 0.0021592299453914165,
27
- "arousal_p95_abs_d": 0.019707761704921722,
28
- "speech_rate_p95_rel_d": 0.03602607175707817,
29
  "auroc_sarcasm": {
30
- "reference": 0.7930224293160638,
31
- "other": 0.7871979150213236
32
  },
33
  "auroc_sarcasm_both": {
34
- "reference": 0.8221916357861302,
35
- "other": 0.819777660137639
36
  },
37
  "auroc_sarcasm_voice_only": {
38
- "reference": 0.8642516826741284,
39
- "other": 0.8534371927701732
40
  },
41
  "tone_macro_f1": {
42
- "reference": 0.45427391583654375,
43
- "other": 0.46146068669445456,
44
  "rows": 1546
45
  }
46
  }
 
1
  {
2
+ "release": "rc2",
3
  "folder": "voice-reader-tiny",
4
+ "run": "student-tiny-v4-sad",
5
+ "staged": "2026-11-04T17:14:31+00:00",
6
  "files": {
7
  "model.onnx": {
8
+ "sha256": "ad95fc8ae3370f98b91630bc4a12b71aec43d0d652f09b37dfed085e5238a833",
9
+ "bytes": 53366574
10
  },
11
  "model.int8.onnx": {
12
+ "sha256": "31d06f9a42656a3144997efa1f085abe29bbcbf20722e5711ba34d86f430dcbf",
13
+ "bytes": 14140364
14
  },
15
  "config.json": {
16
+ "sha256": "a32094d77c53f0fda50cd5328d4f4fc937535ffe1bfaefba760cb1771b1fb60d",
17
+ "bytes": 665
18
  }
19
  },
20
  "parity_int8": {
21
  "windows": 2000,
22
+ "tone_argmax_agreement": 0.983,
23
+ "tone_max_abs_dp": 0.21781721711158752,
24
+ "sarcasm_p95_abs_dp": 0.009449773468077183,
25
+ "sarcasm_max_abs_dp": 0.0660611093044281,
26
+ "laughter_p95_abs_dp": 0.003110053250566125,
27
+ "arousal_p95_abs_d": 0.019918425008654594,
28
+ "speech_rate_p95_rel_d": 0.023557225242257118,
29
  "auroc_sarcasm": {
30
+ "reference": 0.8067347180540199,
31
+ "other": 0.8055303269625651
32
  },
33
  "auroc_sarcasm_both": {
34
+ "reference": 0.8388777130757015,
35
+ "other": 0.8375224986765485
36
  },
37
  "auroc_sarcasm_voice_only": {
38
+ "reference": 0.8949557589049384,
39
+ "other": 0.886863797927853
40
  },
41
  "tone_macro_f1": {
42
+ "reference": 0.47394276820506326,
43
+ "other": 0.4619192915760122,
44
  "rows": 1546
45
  }
46
  }
voice-reader-tiny/config.json CHANGED
@@ -12,22 +12,23 @@
12
  "laughter",
13
  "arousal",
14
  "speech_rate",
15
- "upset"
 
16
  ],
17
- "sarcasm_tau": 0.06590602517797733,
18
  "calibration": {
19
  "sarcasm": {
20
- "a": 1.7731141168712765,
21
- "b": 0.06321434642450725
22
  },
23
  "laughter": {
24
- "a": 0.8303770152621825,
25
- "b": -0.17392277961573782
26
  },
27
- "tone_temperature": 0.7673753909755566
28
  },
29
- "run": "student-tiny-v3",
30
  "init": "laion-tiny",
31
- "best_epoch": 10,
32
  "note": "sarcasm_tau is the dev 2 % FPR threshold on the Platt-calibrated probability; tone temperature and laughter Platt curve fitted on dev"
33
  }
 
12
  "laughter",
13
  "arousal",
14
  "speech_rate",
15
+ "upset",
16
+ "sad"
17
  ],
18
+ "sarcasm_tau": 0.06329724596107583,
19
  "calibration": {
20
  "sarcasm": {
21
+ "a": 1.778761607691192,
22
+ "b": 0.2934051131149342
23
  },
24
  "laughter": {
25
+ "a": 0.9323411247145696,
26
+ "b": 0.17735405909388247
27
  },
28
+ "tone_temperature": 0.8733854440751974
29
  },
30
+ "run": "student-tiny-v4-sad",
31
  "init": "laion-tiny",
32
+ "best_epoch": 15,
33
  "note": "sarcasm_tau is the dev 2 % FPR threshold on the Platt-calibrated probability; tone temperature and laughter Platt curve fitted on dev"
34
  }
voice-reader-tiny/model.int8.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1823e7d7e861586f3981ffa470150a5c3653f5d017df97a49e606ccc4e6c6c42
3
- size 13332966
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31d06f9a42656a3144997efa1f085abe29bbcbf20722e5711ba34d86f430dcbf
3
+ size 14140364
voice-reader-tiny/model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:873b5e5991f607483331ca7078b175dadaee3a1ec2046507ae65f008db64cad0
3
- size 50202707
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad95fc8ae3370f98b91630bc4a12b71aec43d0d652f09b37dfed085e5238a833
3
+ size 53366574