Delete architecture-comparison-full-report.md with huggingface_hub
Browse files
architecture-comparison-full-report.md
DELETED
|
@@ -1,76 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
tags:
|
| 3 |
-
- turn-taking-detection
|
| 4 |
-
- audio-classification
|
| 5 |
-
- ecapa-tdnn
|
| 6 |
-
- whisper
|
| 7 |
-
- causal
|
| 8 |
-
license: mit
|
| 9 |
-
---
|
| 10 |
-
|
| 11 |
-
# Turn-Taking Detection Models — Collection complete v4
|
| 12 |
-
|
| 13 |
-
## Comparaison d'architectures acoustiques a budget de parametres variable (config AppTek seul, 3 accents)
|
| 14 |
-
|
| 15 |
-
Trois architectures testees a budget croissant (1x a 2x pour toutes ; Conformer aussi teste a 3x et 4x), en mode seul (acoustique isolee) et en mode fusion (avec Whisper gele).
|
| 16 |
-
|
| 17 |
-
### Moyennes sur 3 accents (en-CN, en-GB_WLS, en-IE)
|
| 18 |
-
|
| 19 |
-
| Architecture | Budget (params) | Seul Test | Seul OOD | Fusion Test | Fusion OOD |
|
| 20 |
-
|---|---|---|---|---|---|
|
| 21 |
-
| ECAPA-TDNN | 1x (462K) | 64.68% | 57.42% | 78.97% | 78.21% |
|
| 22 |
-
| ECAPA-TDNN | 2x (911K) | 64.51% | 57.46% | 78.05% | 77.40% |
|
| 23 |
-
| BiGRU | 1x (461K) | 70.19% | 65.50% | 75.70% | 74.48% |
|
| 24 |
-
| BiGRU | 2x (909K) | 70.54% | 65.76% | 76.61% | 74.04% |
|
| 25 |
-
| Conformer | 1x (430K) | 67.99% | 61.59% | 77.95% | 75.93% |
|
| 26 |
-
| **Conformer** | **2x (904K)** | **72.22%** | **68.93%** | **78.32%** | **77.78%** |
|
| 27 |
-
| Conformer | 3x (1.36M) | 70.74% | 70.09% | 77.18% | 77.04% |
|
| 28 |
-
| Conformer | 4x (1.83M) | 65.14%* | 63.13%* | 78.08% | 78.34% |
|
| 29 |
-
|
| 30 |
-
*Instabilite d'entrainement observee sur en-CN a 4x (effondrement isole), affectant la moyenne.
|
| 31 |
-
|
| 32 |
-
### Detail par accent
|
| 33 |
-
|
| 34 |
-
| Accent | Archi-Budget | Seul Test | Seul OOD | Fusion Test | Fusion OOD |
|
| 35 |
-
|---|---|---|---|---|---|
|
| 36 |
-
| en-CN | ECAPA 1x | 65.11% | 56.36% | 79.24% | 75.20% |
|
| 37 |
-
| en-CN | ECAPA 2x | 65.67% | 53.18% | 79.98% | 73.69% |
|
| 38 |
-
| en-CN | BiGRU 1x | 70.35% | 62.48% | 75.85% | 70.51% |
|
| 39 |
-
| en-CN | BiGRU 2x | 70.70% | 61.37% | 76.43% | 67.89% |
|
| 40 |
-
| en-CN | Conformer 1x | 70.18% | 55.64% | 78.71% | 71.70% |
|
| 41 |
-
| en-CN | Conformer 2x | 73.49% | 64.94% | 78.77% | 70.67% |
|
| 42 |
-
| en-CN | Conformer 3x | 71.75% | 65.90% | 75.75% | 70.11% |
|
| 43 |
-
| en-CN | Conformer 4x | 54.74% | 50.72% | 78.17% | 71.54% |
|
| 44 |
-
| en-GB_WLS | ECAPA 1x | 63.40% | 61.70% | 78.21% | 84.94% |
|
| 45 |
-
| en-GB_WLS | ECAPA 2x | 63.82% | 60.29% | 78.29% | 76.02% |
|
| 46 |
-
| en-GB_WLS | BiGRU 1x | 70.06% | 65.60% | 75.73% | 74.42% |
|
| 47 |
-
| en-GB_WLS | BiGRU 2x | 70.74% | 66.11% | 77.96% | 74.49% |
|
| 48 |
-
| en-GB_WLS | Conformer 1x | 68.48% | 65.98% | 77.54% | 74.17% |
|
| 49 |
-
| en-GB_WLS | Conformer 2x | 71.35% | 69.76% | 79.14% | 77.24% |
|
| 50 |
-
| en-GB_WLS | Conformer 3x | 71.78% | 69.57% | 78.08% | 76.60% |
|
| 51 |
-
| en-GB_WLS | Conformer 4x | 70.78% | 70.33% | 78.19% | 77.88% |
|
| 52 |
-
| en-IE | ECAPA 1x | 64.53% | 57.91% | 78.21% | 84.94% |
|
| 53 |
-
| en-IE | ECAPA 2x | 64.04% | 58.90% | 75.87% | 80.49% |
|
| 54 |
-
| en-IE | BiGRU 1x | 70.17% | 68.42% | 75.53% | 78.50% |
|
| 55 |
-
| en-IE | BiGRU 2x | 70.19% | 69.79% | 75.43% | 79.73% |
|
| 56 |
-
| en-IE | Conformer 1x | 65.36% | 63.16% | 77.60% | 81.91% |
|
| 57 |
-
| en-IE | Conformer 2x | 71.81% | 76.09% | 77.06% | 84.42% |
|
| 58 |
-
| en-IE | Conformer 3x | 70.68% | 74.81% | 77.72% | 84.42% |
|
| 59 |
-
| en-IE | Conformer 4x | 69.89% | 71.35% | 77.87% | 85.61% |
|
| 60 |
-
|
| 61 |
-
## Resultats cles de cette comparaison
|
| 62 |
-
|
| 63 |
-
1. **A budget egal (~460K), BiGRU domine en mode seul**, mais ECAPA-TDNN reprend l'avantage en mode fusion — le choix optimal depend du contexte d'usage.
|
| 64 |
-
|
| 65 |
-
2. **Conformer beneficie clairement d'un budget accru jusqu'a ~900K parametres (2x)**, avec un gain marque par rapport a 1x (+4 points test, +7 points OOD en mode seul) — suggerant qu'il etait sous-parametre a 430K.
|
| 66 |
-
|
| 67 |
-
3. **Au-dela de 2x, Conformer ne progresse plus de facon monotone** — le budget 3x reste proche du pic de 2x, et le budget 4x montre une degradation (possible instabilite d'entrainement / surapprentissage sur un jeu de donnees limite a ~17K exemples). Un optimum semble se situer autour de 900K parametres pour cette tache et ce volume de donnees.
|
| 68 |
-
|
| 69 |
-
4. **En mode fusion avec Whisper, la performance reste globalement stable a travers tous les budgets testes** (77-78% de test, quel que soit le budget de l'encodeur acoustique) — la fusion semble compenser les variations de qualite de l'encodeur seul, limitant l'impact du choix architectural une fois combine a la semantique.
|
| 70 |
-
|
| 71 |
-
## Fichiers de resultats bruts (JSON)
|
| 72 |
-
|
| 73 |
-
- `arch-comparison-{arch}-{accent}.json` — comparaison architectures, budget 1x
|
| 74 |
-
- `arch2x-comparison-{arch}-{accent}.json` — budget 2x
|
| 75 |
-
- `archscale-comparison-conformer-{3x,4x}-{accent}.json` — extension Conformer, budgets 3x et 4x
|
| 76 |
-
- `architecture-comparison-consolidated.json` — consolide 1x et 2x
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|