Instructions to use ModelsLab/midashenglm-gen-wer-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use ModelsLab/midashenglm-gen-wer-lora with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Put the before/after pairs on the card
Browse files- .gitattributes +9 -0
- README.md +35 -0
- samples/ambience-rain__BASE.wav +3 -0
- samples/ambience-rain__V8.wav +3 -0
- samples/music-piano__BASE.wav +3 -0
- samples/music-piano__V8.wav +3 -0
- samples/sfx-gravel__BASE.wav +3 -0
- samples/sfx-gravel__V8.wav +3 -0
- samples/speech-16w__BASE.wav +3 -0
- samples/speech-16w__V8.wav +3 -0
- samples/speech-4w__BASE.wav +3 -0
- samples/speech-4w__V8.wav +0 -0
.gitattributes
CHANGED
|
@@ -168,3 +168,12 @@ samples/step-02500/ambience-rain.wav filter=lfs diff=lfs merge=lfs -text
|
|
| 168 |
samples/step-02500/music-piano.wav filter=lfs diff=lfs merge=lfs -text
|
| 169 |
samples/step-02500/sfx-gravel.wav filter=lfs diff=lfs merge=lfs -text
|
| 170 |
samples/step-02500/speech-16w.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
samples/step-02500/music-piano.wav filter=lfs diff=lfs merge=lfs -text
|
| 169 |
samples/step-02500/sfx-gravel.wav filter=lfs diff=lfs merge=lfs -text
|
| 170 |
samples/step-02500/speech-16w.wav filter=lfs diff=lfs merge=lfs -text
|
| 171 |
+
samples/speech-4w__BASE.wav filter=lfs diff=lfs merge=lfs -text
|
| 172 |
+
samples/speech-16w__BASE.wav filter=lfs diff=lfs merge=lfs -text
|
| 173 |
+
samples/speech-16w__V8.wav filter=lfs diff=lfs merge=lfs -text
|
| 174 |
+
samples/sfx-gravel__BASE.wav filter=lfs diff=lfs merge=lfs -text
|
| 175 |
+
samples/sfx-gravel__V8.wav filter=lfs diff=lfs merge=lfs -text
|
| 176 |
+
samples/music-piano__BASE.wav filter=lfs diff=lfs merge=lfs -text
|
| 177 |
+
samples/music-piano__V8.wav filter=lfs diff=lfs merge=lfs -text
|
| 178 |
+
samples/ambience-rain__BASE.wav filter=lfs diff=lfs merge=lfs -text
|
| 179 |
+
samples/ambience-rain__V8.wav filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -70,6 +70,41 @@ Caption adherence or realism fell on: sfx clap_text -0.023, ambience clap_text -
|
|
| 70 |
|
| 71 |
**Use this for speech.** For beds, effects and mixed scenes, A/B it against the base model on your own prompts first — the held-out flow loss improved on every capability, and on the scene arms that did not translate.
|
| 72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
## Training
|
| 74 |
|
| 75 |
| | |
|
|
|
|
| 70 |
|
| 71 |
**Use this for speech.** For beds, effects and mixed scenes, A/B it against the base model on your own prompts first — the held-out flow loss improved on every capability, and on the scene arms that did not translate.
|
| 72 |
|
| 73 |
+
## Hear it
|
| 74 |
+
|
| 75 |
+
Same prompt, same seed in both arms — `generate()` seeds the global RNG, so the
|
| 76 |
+
solver draws identical noise and every audible difference is the adapter.
|
| 77 |
+
|
| 78 |
+
**Speech, 4-word line — "The river remembers everything." The bucket that went 13.9% → 0.0% WER; the base arm garbles it.**
|
| 79 |
+
|
| 80 |
+
| base | scene-v8 |
|
| 81 |
+
|---|---|
|
| 82 |
+
| <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-4w__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-4w__V8.wav"></audio> |
|
| 83 |
+
|
| 84 |
+
**Speech, 16-word line — both arms intelligible; the adapter is cleaner and tighter.**
|
| 85 |
+
|
| 86 |
+
| base | scene-v8 |
|
| 87 |
+
|---|---|
|
| 88 |
+
| <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-16w__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-16w__V8.wav"></audio> |
|
| 89 |
+
|
| 90 |
+
**SFX — footsteps on gravel. Texture and caption match improve; the adapter clip runs shorter, the one real remaining cost.**
|
| 91 |
+
|
| 92 |
+
| base | scene-v8 |
|
| 93 |
+
|---|---|
|
| 94 |
+
| <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/sfx-gravel__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/sfx-gravel__V8.wav"></audio> |
|
| 95 |
+
|
| 96 |
+
**Music — slow solo piano. The adapter plays longer and closer to the caption at this seed.**
|
| 97 |
+
|
| 98 |
+
| base | scene-v8 |
|
| 99 |
+
|---|---|
|
| 100 |
+
| <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/music-piano__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/music-piano__V8.wav"></audio> |
|
| 101 |
+
|
| 102 |
+
**Ambience — steady rain on a metal roof. Near-identical by design: this arm held.**
|
| 103 |
+
|
| 104 |
+
| base | scene-v8 |
|
| 105 |
+
|---|---|
|
| 106 |
+
| <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/ambience-rain__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/ambience-rain__V8.wav"></audio> |
|
| 107 |
+
|
| 108 |
## Training
|
| 109 |
|
| 110 |
| | |
|
samples/ambience-rain__BASE.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ae61bb154bc087beb5c58bbc7ef246ff2ddd5a23ee3927c57194a1741068ebe1
|
| 3 |
+
size 320044
|
samples/ambience-rain__V8.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95f6f42f7ca14bbc6593026a5a08c3fad51d0540175de2ebe6d4fd53de65380c
|
| 3 |
+
size 313644
|
samples/music-piano__BASE.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fc8fabcc15a58e73d92348450231418597eded29adda8c9f522cc9f63c2d8f92
|
| 3 |
+
size 307244
|
samples/music-piano__V8.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d65b28827d848b4e36d3973a1d5b2548b5daf4583917456d9dbb085f4be636f2
|
| 3 |
+
size 307244
|
samples/sfx-gravel__BASE.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ec540c812553ef2775bf11ffd098214e7ef8c4c317777d7d15128b819b3cedd
|
| 3 |
+
size 320044
|
samples/sfx-gravel__V8.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:63efb0d640d4470509eb983e383861858888d0ecd3b9eabd4043367433aacc9a
|
| 3 |
+
size 179244
|
samples/speech-16w__BASE.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a7f2f5597d244f286f77f49c142803a50f02e7bb5794a9f7417aa67fb3afe58d
|
| 3 |
+
size 217644
|
samples/speech-16w__V8.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bd9da236f3efdf5b589798c00aeadd3e86c263e82c1134c069ba7aca5237084a
|
| 3 |
+
size 140844
|
samples/speech-4w__BASE.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7e996bf73eb251cc0d0bfb4fe64d6b2d6045a0e90ff6ddf212e1a274cd73c8f3
|
| 3 |
+
size 172844
|
samples/speech-4w__V8.wav
ADDED
|
Binary file (51.2 kB). View file
|
|
|