Text-to-Audio
PEFT
Safetensors
audio-generation
lora
flow-matching
adhikjoshi commited on
Commit
90d9539
·
verified ·
1 Parent(s): 8df164e

Put the before/after pairs on the card

Browse files
.gitattributes CHANGED
@@ -168,3 +168,12 @@ samples/step-02500/ambience-rain.wav filter=lfs diff=lfs merge=lfs -text
168
  samples/step-02500/music-piano.wav filter=lfs diff=lfs merge=lfs -text
169
  samples/step-02500/sfx-gravel.wav filter=lfs diff=lfs merge=lfs -text
170
  samples/step-02500/speech-16w.wav filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
168
  samples/step-02500/music-piano.wav filter=lfs diff=lfs merge=lfs -text
169
  samples/step-02500/sfx-gravel.wav filter=lfs diff=lfs merge=lfs -text
170
  samples/step-02500/speech-16w.wav filter=lfs diff=lfs merge=lfs -text
171
+ samples/speech-4w__BASE.wav filter=lfs diff=lfs merge=lfs -text
172
+ samples/speech-16w__BASE.wav filter=lfs diff=lfs merge=lfs -text
173
+ samples/speech-16w__V8.wav filter=lfs diff=lfs merge=lfs -text
174
+ samples/sfx-gravel__BASE.wav filter=lfs diff=lfs merge=lfs -text
175
+ samples/sfx-gravel__V8.wav filter=lfs diff=lfs merge=lfs -text
176
+ samples/music-piano__BASE.wav filter=lfs diff=lfs merge=lfs -text
177
+ samples/music-piano__V8.wav filter=lfs diff=lfs merge=lfs -text
178
+ samples/ambience-rain__BASE.wav filter=lfs diff=lfs merge=lfs -text
179
+ samples/ambience-rain__V8.wav filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -70,6 +70,41 @@ Caption adherence or realism fell on: sfx clap_text -0.023, ambience clap_text -
70
 
71
  **Use this for speech.** For beds, effects and mixed scenes, A/B it against the base model on your own prompts first — the held-out flow loss improved on every capability, and on the scene arms that did not translate.
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ## Training
74
 
75
  | | |
 
70
 
71
  **Use this for speech.** For beds, effects and mixed scenes, A/B it against the base model on your own prompts first — the held-out flow loss improved on every capability, and on the scene arms that did not translate.
72
 
73
+ ## Hear it
74
+
75
+ Same prompt, same seed in both arms — `generate()` seeds the global RNG, so the
76
+ solver draws identical noise and every audible difference is the adapter.
77
+
78
+ **Speech, 4-word line — "The river remembers everything." The bucket that went 13.9% → 0.0% WER; the base arm garbles it.**
79
+
80
+ | base | scene-v8 |
81
+ |---|---|
82
+ | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-4w__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-4w__V8.wav"></audio> |
83
+
84
+ **Speech, 16-word line — both arms intelligible; the adapter is cleaner and tighter.**
85
+
86
+ | base | scene-v8 |
87
+ |---|---|
88
+ | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-16w__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/speech-16w__V8.wav"></audio> |
89
+
90
+ **SFX — footsteps on gravel. Texture and caption match improve; the adapter clip runs shorter, the one real remaining cost.**
91
+
92
+ | base | scene-v8 |
93
+ |---|---|
94
+ | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/sfx-gravel__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/sfx-gravel__V8.wav"></audio> |
95
+
96
+ **Music — slow solo piano. The adapter plays longer and closer to the caption at this seed.**
97
+
98
+ | base | scene-v8 |
99
+ |---|---|
100
+ | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/music-piano__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/music-piano__V8.wav"></audio> |
101
+
102
+ **Ambience — steady rain on a metal roof. Near-identical by design: this arm held.**
103
+
104
+ | base | scene-v8 |
105
+ |---|---|
106
+ | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/ambience-rain__BASE.wav"></audio> | <audio controls src="https://huggingface.co/ModelsLab/midashenglm-gen-wer-lora/resolve/main/samples/ambience-rain__V8.wav"></audio> |
107
+
108
  ## Training
109
 
110
  | | |
samples/ambience-rain__BASE.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae61bb154bc087beb5c58bbc7ef246ff2ddd5a23ee3927c57194a1741068ebe1
3
+ size 320044
samples/ambience-rain__V8.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f6f42f7ca14bbc6593026a5a08c3fad51d0540175de2ebe6d4fd53de65380c
3
+ size 313644
samples/music-piano__BASE.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc8fabcc15a58e73d92348450231418597eded29adda8c9f522cc9f63c2d8f92
3
+ size 307244
samples/music-piano__V8.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d65b28827d848b4e36d3973a1d5b2548b5daf4583917456d9dbb085f4be636f2
3
+ size 307244
samples/sfx-gravel__BASE.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ec540c812553ef2775bf11ffd098214e7ef8c4c317777d7d15128b819b3cedd
3
+ size 320044
samples/sfx-gravel__V8.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63efb0d640d4470509eb983e383861858888d0ecd3b9eabd4043367433aacc9a
3
+ size 179244
samples/speech-16w__BASE.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a7f2f5597d244f286f77f49c142803a50f02e7bb5794a9f7417aa67fb3afe58d
3
+ size 217644
samples/speech-16w__V8.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd9da236f3efdf5b589798c00aeadd3e86c263e82c1134c069ba7aca5237084a
3
+ size 140844
samples/speech-4w__BASE.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e996bf73eb251cc0d0bfb4fe64d6b2d6045a0e90ff6ddf212e1a274cd73c8f3
3
+ size 172844
samples/speech-4w__V8.wav ADDED
Binary file (51.2 kB). View file