myaccountfor commited on
Commit
3b09382
·
1 Parent(s): f4a2b13

Update examples to show code-switching improvements

Browse files

- Replaced examples with cleaner code-switching demonstrations
- All examples now show both Chinese and English content
- Removed preamble-related example
- Examples show: translation prevention, technical term preservation, complex code-switching

Files changed (1) hide show
  1. README.md +25 -25
README.md CHANGED
@@ -40,45 +40,45 @@ A LoRA adapter for [Qwen/Qwen2-Audio-7B-Instruct](https://huggingface.co/Qwen/Qw
40
 
41
  Below are examples showing improvements from baseline to DPO-trained model:
42
 
43
- ### Example 1: Removed Unnecessary Translation
44
  | | Transcription |
45
  |---|---|
46
- | **Ground Truth** | how long she is been in the church |
47
- | **Baseline** | 她已经在这个教堂里待了五年多了。 *(Chinese translation)* |
48
- | **This Model** | how long she's been in the church. |
49
- | **MER** | 2.62 → **0.25** |
50
 
51
- ### Example 2: Code-Switching Preserved
52
  | | Transcription |
53
  |---|---|
54
- | **Ground Truth** | (err) you can choose primary school or secondary school |
55
- | **Baseline** | 呃你你可以选择公立学校或私立学校。 *(fully translated)* |
56
- | **This Model** | 呃你 you can choose primary school or secondary school |
57
- | **MER** | 2.20 → **0.10** |
58
 
59
- ### Example 3: Mixed Language Preserved
60
  | | Transcription |
61
  |---|---|
62
- | **Ground Truth** | which countries |
63
- | **Baseline** | 你去过哪个国家 *(fully translated to Chinese)* |
64
- | **This Model** | 你去过 which country |
65
- | **MER** | 2.40 → **0.40** |
66
 
67
- ### Example 4: Clean Output Without Preamble
68
  | | Transcription |
69
  |---|---|
70
- | **Ground Truth** | then you suggest english |
71
- | **Baseline** | The original content of this audio is: 'Then you suggest English? |
72
- | **This Model** | Then you suggest English. |
73
- | **MER** | 1.75 → **0.00** |
74
 
75
- ### Example 5: Code-Switching Session Preserved
76
  | | Transcription |
77
  |---|---|
78
- | **Ground Truth** | 你们 一首 也是 教 一个 session [啊] [哦] [嗯] |
79
- | **Baseline** | 你们是一首歌也是叫一个session啊哦我们就是来拍一次session的product然后... *(hallucinated extra content)* |
80
- | **This Model** | 你们是一首歌也是叫一个 session |
81
- | **MER** | 1.80 → **0.20** |
82
 
83
  ## Training Configuration
84
 
 
40
 
41
  Below are examples showing improvements from baseline to DPO-trained model:
42
 
43
+ ### Example 1: Code-Switching Preserved (Lifestyle)
44
  | | Transcription |
45
  |---|---|
46
+ | **Ground Truth** | 我们 应该 pursue a healthy lifestyle |
47
+ | **Baseline** | 我们都应该追求健康的生活方式 *(fully translated to Chinese)* |
48
+ | **This Model** | 我们都应该 pursue a healthy lifestyle |
49
+ | **MER** | 1.00 → **0.00** |
50
 
51
+ ### Example 2: Mixed Language Preserved (Christmas)
52
  | | Transcription |
53
  |---|---|
54
+ | **Ground Truth** | every christmas 应该 没有 跟我 庆祝 了 [啦] |
55
+ | **Baseline** | every christmas i would - should be no one to tell me *(Chinese translated to English)* |
56
+ | **This Model** | every christmas 我就应该是没有人跟我庆祝了啦 |
57
+ | **MER** | 0.88 → **0.00** |
58
 
59
+ ### Example 3: Technical Terms Preserved
60
  | | Transcription |
61
  |---|---|
62
+ | **Ground Truth** | (呃) 每个 lecture different lecturer 那个 notes 不 不怎么 好的 [啦] |
63
+ | **Baseline** | 呃那个老师不同风格的老师 *(lost technical terms)* |
64
+ | **This Model** | 每个 lecture different lecturer 那个 notes 不不怎么好的啦 |
65
+ | **MER** | 0.75 → **0.00** |
66
 
67
+ ### Example 4: Complex Code-Switching Preserved
68
  | | Transcription |
69
  |---|---|
70
+ | **Ground Truth** | [哦] 还有 什么 好吃 的 吗 还是 你 只是 去 那些 very expensive places like dempsey to eat |
71
+ | **Baseline** | Oh, what else? Oh, yeah, there's always that expensive place like... to eat *(lost Chinese content)* |
72
+ | **This Model** | 还有什么好吃的吗 还是你只是去那些 very expensive places like dancy to eat |
73
+ | **MER** | 0.83 → **0.04** |
74
 
75
+ ### Example 5: Professional Terms Preserved
76
  | | Transcription |
77
  |---|---|
78
+ | **Ground Truth** | [哦] 因为 是个 professional degree |
79
+ | **Baseline** | 哦因为他有个专业的学位 *(translated to Chinese)* |
80
+ | **This Model** | 因为 是个 professional degree |
81
+ | **MER** | 1.00 → **0.00** |
82
 
83
  ## Training Configuration
84