artitsu commited on
Commit
2f437be
·
1 Parent(s): 4fbca9f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md CHANGED
@@ -1,3 +1,72 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ language:
4
+ - th
5
+ metrics:
6
+ - cer
7
+ - wer
8
+ library_name: espnet
9
+ pipeline_tag: automatic-speech-recognition
10
  ---
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+ This is the baseline model of Pattani in [Thai-dialect corpus](https://github.com/SLSCU/thai-dialect-corpus).
16
+
17
+ The training recipe was based on wsj recipe in [espnet](https://github.com/espnet/espnet/).
18
+
19
+
20
+
21
+ ### Model Description
22
+
23
+ <!-- Provide a longer summary of what this model is. -->
24
+
25
+ This model is Hybrid CTC/Attention model with pre-trained HuBERT encoder.
26
+
27
+
28
+
29
+ ## Evaluation
30
+
31
+ <!-- This section describes the evaluation protocols and provides the results. -->
32
+
33
+ For evaluation, the metrics are CER and WER. before WER evaluation, transcriptions were re-tokenized using newmm tokenizer in [PyThaiNLP](https://github.com/PyThaiNLP/pythainlp)
34
+
35
+ In this reposirity, we also provide the vocabulary for building the newmm tokenizer using this script:
36
+
37
+ ```python
38
+ from pythainlp import Tokenizer
39
+
40
+ def get_tokenizer(vocab):
41
+
42
+ custom_vocab = set(vocab)
43
+ custom_tokenizer = Tokenizer(custom_vocab, engine='newmm')
44
+ return custom_tokenizer
45
+
46
+ with open(<vocab_path>,'r',encoding='utf-8') as f:
47
+ vocab = []
48
+ for line in f.readlines():
49
+ vocab.append(line.strip())
50
+
51
+ custom_tokenizer = get_tokenizer(vocab)
52
+
53
+ tokenized_sentence_list = custom_tokenizer.word_tokenize(<your_sentence>)
54
+ ```
55
+
56
+ |Micro CER|Macro CER|Survival CER|E-commerce WER|Micro WER|Macro WER|Survival WER|E-commerce WER|
57
+ |---|---|---|---|---|---|---|---|
58
+ |18.17|22.38|31.01|13.75|31.74|37.68|50.54|24.82|
59
+
60
+ ## Paper
61
+
62
+ [Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition](https://www.isca-speech.org/archive/pdfs/interspeech_2023/suwanbandit23_interspeech.pdf)
63
+ ```
64
+ @inproceedings{suwanbandit23_interspeech,
65
+ author={Artit Suwanbandit and Burin Naowarat and Orathai Sangpetch and Ekapol Chuangsuwanich},
66
+ title={{Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition}},
67
+ year=2023,
68
+ booktitle={Proc. INTERSPEECH 2023},
69
+ pages={4069--4073},
70
+ doi={10.21437/Interspeech.2023-1828}
71
+ }
72
+ ```