OldestSalt commited on
Commit
775ee45
·
verified ·
1 Parent(s): 846d6c7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -2
README.md CHANGED
@@ -17,11 +17,33 @@ language:
17
 
18
  Bimba is *almost* linear SimulMT model trained with wait-k policy (k = 3, 5, 7, 9, 11) on en-ru translation dataset.
19
 
 
 
20
  The model has encoder-decoder architecture, where self-attention blocks are Mamba-2 blocks instead. It means that encoder is linear, but cross-attention's input is all outputs of encoder, and this means that complexity of Bimba is O(S * T), which is not *exactly* linear
21
 
22
  ![Bimba inference](https://huggingface.co/OldestSalt/Bimba/resolve/main/hybrid.png)
23
 
24
- Bimba was developed as a part of master's thesis, and I hope that I can continue research in Linear SimulMT field.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
  - Code: https://github.com/OldestSalt/LinearSimultMT
27
- - Paper: Soon
 
17
 
18
  Bimba is *almost* linear SimulMT model trained with wait-k policy (k = 3, 5, 7, 9, 11) on en-ru translation dataset.
19
 
20
+ ## Architecture
21
+
22
  The model has encoder-decoder architecture, where self-attention blocks are Mamba-2 blocks instead. It means that encoder is linear, but cross-attention's input is all outputs of encoder, and this means that complexity of Bimba is O(S * T), which is not *exactly* linear
23
 
24
  ![Bimba inference](https://huggingface.co/OldestSalt/Bimba/resolve/main/hybrid.png)
25
 
26
+ Bimba was developed and trained as a part of master's thesis, and I hope that I will continue research in the Linear SimulMT field.
27
+
28
+ ## Using
29
+
30
+ To download Bimba you can clone GitHub repository and use the HybridMamba2MT class:
31
+
32
+ ```python
33
+ from model_classes import HybridMamba2MT
34
+ from transformers import AutoTokenizer
35
+
36
+ tokenizer = AutoTokenizer.from_pretrained("OldestSalt/Bimba")
37
+ model = HybridMamba2MT.from_pretrained("OldestSalt/Bimba")
38
+ ```
39
+
40
+ ### Translation
41
+
42
+ Maybe someday I will write here an example of simultaneous translation
43
+
44
+ ## Tokenizer
45
+
46
+ This model was distilled from [NLLB-200-1.3B](https://huggingface.co/facebook/nllb-200-1.3B), so Bimba uses its'
47
 
48
  - Code: https://github.com/OldestSalt/LinearSimultMT
49
+ - Paper: Soon (I hope)