Airin-chan commited on
Commit
89301f2
·
verified ·
1 Parent(s): f07b5f2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -3
README.md CHANGED
@@ -1,3 +1,37 @@
1
- ---
2
- license: cc-by-4.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ ---
4
+
5
+ LCT (Laten Connection Model)
6
+ -------------------------------
7
+ LCT is Transformers variant model that replaced Feed forward network with LCM , how its work ? LCM looking attention as upgraded signal by step1
8
+ step2 for looking attention. and then, LCM will do residural connection (attention (input), laten (output)) for making reached attention signal.
9
+
10
+ LCT look processing following:
11
+
12
+ embedding => LCT Block (Attention + LCM ) xN sub model => FFN as decoder => Linear
13
+
14
+ Here LCM ploating bechmark:
15
+ --------------------------------
16
+ ![sample](./LCT_bechmark.png)
17
+
18
+ LCT install:
19
+ --------------------
20
+ bash
21
+ ```
22
+ import LCT_architecture
23
+ model = keras.models.load(LCt-Tiny-version.keras)
24
+ ```
25
+
26
+ About LCM: https://zenodo.org/records/17501400?token=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6Ijk2ZmJmNDg3LWI3MTYtNDVlNy05OWEzLTRiOTZkNGFhOTkzMyIsImRhdGEiOnt9LCJyYW5kb20iOiJiZTNhZjBmMGJmN2NmN2EyNWYyMzRiZWI3MjJkMjcwZCJ9.1Tcsiz_aRDDHbR2MmUdf2MkcUPbyKsI88dRGsv1O3MpA-dxBMk7B4JiSvfwk0RKG9SBzV7WGHY3mnth_iEwhTg
27
+
28
+ support LCT: https://ko-fi.com/alpin92578
29
+
30
+ Author and Researcher LCT:
31
+ -------------------------
32
+ Candra Alpin gunawan
33
+
34
+ note:
35
+ ---------------
36
+ this model trained by 3K Conversations Dataset for ChatBot from kaggle user Kreesh Rajani
37
+ datasets link: https://www.kaggle.com/datasets/kreeshrajani/3k-conversations-dataset-for-chatbot