Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,37 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: cc-by-4.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
LCT (Laten Connection Model)
|
| 6 |
+
-------------------------------
|
| 7 |
+
LCT is Transformers variant model that replaced Feed forward network with LCM , how its work ? LCM looking attention as upgraded signal by step1
|
| 8 |
+
step2 for looking attention. and then, LCM will do residural connection (attention (input), laten (output)) for making reached attention signal.
|
| 9 |
+
|
| 10 |
+
LCT look processing following:
|
| 11 |
+
|
| 12 |
+
embedding => LCT Block (Attention + LCM ) xN sub model => FFN as decoder => Linear
|
| 13 |
+
|
| 14 |
+
Here LCM ploating bechmark:
|
| 15 |
+
--------------------------------
|
| 16 |
+

|
| 17 |
+
|
| 18 |
+
LCT install:
|
| 19 |
+
--------------------
|
| 20 |
+
bash
|
| 21 |
+
```
|
| 22 |
+
import LCT_architecture
|
| 23 |
+
model = keras.models.load(LCt-Tiny-version.keras)
|
| 24 |
+
```
|
| 25 |
+
|
| 26 |
+
About LCM: https://zenodo.org/records/17501400?token=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6Ijk2ZmJmNDg3LWI3MTYtNDVlNy05OWEzLTRiOTZkNGFhOTkzMyIsImRhdGEiOnt9LCJyYW5kb20iOiJiZTNhZjBmMGJmN2NmN2EyNWYyMzRiZWI3MjJkMjcwZCJ9.1Tcsiz_aRDDHbR2MmUdf2MkcUPbyKsI88dRGsv1O3MpA-dxBMk7B4JiSvfwk0RKG9SBzV7WGHY3mnth_iEwhTg
|
| 27 |
+
|
| 28 |
+
support LCT: https://ko-fi.com/alpin92578
|
| 29 |
+
|
| 30 |
+
Author and Researcher LCT:
|
| 31 |
+
-------------------------
|
| 32 |
+
Candra Alpin gunawan
|
| 33 |
+
|
| 34 |
+
note:
|
| 35 |
+
---------------
|
| 36 |
+
this model trained by 3K Conversations Dataset for ChatBot from kaggle user Kreesh Rajani
|
| 37 |
+
datasets link: https://www.kaggle.com/datasets/kreeshrajani/3k-conversations-dataset-for-chatbot
|