Update README.md

README.md

@@ -8,6 +8,28 @@ tags:
 
 # LookThem: Ratio-based Attention
 
+# Results
+## MNIST
+- 11 Epoch training, train accuracy: 99.02%
+- Test accuracy: 98.66%
+## CIFAR-10
+- 10 Epoch training, train accuracy: 67.89%
+- Test accuracy (10 epoch): 73.42%
+- 40 Epoch training, train accuracy: 76.63%
+- Test accuracy: 79.79%
+## Tiny-ImageNet
+- 15 Epoch training, train accuracy: 32.07%
+- Test accuracy (15 epoch): ? (the result are lying)
+- 30 Epoch training, train accuracy: 37.20%
+- Test accuracy: ? (the results are lying)
+- File size: 5.72MB
+## Tiny-ImageNet 2 (V5)
+- 20 Epoch training, train accuracy: 36.98%
+- Test accuracy: 34.2%
+- 40 Epoch training, train accuracy: 46.58%
+- Test accuracy: 35.46%
+- Code in second and third notebook
+
 # Explanation
 I was courious, what if a token look at other tokens, but without QKV? Instead, it's like make transformation for two tokens (current token and another token), then divide them. Let's say current token is token A and another token is token B. It's divide like "transformA(A) / transformB(B)" which the transform is a linear NN. With tanh for normalizing (to make it don't explode). And, the reverse ("transformA(B) / transformB(A)"). Then, the result of "transformA(A) / transformB(B)" multiply with A, and the reverse multiply with B. Then add them, then divide by 2. That's the new number for that interaction. Add to temp variable. Loop again for another token interaction (but for the code it's vectorized). Then, that variable averaged. That's the new A.
 
@@ -135,25 +157,7 @@ class LookThemLayer(nn.Module):
 
 ## Colab notebook in this repo
 
-# Results
-## MNIST
-- 11 Epoch training, train accuracy: 99.02%
-- Test accuracy: 98.66%
-## CIFAR-10
-- 10 Epoch training, train accuracy: 67.89%
-- Test accuracy (10 epoch): 73.42%
-- 40 Epoch training, train accuracy: 76.63%
-- Test accuracy: 79.79%
-## Tiny-ImageNet
-- 15 Epoch training, train accuracy: 32.07%
-- Test accuracy (15 epoch): ? (the result are lying)
-- 30 Epoch training, train accuracy: 37.20%
-- Test accuracy: ? (the results are lying)
-- File size: 5.72MB
-## Tiny-ImageNet 2 (V5)
-- 20 Epoch training, train accuracy: 36.98%
-- Test accuracy: 34.2%
-- Code in second notebook
+
 
 More detail in notebook