Create README.md

README.md

@@ -0,0 +1,8 @@
+## Knowledge encoding by examples of Word2Vec and LLM training
+This repository contains weights for a list of language models:
+
+- word2vec.pt: embedding trained on 150mil pairs of text tokens subsampled from text8 dataset. SkipGram method with negative sampling was used as described in the original [paper](https://arxiv.org/abs/1402.3722). 
+- mlp.pt: 2-layers MLP trained on the same dataset and using pretrained embeddings.
+- mlp_norm.pt: Version of the MLP model utilizing LayerNorm for better scaling of the learned features distribution.
+
+[Training code can be found on GitHub](https://github.com/RuslanPeresy/knowledge-encode).