add models

.gitattributes

@@ -0,0 +1,21 @@
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+
+*.gensim filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+

README.md

@@ -0,0 +1,119 @@
+---
+language: pl
+tags:
+- word2vec
+datasets:
+- KGR10
+---
+
+# KGR10 word2vec Polish word embeddings
+
+Distributional language models for Polish trained on the KGR10 corpora.
+
+## Models
+
+In the repository you can find two selected models, that were selected after evaluation (see table below). 
+A model that performed the best is the default model/config (see `default_config.json`). 
+
+|method|dimension|hs|mwe||
+|---|---|---|---| --- |
+|cbow|300|false|true| <-- default |
+|skipgram|300|true|true|
+
+
+## Usage
+
+To use these embedding models easily, it is required to install [embeddings](https://github.com/CLARIN-PL/embeddings).
+
+```bash
+pip install clarinpl-embeddings
+```
+
+### Utilising the default model (the easiest way)
+
+Word embedding:
+
+```python
+from embeddings.embedding.auto_flair import AutoFlairWordEmbedding
+from flair.data import Sentence
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+
+embedding = AutoFlairWordEmbedding.from_hub("clarin-pl/word2vec-kgr10")
+embedding.embed([sentence])
+
+for token in sentence:
+    print(token)
+    print(token.embedding)
+```
+
+Document embedding (averaged over words):
+
+```python
+from embeddings.embedding.auto_flair import AutoFlairDocumentEmbedding
+from flair.data import Sentence
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+
+embedding = AutoFlairDocumentEmbedding.from_hub("clarin-pl/word2vec-kgr10")
+embedding.embed([sentence])
+
+print(sentence.embedding)
+```
+
+### Customisable way
+
+Word embedding:
+
+```python
+from embeddings.embedding.static.embedding import AutoStaticWordEmbedding
+from embeddings.embedding.static.word2vec import KGR10Word2VecConfig
+from flair.data import Sentence
+
+config = KGR10Word2VecConfig(method='skipgram', hs=False)
+embedding = AutoStaticWordEmbedding.from_config(config)
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+embedding.embed([sentence])
+
+for token in sentence:
+    print(token)
+    print(token.embedding)
+```
+
+Document embedding (averaged over words):
+
+```python
+from embeddings.embedding.static.embedding import AutoStaticDocumentEmbedding
+from embeddings.embedding.static.word2vec import KGR10Word2VecConfig
+from flair.data import Sentence
+
+config = KGR10Word2VecConfig(method='skipgram', hs=False)
+embedding = AutoStaticDocumentEmbedding.from_config(config)
+
+sentence = Sentence("Myśl z duszy leci bystro, Nim się w słowach złamie.")
+embedding.embed([sentence])
+
+print(sentence.embedding)
+```
+
+## Citation
+
+```
+Piasecki, Maciej; Janz, Arkadiusz; Kaszewski, Dominik; et al., 2017,  Word Embeddings for Polish, CLARIN-PL digital repository, http://hdl.handle.net/11321/442.
+```
+
+or 
+
+
+```
+@misc{11321/442,	
+ title = {Word Embeddings for Polish},	
+ author = {Piasecki, Maciej and Janz, Arkadiusz and Kaszewski, Dominik and Czachor, Gabriela},	
+ url = {http://hdl.handle.net/11321/442},	
+ note = {{CLARIN}-{PL} digital repository},	
+ copyright = {{GNU} {GPL3}},	
+ year = {2017}	
+}
+```
+

cbow.v300.m8.hs.mwe.w2v.gensim

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7354e9b6ca9f5e5ab285a1e486ad94475d4d815cb3522cb0850d88aa6f9affcc
+size 138822242

cbow.v300.m8.hs.mwe.w2v.gensim.vectors.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c972cacbe38ce49a5e7430432290cf951f301740f2c49badc305e99c37e67f18
+size 2740052528

default_config.json

@@ -0,0 +1,6 @@
+{
+  "method": "cbow",
+  "dimension": 300,
+  "hs": true,
+  "mwe": true
+}

module.json

@@ -0,0 +1,3 @@
+{
+  "type": "embeddings.embedding.static.word2vec.KGR10Word2VecEmbedding"
+}

skipgram.v300.m8.ns.mwe.w2v.gensim

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7354e9b6ca9f5e5ab285a1e486ad94475d4d815cb3522cb0850d88aa6f9affcc
+size 138822242

skipgram.v300.m8.ns.mwe.w2v.gensim.vectors.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36da9ee479a7858f4b0a5a80ac3fe8298026e8edae0ea47036833fcf3eaec53f
+size 2740052528

test/dummy.model.gensim

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d162cf4354c63549da5b47745a17538ca869b69bfface2c020fb5bf782d2638
+size 28197