readme: add initial version

README.md

@@ -7,4 +7,25 @@ sdk: static
 pinned: false
 ---
 
-Edit this `README.md` markdown file to author your organization card.
+# German Wikipedia LMs (GWLMs)
+
+We present Language Models ([BERT](https://huggingface.co/gwlms/bert-base-dewiki-v1), [BERT with Token Dropping](https://huggingface.co/gwlms/bert-base-token-dropping-dewiki-v1), [TEAMS](https://huggingface.co/gwlms/teams-base-dewiki-v1-discriminator), [T5](https://huggingface.co/gwlms/t5-efficient-large-dewiki-v1)) pretrained on German Wikipedia.
+
+This is an ongoing project!
+
+# German Wikipedia Corpus
+
+We use a recent Wikipedia Dump, that can can be accessed [here](https://huggingface.co/datasets/gwlms/dewiki-20230701). Additionally, a sentence-segmented (using NLTK) is available [here](https://huggingface.co/datasets/gwlms/dewiki-20230701-nltk-corpus).
+
+# Fine-tuned Models
+
+We fine-tuned NER models using [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) library on GermEval 2014 NER dataset and upload the best models:
+
+* [GermEval 2014 NER Model with BERT as backbone LM](https://huggingface.co/gwlms/span-marker-bert-germeval14)
+* [GermEval 2014 NER with BERT + Token Dropping as backbone LM](https://huggingface.co/gwlms/span-marker-token-dropping-bert-germeval14)
+* [GermEval 2014 NER Model with TEAMS as backbone LM](https://huggingface.co/gwlms/span-marker-teams-germeval14)
+
+# Acknowledgements
+
+Research supported with Cloud TPUs from Google's [TPU Research Cloud](https://sites.research.google/trc/about/) (TRC).
+Many Thanks for providing access to the TPUs ❤️