Update README.md

README.md

@@ -7,4 +7,65 @@ language:
 - sr
 - bs
 library_name: transformers
----
+---
+
+# OPT_sl 1B
+
+This is the 1B OPT model additionally pretrained on Slovene data. The model was created as a part of project Povejmo: https://www.cjvt.si/povejmo/.
+
+This is the base version of the model and is not instruction-tuned.
+
+## Data
+
+The model was additionally pretrained on the following Slovene, English, and Croatian-Bosnian-Serbian (CBS) corpora:
+| Corpus | Language | # Tokens | Percentage |
+| :----- | :------- | :------: | :--------: |
+| Metafida | Slovene | 6.59 B | 13.89 % |
+| KAS | Slovene | 3.61 B | 7.62 % |
+| Trendi | Slovene | 1.4 B | 2.96 % |
+| mC4 | Slovene | 5.5 B | 11.6 % |
+| MaCoCu | Slovene | 4.68 B | 9.86 % |
+| CC100 | Slovene | 0.54 B | 1.14 % |
+| Rižnica | Croatian | 0.21 B | 0.44 % |
+| Hr News | Croatian | 4.16 B | 8.77 % |
+| MaCoCu HBS | CBS | 15.65 B | 32.98 % |
+| Wikipedia | English | 4.7 B | 9.9 % |
+| CC-News | English | 0.4 B | 0.83 % |
+
+The total size of additional training data is **47.44 B** tokens.
+
+## Model usage
+
+The inference can be done using the following snippet of code:
+
+```{python}
+from transformers import AutoTokenizer, pipeline
+
+tokenizer = AutoTokenizer.from_pretrained("cjvt/OPT_sl")
+
+pline = pipeline(
+    "text-generation",
+    model="cjvt/OPT_sl",
+    tokenizer=tokenizer,
+    device_map="auto"
+)
+
+prompts = [
+    "The examples of antonyms are:\nhigh => low\nwide => narrow\nbig =>",
+    "Pristanek je bil prvi nadzorovani spust ameriškega vesoljskega plovila na površje Lune po Apollu 17 leta 1972, ko je na Luni pristala zadnja Nasina misija s posadko.\nDoslej so na Luni pristala vesoljska plovila le iz štirih drugih držav –",
+    "U četvrtak je bila prva polufinalna večer Dore, a komentari na društvenim mrežama ne prestaju. U nedjeljno finale prošli su:"
+]
+
+sequences = pline(
+    prompts,
+    max_length=1000,
+    do_sample=False,
+    num_return_sequences=1,
+    eos_token_id=tokenizer.eos_token_id
+)
+
+for seq in sequences:
+    print("--------------------------")
+    print(f"Result: {seq[0]['generated_text']}")
+    print("--------------------------\n")
+```