Update README.md

README.md

@@ -11,7 +11,7 @@ library_name: transformers
 
 # OPT_GaMS 1B
 
-We proudly introduce the familly of GaMS (Generative Model for Slovene) models. The 1B version is based on [Facebook's OPT model](https://huggingface.co/facebook/opt-1.3b) and is adapted for Slovene.
+We proudly introduce the familly of GaMS (Generative Model for Slovene) models. The 1B version is based on [Facebook's OPT model](https://huggingface.co/facebook/opt-1.3b) and is adapted for Slovene. OPT_GaMS models use original OPT tokenizer.
 
 ## Acknowledgment
 
@@ -82,4 +82,26 @@ The total size of additional training data is **47.44 B** tokens.
 
 ### Training Procedure
 
-The model was trained using NeMo framework on Slovene HPC Vega, utilizing 64 A100 GPUs at once. Training took approximately 16 hours. The model was trained with batch size 1024 (2 million tokens) using Adam optimizer and cosine learning rate scheduler with 1000 warmup and constant steps.
+The model was trained using NeMo framework on Slovene HPC Vega, utilizing 64 A100 GPUs at once. Training took approximately 16 hours. The model was trained with batch size 1024 (2 million tokens) using Adam optimizer and cosine learning rate scheduler with 1000 warmup and constant steps.
+
+## Evaluation
+
+The model was evaluated using [Slovene SuperGLUE](https://slobench.cjvt.si/leaderboard/view/3) and [SI-NLI](https://slobench.cjvt.si/leaderboard/view/9) tasks on [SloBench](https://slobench.cjvt.si). Additionally, the models was evaluated on imporved version of Slovenian-LLM-eval introduced by Aleksa Gordić. All GaMS models were evaluated using few-shot prompts and were not finetuned on the benchmark (except for the two versions with finetuned in the name).
+
+### SuperGLUE results
+
+| Model | SuperGLUE Average | BoolQ Accuracy | CB Accuracy | CB F1 Score | CB Average |	COPA Accuracy |	MultiRC EM | MultiRC F1a Score | MultiRC Average | RTE Accuracy | WSC Accuracy |
+| :---- | :---------------: | :------------: | :---------: | :---------: | :--------: | :-----------: | :--------: | :---------------: | :-------------: | :----------: | :----------: |
+| OPT_GaMS-1B                | 0.4408     | 0.5667     | 0.5040     | 0.3885     | 0.4463     | 0.5020     | 0.0961     | 0.2543     | 0.1752     | 0.4138     | 0.5411     |
+| GaMS-1B                    | 0.4604     | 0.5000     | 0.6200     | 0.4565     | 0.5382     | 0.4920     | 0.1351     | 0.2675     | 0.2013     | 0.4828     | 0.5479     |
+| OPT_GaMS-1B-Chat           | 0.4165     | 0.7000     | 0.3720     | 0.2961     | 0.3341     | 0.4600     | 0.1111     | 0.3448     | 0.2280     | 0.4138     | 0.3630     |
+| GaMS-1B-Chat               | 0.4570     | **0.8000** | 0.4880     | 0.3023     | 0.3951     | 0.4840     | 0.1081     | 0.2428     | 0.1755     | 0.5172     | 0.3699     |
+| OPT_GaMS-1B-Chat finetuned | 0.5645     | 0.7000     | 0.8040     | 0.5884     | 0.6962     | 0.5860     | 0.1021     | 0.4808     | 0.2914     | 0.5862     | 0.5274     |
+| GaMS-1B-Chat finetuned     | 0.5806     | 0.7333     | **0.8120** | 0.5592     | 0.6856     | 0.5080     | 0.1381     | 0.4882     | 0.3132     | 0.5862     | **0.6575** |
+| SlovenianGPT-Chat*         | 0.5078     | 0.7333     | 0.3920     | 0.3829     | 0.3874     | **0.6840** | **0.2432** | 0.4944     | **0.3688** | 0.5172     | 0.3562     |
+| CroSloEngual BERT          | **0.6078** | 0.7333     | 0.7920     | **0.7437** | **0.7679** | 0.5720     | 0.0931     | **0.5241** | 0.3086     | **0.6552** | 0.6096     |
+*SlovenianGPT-Chat was obtained by instruction-tuning Aleksa Gordić's [SlovenianGPT](https://huggingface.co/gordicaleksa/SlovenianGPT) on our instruction dataset.
+
+### SI-NLI results
+
+### Slovenian-LLM-eval results