Create README.md

README.md

@@ -0,0 +1,64 @@
+---
+license: apache-2.0
+language:
+- it
+- en
+---
+
+# Mistral-7B-v0.1-Italian-RANDOM
+<div align="center">
+
+<img src="https://github.com/Andrew-Wyn/images/blob/master/sava/italian_adapt-img.jpg?raw=true" width="400" height="400" style="border-radius:10%" />
+
+</div>
+
+The **Mistral-7B-v0.1-Adapted** collection of large language models (LLMs), is a collection of adapted generative models in 7B (text in/text out), adapted models from **Mistral-7B-Base-v0.1**.
+
+*Mistral-v0.1-Italian-RANDOM* is a continual trained mistral model, after tokenizer substitution.
+
+The tokenizer of this models after adaptation is the same of [Minverva-3B](https://huggingface.co/sapienzanlp/Minerva-3B-base-v1.0).
+
+**Model developer:** SapienzaNLP, ISTI-CNR, ILC-CNR
+
+**Model Architecture:** Mistral-7B-v0.1-Adapted is an auto-regressive language model that uses an optimized transformer architecture.
+
+## Data used for the adaptation
+
+The **Mistral-7B-v0.1-Adapted** model are trained on a collection of Italian and English data extracted from [CulturaX](https://huggingface.co/datasets/uonlp/CulturaX).
+The data are extracted to be skewed toward Italian language with a ration of one over four. Extracting the first 9B tokens from Italian part of CulturaX and the first 3B tokens from English part of CulturaX.
+
+
+## Use with Transformers
+
+You can run conversational inference using the Transformers pipeline abstraction or by leveraging the Auto classes with the generate() function.
+
+Make sure to update your transformers installation via pip install --upgrade transformers.
+
+```python
+import transformers
+import torch
+
+model_id = "SemanticAlignment/Mistral-v0.1-Italian-RANDOM"
+
+pipeline = transformers.pipeline(
+    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
+)
+
+pipeline("Cosa si può fare in una bella giornata di sole?")
+```
+
+## Citation
+
+If you use any part of this work, please consider citing the paper as follows:
+
+```bibtex
+@misc{moroni2025optimizingllmsitalianreducing,
+      title={Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation}, 
+      author={Luca Moroni and Giovanni Puccetti and Pere-Lluis Huguet Cabot and Andrei Stefan Bejgu and Edoardo Barba and Alessio Miaschi and Felice Dell'Orletta and Andrea Esuli and Roberto Navigli},
+      year={2025},
+      eprint={2504.17025},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2504.17025}, 
+}
+```