🇮🇹 BLOOM-IT
Collection
Models and datasets for BLOOM adapted to Italian. • 10 items • Updated
Model Card for Model ID
The model is obtained by performing language adaptation on the original bloom-1b7 model. In detail, we continued the pre-training on Italian-specific data without adaptation of the vocabulary. We use about 2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024). The model is trained for one epoch using LoRA and SFT.
Model Details
Model Description
- Developed by: SWAP Research Group, Department of Computer Science, University of Bari Aldo Moro
- Model type: BLOOM
- Language(s) (NLP): Italian
- License: bigscience-bloom-rail-1.0
- Finetuned from model [optional]: bloom-1b7
Training Details
Training Data
2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024).
Training Procedure
LoRA and SFT.
Training Hyperparameters
- Training regime: fp16
Citation [optional]
BibTeX:
APA:
Model Card Authors [optional]
Pierpaolo Basile, University of Bari Aldo Moro, Italy.
Model Card Contact
Pierpaolo Basile, University of Bari Aldo Moro, Italy.
- Downloads last month
- 6