Silvina Ocampo - Modelo de Exploración

Model description

Este modelo es una versión fine-tuneada de DeepESP/gpt2-spanish, con un dataset de Autores y Autoras Latinoamericanos/as curado por Karen Palacio (https://github.com/karen-pal/borges), del cual seleccionamos a Silvina Ocampo. Fue creado durante un taller enfocado en exploración de LLMs por miembros de LAIA (laia.ar), siguiendo en grupo el Taller de Adaptación de Modelos de Lenguaje de Fundación Via Libre (https://github.com/nanom/llm_adaptation_workshop)

This model is a fine-tuned version of DeepESP/gpt2-spanish, with a dataset of Latin American authors by Karen Palacio (https://github.com/karen-pal/borges), of which Silvina Ocampo's work was chosen. It was created during a workshop focused on LLM exploration by members of LAIA (laia.ar), by group-following the LLM Adaptation Workshop by Fundación Via Libre (https://github.com/nanom/llm_adaptation_workshop)

It achieves the following results on the evaluation set:

• Loss: 2.2787

Intended uses & limitations

El modelo fue fine-tuneado como ejercicio educativo. This model was fine tuned as an educational exercise.

Training and evaluation data

• ver https://github.com/karen-pal/borges para los datasets. En este ejercicio, también agregamos un cuento no disponible en los datos originales.
• see the link above for the datasets

Training procedure

• ver https://github.com/nanom/llm_adaptation_workshop para el procedimiento de entrenamiento.
• see the link above for the training procedure.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
2.6172	1.0	58	2.4691
2.1861	2.0	116	2.3365
1.9253	3.0	174	2.2929
2.0581	4.0	232	2.2787

Framework versions

• Transformers 4.36.1
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0