Upload Ejemplo de uso RoBERTa-k-MMG.ipynb

Ejemplo de uso RoBERTa-k-MMG.ipynb

@@ -0,0 +1,476 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "fb0e0d02",
+   "metadata": {},
+   "source": [
+    "## Carga del modelo"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2eb92ef8",
+   "metadata": {},
+   "source": [
+    "Vamos a ejecutar un ejemplo para ver como funciona el modelo seleccionado, RoBERTa-k-MMGb.\n",
+    "Importamos las librerias"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "5aeb9c06",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2023-08-29 23:55:16.877559: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import tensorflow as tf\n",
+    "from tensorflow import keras"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fd2d4359",
+   "metadata": {},
+   "source": [
+    "Ahora vamos a cargar el modelo con load_model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "f9f8acde",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "WARNING:tensorflow:`compile()` was not called as part of model loading because the model's `compile()` method is custom. All subclassed Models that have `compile()` overridden should also override `get_compile_config()` and `compile_from_config(config)`. Alternatively, you can call `compile()` manually after loading.\n"
+     ]
+    }
+   ],
+   "source": [
+    "model = tf.keras.models.load_model(\"./RoBERTa-k-MMG.keras\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "12dda010",
+   "metadata": {},
+   "source": [
+    "Comprobamos si se nos ha cargado el modelo con summary"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "6291f88c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Model: \"tf_roberta_for_sequence_classification_1\"\n",
+      "_________________________________________________________________\n",
+      " Layer (type)                Output Shape              Param #   \n",
+      "=================================================================\n",
+      " roberta (TFRobertaMainLaye  multiple                  124052736 \n",
+      " r)                                                              \n",
+      "                                                                 \n",
+      " classifier (TFRobertaClass  multiple                  592130    \n",
+      " ificationHead)                                                  \n",
+      "                                                                 \n",
+      "=================================================================\n",
+      "Total params: 124644866 (475.48 MB)\n",
+      "Trainable params: 124644866 (475.48 MB)\n",
+      "Non-trainable params: 0 (0.00 Byte)\n",
+      "_________________________________________________________________\n"
+     ]
+    }
+   ],
+   "source": [
+    "model.summary()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "50711238",
+   "metadata": {},
+   "source": [
+    "En los warning aparece que debemos compilar otra vez el modelo para utilizarlo, así que vamos a ello"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "03e2faa9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.compile(\n",
+    "optimizer=tf.keras.optimizers.Adam(learning_rate=5e-5),\n",
+    "loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
+    "metrics=tf.metrics.SparseCategoricalAccuracy(),\n",
+    ")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cc8aa427",
+   "metadata": {},
+   "source": [
+    "## Comprobación de sus métricas"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c2badbb3",
+   "metadata": {},
+   "source": [
+    "Ahora vamos a comprobar que efectivamente tiene el loss y el accuracy que se asegura, para ello voy a cargar el conjunto de test"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "cd2031f8",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'label': 1,\n",
+       " 'text': 'Le recorta la comida para que no se ponga gordo o suba de peso'}"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from datasets import load_dataset\n",
+    "data_file = {\"test\": \"test.csv\"}\n",
+    "dataset = load_dataset(\"../toxic-teenage-relationships\", data_files=data_file, sep=\";\")\n",
+    "dataset['test'][8]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "04e69690",
+   "metadata": {},
+   "source": [
+    "Cargamos el tokenizador y con .map se pasa al dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "541d8cd7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#en este ejemplo, utilizamos el AutoTokenizer general\n",
+    "from transformers import AutoTokenizer\n",
+    "\n",
+    "\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"PlanTL-GOB-ES/roberta-base-bne\")\n",
+    "\n",
+    "\n",
+    "def tokenize_function(examples):\n",
+    "    return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n",
+    "\n",
+    "#para tokenizar el conjunto de datos, usamos la función map que acelera la tokenización\n",
+    "tokenized_datasets = dataset.map(tokenize_function, batched=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fc088df2",
+   "metadata": {},
+   "source": [
+    "Ahora vamos a convertir el dataset en formator de TensorFlow. Para eso usamos DefaultDataCollator, que junta los tensores en un batch para que el modelo se entrene en él. Debemos especificar el argumento return_tensors=\"tf\".\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "0106a482",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from transformers import DefaultDataCollator\n",
+    "data_collator = DefaultDataCollator(return_tensors=\"tf\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "814f2aa5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n",
+    "eval_dataset = tokenized_datasets[\"test\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "02dd4812",
+   "metadata": {},
+   "source": [
+    "Ahora vamos a convertir los datasets tokenizados en datasets de TensorFlow con el método .to_tf_dataset. Las entradas están en columns y la etiqueta en label_cols. El bach size es el número de ejemplos que se introducen en la red para que se entrene cada vez"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "dc19171f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "tf_validation_dataset= eval_dataset.to_tf_dataset(\n",
+    "columns=[\"attention_mask\", \"input_ids\"],\n",
+    "label_cols=\"labels\",\n",
+    "shuffle=False,\n",
+    "collate_fn=data_collator,\n",
+    "batch_size=8,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a832ba0b",
+   "metadata": {},
+   "source": [
+    "Evaluamos el modelo y, efectivamente, aparecen las cifras que esperábamos"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "4d2bef7d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "('loss', 0.39804112911224365)\n",
+      "('sparse_categorical_accuracy', 0.9090909361839294)\n"
+     ]
+    }
+   ],
+   "source": [
+    "scores= model.evaluate(tf_validation_dataset, verbose=0)\n",
+    "print((model.metrics_names[0], scores[0]))\n",
+    "print((model.metrics_names[1], scores[1]))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "494bee3b",
+   "metadata": {},
+   "source": [
+    "## Predicciones"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "50182958",
+   "metadata": {},
+   "source": [
+    "En primer lugar, leemos el fichero que contiene las predicciones"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "0aaf73da",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with open('ejemplo.txt', 'r', encoding='utf-8') as file:\n",
+    "    lines = file.readlines()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "156713a2",
+   "metadata": {},
+   "source": [
+    "Ahora procedemos a hacer las tokenizar y hacer predicciones"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "7e878eb3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#tokenizamos los datos\n",
+    "encoded_data= tokenizer(lines, padding=\"max_length\", truncation=True, return_tensors=\"tf\")\n",
+    "#Obtenemos los input_ids, que son la matriz de enteros que representa los tokens tokenizados del texto de entrada\n",
+    "input_ids = encoded_data['input_ids']\n",
+    "#Obtenemos attention_mask, que es la matrizz que indica que tokens son reales (1) y cuales son relleno (0)\n",
+    "attention_mask = encoded_data['attention_mask']\n",
+    "    "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "03bb7d36",
+   "metadata": {},
+   "source": [
+    "Ahora vamos a realizar las prediciones, y para eso le pasamos estos dos componentes de los datos tokenizados"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "44e5b58d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "1/1 [==============================] - 25s 25s/step\n"
+     ]
+    }
+   ],
+   "source": [
+    "predictions = model.predict({'input_ids':input_ids, 'attention_mask':attention_mask})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c3b72027",
+   "metadata": {},
+   "source": [
+    "Obtenemos las probabilidades de haber obtenido una clase positiva (tóxica)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "5c195a22",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "probs_toxic = predictions.logits[:,1]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "111b7a90",
+   "metadata": {},
+   "source": [
+    "Ahora, para hacer la clasificación, vamos a escribir este umbral"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "13abeef0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "threshold=0.5\n",
+    "decoded_predictions = [1 if prob >= threshold else 0 for prob in probs_toxic]\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1ae09b99",
+   "metadata": {},
+   "source": [
+    "Finalmente, mostramos las predicciones obtenidas"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "c3ae29b8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "Texto: No se que piensas, pareces tonto.\n",
+      " Etiqueta de predicción: Tóxico\n",
+      " Probabilidad de Toxicidad: 2.6194\n",
+      "====================\n",
+      "\n",
+      "Texto: Maquillada así pareces una puta.\n",
+      " Etiqueta de predicción: Tóxico\n",
+      " Probabilidad de Toxicidad: 2.6675\n",
+      "====================\n",
+      "\n",
+      "Texto: Me encanta verte sonreir.\n",
+      " Etiqueta de predicción: Sano\n",
+      " Probabilidad de Toxicidad: -0.1724\n",
+      "====================\n",
+      "\n",
+      "Texto: Me deja de hablar tres días si hago algo que no le gusta.\n",
+      " Etiqueta de predicción: Tóxico\n",
+      " Probabilidad de Toxicidad: 2.4431\n",
+      "====================\n"
+     ]
+    }
+   ],
+   "source": [
+    "for line, prediction, prob_toxic in zip(lines, decoded_predictions, probs_toxic):\n",
+    "    label = \"Tóxico\" if prediction == 1 else \"Sano\"\n",
+    "    print()\n",
+    "    print(f\"Texto: {line.strip()}\")\n",
+    "    print(f\" Etiqueta de predicción: {label}\")\n",
+    "    print(f\" Probabilidad de Toxicidad: {prob_toxic:.4f}\")\n",
+    "    print(\"=\"*20)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f30dbd07",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}