First commit

notebooks/heartfelt_model_training.ipynb

@@ -0,0 +1,611 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "94f179d1-d39e-4fcd-83da-a879c3aa641a",
+   "metadata": {},
+   "source": [
+    "# 1. Config"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "0e81b039-7320-47a8-8b18-7cdf03a5b0a5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import pandas as pd\n",
+    "\n",
+    "pd.set_option('display.max_colwidth', 500)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "fe74a27c-0ecf-499d-afaf-980fb80b463a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "131\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>text</th>\n",
+       "      <th>is_heartfelt</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>I would love for our school to be considered for some gardening equipment. We lost a member of staff in February who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening. Since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place. Mrs.Upham used to run a gardening club but she ...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>Our year 1 children have been inspired by a book called Omar, the Bees and Me, where two children from different backgrounds bond over their interest in bees and decide to make a 'bee corridor' between their school and the local park. They send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes, the whole neighborhood is alive with flowers and insects. Our year 1 classes would love to do something like this in our community. We...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>I am the SEN and Pastoral Lead in my school, Derryhale Primary School, Co Armagh N Ireland. We recently lost a valued member of our PTA team, a lovely mummy of two children, a little girl in Y2, and a little boy in Y4.  This lovely lady was just 31 years old and lost her fight for life to cancer. Budget is tight in school. We are a small rural school of 76 pupils and everyone has been effected by this loss. All the children in our care worry now that they will loose a family member and pasto...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  text  \\\n",
+       "0  I would love for our school to be considered for some gardening equipment. We lost a member of staff in February who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening. Since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place. Mrs.Upham used to run a gardening club but she ...   \n",
+       "1  Our year 1 children have been inspired by a book called Omar, the Bees and Me, where two children from different backgrounds bond over their interest in bees and decide to make a 'bee corridor' between their school and the local park. They send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes, the whole neighborhood is alive with flowers and insects. Our year 1 classes would love to do something like this in our community. We...   \n",
+       "2  I am the SEN and Pastoral Lead in my school, Derryhale Primary School, Co Armagh N Ireland. We recently lost a valued member of our PTA team, a lovely mummy of two children, a little girl in Y2, and a little boy in Y4.  This lovely lady was just 31 years old and lost her fight for life to cancer. Budget is tight in school. We are a small rural school of 76 pupils and everyone has been effected by this loss. All the children in our care worry now that they will loose a family member and pasto...   \n",
+       "\n",
+       "   is_heartfelt  \n",
+       "0          True  \n",
+       "1          True  \n",
+       "2          True  "
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "raw_df = pd.read_csv('data/exports/combined_heartfelt_data.csv')\n",
+    "\n",
+    "print(len(raw_df))\n",
+    "raw_df.head(3)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "44efbc86-827f-43a2-b9fa-046878e9243d",
+   "metadata": {},
+   "source": [
+    "# 2. Data Preprocessing"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c609ca9c-4fb5-4b8b-9734-712a4c67ce6c",
+   "metadata": {},
+   "source": [
+    "## 2.1 Normalising Text"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "6a0474b4-9a0c-4f27-8700-f9331c225210",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import string"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "6340d648-b2fb-4af2-8fcd-1247bf4c2f2f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "norm_text = raw_df.copy()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "ca4e4321-60fb-41a5-8dcd-28d5a96aead6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def normalise_text(text):\n",
+    "    if isinstance(text, str):\n",
+    "        text = text.lower()\n",
+    "        text = text.translate(str.maketrans('','', string.punctuation))\n",
+    "        text = text.strip()\n",
+    "\n",
+    "        return  ' '.join(text.split())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "07fe53af-0d0c-4e94-8bea-62a5d9967d48",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "norm_text['text'] = norm_text['text'].map(normalise_text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "7b6a2e48-6ef7-4e57-8bd4-fee08b7a993c",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>text</th>\n",
+       "      <th>is_heartfelt</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>i would love for our school to be considered for some gardening equipment we lost a member of staff in february who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place mrsupham used to run a gardening club but she dona...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>our year 1 children have been inspired by a book called omar the bees and me where two children from different backgrounds bond over their interest in bees and decide to make a bee corridor between their school and the local park they send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes the whole neighborhood is alive with flowers and insects our year 1 classes would love to do something like this in our community well need ...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>i am the sen and pastoral lead in my school derryhale primary school co armagh n ireland we recently lost a valued member of our pta team a lovely mummy of two children a little girl in y2 and a little boy in y4 this lovely lady was just 31 years old and lost her fight for life to cancer budget is tight in school we are a small rural school of 76 pupils and everyone has been effected by this loss all the children in our care worry now that they will loose a family member and pastorally i am ...</td>\n",
+       "      <td>True</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  text  \\\n",
+       "0  i would love for our school to be considered for some gardening equipment we lost a member of staff in february who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place mrsupham used to run a gardening club but she dona...   \n",
+       "1  our year 1 children have been inspired by a book called omar the bees and me where two children from different backgrounds bond over their interest in bees and decide to make a bee corridor between their school and the local park they send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes the whole neighborhood is alive with flowers and insects our year 1 classes would love to do something like this in our community well need ...   \n",
+       "2  i am the sen and pastoral lead in my school derryhale primary school co armagh n ireland we recently lost a valued member of our pta team a lovely mummy of two children a little girl in y2 and a little boy in y4 this lovely lady was just 31 years old and lost her fight for life to cancer budget is tight in school we are a small rural school of 76 pupils and everyone has been effected by this loss all the children in our care worry now that they will loose a family member and pastorally i am ...   \n",
+       "\n",
+       "   is_heartfelt  \n",
+       "0          True  \n",
+       "1          True  \n",
+       "2          True  "
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "norm_text.head(3)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ca01ec92-9289-4a52-b972-60ff58d48d34",
+   "metadata": {},
+   "source": [
+    "## 2.2 Stopword Removal"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "ae999021-1b3d-48b6-b34e-e9e3dc6d7e16",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import spacy\n",
+    "import spacy_cleaner\n",
+    "from spacy_cleaner.processing import removers, mutators\n",
+    "\n",
+    "nlp = spacy.load('en_core_web_md')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "afba3ebd-27b5-4171-84c7-155429242e98",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pipeline = spacy_cleaner.Cleaner(\n",
+    "    nlp,\n",
+    "    removers.remove_stopword_token,\n",
+    "    mutators.mutate_lemma_token,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "id": "72a40f5e-3b16-4522-b696-8a6aab553649",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def clean_text_with_pipeline(text):\n",
+    "    text = pipeline.clean(text)\n",
+    "\n",
+    "    return text"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "3aa72b37-6d39-49a6-9321-e6f8bcdf46a5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "preprocessed_df = norm_text.copy()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "326c9e80-a2c2-412e-b651-5424f1a0ba97",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Cleaning Progress: 100%|██████████████████████████████████████████| 131/131 [00:00<00:00, 313.08it/s]\n"
+     ]
+    }
+   ],
+   "source": [
+    "preprocessed_df['clean_text'] = pipeline.clean(preprocessed_df['text'].to_list())"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "900c578b-5a30-450f-8a9d-afd461fefb26",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>text</th>\n",
+       "      <th>is_heartfelt</th>\n",
+       "      <th>clean_text</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>i would love for our school to be considered for some gardening equipment we lost a member of staff in february who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place mrsupham used to run a gardening club but she dona...</td>\n",
+       "      <td>True</td>\n",
+       "      <td>love school consider gardening equipment lose member staff february school 30 year heavily involved teach child inner city school environment gardening death countless child ask use piece small ground playground use memory plant seed happy place mrsupham run gardening club donate thing 50 child volunteer help positive outcome sad loss</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>our year 1 children have been inspired by a book called omar the bees and me where two children from different backgrounds bond over their interest in bees and decide to make a bee corridor between their school and the local park they send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes the whole neighborhood is alive with flowers and insects our year 1 classes would love to do something like this in our community well need ...</td>\n",
+       "      <td>True</td>\n",
+       "      <td>year 1 child inspire book call omar bee child different background bond interest bee decide bee corridor school local park send envelope wildflower seed house building route time late spring come neighborhood alive flower insect year 1 class love like community need buy lot wildflower seed produce leaflet promote idea locally handdeliver envelope local area hope turn local area beefriendly community</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>i am the sen and pastoral lead in my school derryhale primary school co armagh n ireland we recently lost a valued member of our pta team a lovely mummy of two children a little girl in y2 and a little boy in y4 this lovely lady was just 31 years old and lost her fight for life to cancer budget is tight in school we are a small rural school of 76 pupils and everyone has been effected by this loss all the children in our care worry now that they will loose a family member and pastorally i am ...</td>\n",
+       "      <td>True</td>\n",
+       "      <td>sen pastoral lead school derryhale primary school co armagh n ireland recently lose value member pta team lovely mummy child little girl y2 little boy y4 lovely lady 31 year old lose fight life cancer budget tight school small rural school 76 pupil effect loss child care worry loose family member pastorally support child well resource twinkl support daddy granny child 10 day prior mummy die granda die little child lose beloved mummy granda 10 day love treat child school school trip christmas...</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  text  \\\n",
+       "0  i would love for our school to be considered for some gardening equipment we lost a member of staff in february who had been at our school for 30 years and she was heavily involved with teaching the children in our inner city school about the environment and gardening since her death we have had countless children asking if we can use a piece of small ground in the playground and use it to her memory and plant seeds and make it a happy place mrsupham used to run a gardening club but she dona...   \n",
+       "1  our year 1 children have been inspired by a book called omar the bees and me where two children from different backgrounds bond over their interest in bees and decide to make a bee corridor between their school and the local park they send out envelopes of wildflower seeds to every house and building along the route and by the time the late spring comes the whole neighborhood is alive with flowers and insects our year 1 classes would love to do something like this in our community well need ...   \n",
+       "2  i am the sen and pastoral lead in my school derryhale primary school co armagh n ireland we recently lost a valued member of our pta team a lovely mummy of two children a little girl in y2 and a little boy in y4 this lovely lady was just 31 years old and lost her fight for life to cancer budget is tight in school we are a small rural school of 76 pupils and everyone has been effected by this loss all the children in our care worry now that they will loose a family member and pastorally i am ...   \n",
+       "\n",
+       "   is_heartfelt  \\\n",
+       "0          True   \n",
+       "1          True   \n",
+       "2          True   \n",
+       "\n",
+       "                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            clean_text  \n",
+       "0                                                                                                                                                                     love school consider gardening equipment lose member staff february school 30 year heavily involved teach child inner city school environment gardening death countless child ask use piece small ground playground use memory plant seed happy place mrsupham run gardening club donate thing 50 child volunteer help positive outcome sad loss  \n",
+       "1                                                                                                   year 1 child inspire book call omar bee child different background bond interest bee decide bee corridor school local park send envelope wildflower seed house building route time late spring come neighborhood alive flower insect year 1 class love like community need buy lot wildflower seed produce leaflet promote idea locally handdeliver envelope local area hope turn local area beefriendly community  \n",
+       "2  sen pastoral lead school derryhale primary school co armagh n ireland recently lose value member pta team lovely mummy child little girl y2 little boy y4 lovely lady 31 year old lose fight life cancer budget tight school small rural school 76 pupil effect loss child care worry loose family member pastorally support child well resource twinkl support daddy granny child 10 day prior mummy die granda die little child lose beloved mummy granda 10 day love treat child school school trip christmas...  "
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "preprocessed_df.head(3)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "32bcaac6-1c7c-4c57-8686-7eb6bc6c8dad",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Index(['text', 'clean_text', 'is_heartfelt'], dtype='object')"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "preprocessed_df = preprocessed_df[['text', 'clean_text', 'is_heartfelt']]\n",
+    "\n",
+    "preprocessed_df.columns"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "407aad51-9680-4203-b86e-f8def0ca7731",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df = preprocessed_df.copy()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "id": "cb71ae69-af1c-4505-8f4a-4a13beb12d00",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "text            0\n",
+       "clean_text      0\n",
+       "is_heartfelt    0\n",
+       "dtype: int64"
+      ]
+     },
+     "execution_count": 18,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "df.isna().sum()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "39ed2ff3-c65c-42c4-9499-78c58d9ee442",
+   "metadata": {},
+   "source": [
+    "# 3. Model Training"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "id": "f2cf4443-b55b-4598-ac65-7176e8c55ef7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold\n",
+    "from sklearn.pipeline import Pipeline\n",
+    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
+    "from sklearn.linear_model import LogisticRegression\n",
+    "from sklearn.metrics import classification_report"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 44,
+   "id": "3f675a40-d095-49e6-9293-1b2f1b78c611",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "X, y = df['text'], df['is_heartfelt']\n",
+    "\n",
+    "# 2. Split out a hold-out set\n",
+    "X_train, X_test, y_train, y_test = train_test_split(\n",
+    "    X, y, stratify=y, test_size=0.2, random_state=42\n",
+    ")\n",
+    "\n",
+    "# 3. Build a TF–IDF + Logistic Regression pipeline\n",
+    "baseline_pipe = Pipeline([\n",
+    "    ('tfidf', TfidfVectorizer(\n",
+    "        lowercase=True,\n",
+    "        ngram_range=(1,2),\n",
+    "        min_df=2\n",
+    "    )),\n",
+    "    ('clf', LogisticRegression(\n",
+    "        solver='liblinear',\n",
+    "        class_weight='balanced',  # if classes are skewed\n",
+    "        random_state=42\n",
+    "    )),\n",
+    "])\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "id": "5bc1dffa-5aca-4b2e-9e85-df9cadbf96c9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Baseline CV F1: 0.916 ± 0.046\n",
+      "              precision    recall  f1-score   support\n",
+      "\n",
+      "       False       1.00      0.91      0.95        11\n",
+      "        True       0.94      1.00      0.97        16\n",
+      "\n",
+      "    accuracy                           0.96        27\n",
+      "   macro avg       0.97      0.95      0.96        27\n",
+      "weighted avg       0.97      0.96      0.96        27\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "# 4. Quick cross-validation\n",
+    "cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)\n",
+    "cv_scores = cross_val_score(\n",
+    "    baseline_pipe, X_train, y_train,\n",
+    "    cv=cv,\n",
+    "    scoring='f1'\n",
+    ")\n",
+    "print(f'Baseline CV F1: {cv_scores.mean():.3f} ± {cv_scores.std():.3f}')\n",
+    "\n",
+    "# 5. Fit and evaluate on held-out test\n",
+    "baseline_pipe.fit(X_train, y_train)\n",
+    "y_pred = baseline_pipe.predict(X_test)\n",
+    "print(classification_report(y_test, y_pred))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 46,
+   "id": "be9b112f-f0a1-4132-9554-c145175e8486",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "> Real=False, Pred=True\n",
+      "i would look to buy supplies for the whole school to use for science i know that some experiments use a lot of materials and would want to buy good kits for circuits i would also look into buying things to improve the learning and teaching of computing science in the school\n",
+      "---\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Example: inspect errors\n",
+    "X_err = X_test[y_test != y_pred]\n",
+    "y_err_true = y_test[y_test != y_pred]\n",
+    "y_err_pred = y_pred[y_test != y_pred]\n",
+    "for text, true, pred in zip(X_err, y_err_true, y_err_pred):\n",
+    "    print(f\"> Real={true}, Pred={pred}\\n{text}\\n---\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "79dd6ca7-4ffd-4350-a36b-48d51ce4c23a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import joblib\n",
+    "\n",
+    "baseline_pipe.fit(X_train, y_train)\n",
+    "\n",
+    "joblib.dump(baseline_pipe, '../src/models/heartfelt_pipeline.joblib')\n",
+    "\n",
+    "# …later, or in a new script/notebook, load it back:\n",
+    "#loaded_pipe = joblib.load('heartfelt_pipeline.joblib')\n",
+    "\n",
+    "# Confirm it still works\n",
+    "y_pred = loaded_pipe.predict(X_test)\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}