Hugging Face's logo

Spaces:
ESCP
/
SE21AppTemplate
Running

App Files Community
126

Upload 5 files

#79
by Cossiepocci - opened
base: refs/heads/main
←
from: refs/pr/79
Discussion Files changed
+9894
-0
Files changed (5) hide show
  1. Big_Data_1 (1).ipynb +0 -0
  2. Hands_On_Activity_III .ipynb +0 -0
  3. Hands_on_activity_IV_(1).ipynb +1315 -0
  4. Hands_on_activity_IV_(2).ipynb +974 -0
  5. SE21_2526_Hands_On_Activity_II_(1) (1).ipynb +0 -0
Big_Data_1 (1).ipynb ADDED
The diff for this file is too large to render. See raw diff
 
Hands_On_Activity_III .ipynb ADDED
The diff for this file is too large to render. See raw diff
 
Hands_on_activity_IV_(1).ipynb ADDED
@@ -0,0 +1,1315 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "4ba6aba8"
7
+ },
8
+ "source": [
9
+ "# 🤖 **Data Collection, Creation, Storage, and Processing**\n"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "jpASMyIQMaAq"
16
+ },
17
+ "source": [
18
+ "## **1.** 📦 Install required packages"
19
+ ]
20
+ },
21
+ {
22
+ "cell_type": "code",
23
+ "execution_count": null,
24
+ "metadata": {
25
+ "colab": {
26
+ "base_uri": "https://localhost:8080/"
27
+ },
28
+ "id": "f48c8f8c",
29
+ "outputId": "56153411-4948-48ba-c6ba-6417f265d2a4"
30
+ },
31
+ "outputs": [
32
+ {
33
+ "output_type": "stream",
34
+ "name": "stdout",
35
+ "text": [
36
+ "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.12/dist-packages (4.13.5)\n",
37
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)\n",
38
+ "Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)\n",
39
+ "Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2)\n",
40
+ "Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)\n",
41
+ "Requirement already satisfied: textblob in /usr/local/lib/python3.12/dist-packages (0.19.0)\n",
42
+ "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (2.8.3)\n",
43
+ "Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (4.15.0)\n",
44
+ "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)\n",
45
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)\n",
46
+ "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.3)\n",
47
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)\n",
48
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)\n",
49
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.61.1)\n",
50
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.4.9)\n",
51
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.0)\n",
52
+ "Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)\n",
53
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)\n",
54
+ "Requirement already satisfied: nltk>=3.9 in /usr/local/lib/python3.12/dist-packages (from textblob) (3.9.1)\n",
55
+ "Requirement already satisfied: click in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (8.3.1)\n",
56
+ "Requirement already satisfied: joblib in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (1.5.3)\n",
57
+ "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (2025.11.3)\n",
58
+ "Requirement already satisfied: tqdm in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (4.67.3)\n",
59
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n"
60
+ ]
61
+ }
62
+ ],
63
+ "source": [
64
+ "!pip install beautifulsoup4 pandas matplotlib seaborn numpy textblob"
65
+ ]
66
+ },
67
+ {
68
+ "cell_type": "markdown",
69
+ "metadata": {
70
+ "id": "lquNYCbfL9IM"
71
+ },
72
+ "source": [
73
+ "## **2.** ⛏ Web-scrape all book titles, prices, and ratings from books.toscrape.com"
74
+ ]
75
+ },
76
+ {
77
+ "cell_type": "markdown",
78
+ "metadata": {
79
+ "id": "0IWuNpxxYDJF"
80
+ },
81
+ "source": [
82
+ "### *a. Initial setup*\n",
83
+ "Define the base url of the website you will scrape as well as how and what you will scrape"
84
+ ]
85
+ },
86
+ {
87
+ "cell_type": "code",
88
+ "execution_count": null,
89
+ "metadata": {
90
+ "id": "91d52125"
91
+ },
92
+ "outputs": [],
93
+ "source": [
94
+ "import requests\n",
95
+ "from bs4 import BeautifulSoup\n",
96
+ "import pandas as pd\n",
97
+ "import time\n",
98
+ "\n",
99
+ "base_url = \"https://books.toscrape.com/catalogue/page-{}.html\"\n",
100
+ "headers = {\"User-Agent\": \"Mozilla/5.0\"}\n",
101
+ "\n",
102
+ "titles, prices, ratings = [], [], []"
103
+ ]
104
+ },
105
+ {
106
+ "cell_type": "markdown",
107
+ "metadata": {
108
+ "id": "oCdTsin2Yfp3"
109
+ },
110
+ "source": [
111
+ "### *b. Fill titles, prices, and ratings from the web pages*"
112
+ ]
113
+ },
114
+ {
115
+ "cell_type": "code",
116
+ "execution_count": null,
117
+ "metadata": {
118
+ "id": "xqO5Y3dnYhxt"
119
+ },
120
+ "outputs": [],
121
+ "source": [
122
+ "# Loop through all 50 pages\n",
123
+ "for page in range(1, 51):\n",
124
+ " url = base_url.format(page)\n",
125
+ " response = requests.get(url, headers=headers)\n",
126
+ " soup = BeautifulSoup(response.content, \"html.parser\")\n",
127
+ " books = soup.find_all(\"article\", class_=\"product_pod\")\n",
128
+ "\n",
129
+ " for book in books:\n",
130
+ " titles.append(book.h3.a[\"title\"])\n",
131
+ " prices.append(float(book.find(\"p\", class_=\"price_color\").text[1:]))\n",
132
+ " ratings.append(book.p.get(\"class\")[1])\n",
133
+ "\n",
134
+ " time.sleep(0.5) # polite scraping delay"
135
+ ]
136
+ },
137
+ {
138
+ "cell_type": "markdown",
139
+ "metadata": {
140
+ "id": "T0TOeRC4Yrnn"
141
+ },
142
+ "source": [
143
+ "### *c. ✋🏻🛑⛔️ Create a dataframe df_books that contains the now complete \"title\", \"price\", and \"rating\" objects*"
144
+ ]
145
+ },
146
+ {
147
+ "cell_type": "code",
148
+ "execution_count": null,
149
+ "metadata": {
150
+ "id": "l5FkkNhUYTHh",
151
+ "colab": {
152
+ "base_uri": "https://localhost:8080/"
153
+ },
154
+ "outputId": "5c428d5a-fa82-4189-c9ed-a4d9be6445c1"
155
+ },
156
+ "outputs": [
157
+ {
158
+ "output_type": "stream",
159
+ "name": "stdout",
160
+ "text": [
161
+ " title price rating\n",
162
+ "0 A Light in the Attic 51.77 Three\n",
163
+ "1 Tipping the Velvet 53.74 One\n",
164
+ "2 Soumission 50.10 One\n",
165
+ "3 Sharp Objects 47.82 Four\n",
166
+ "4 Sapiens: A Brief History of Humankind 54.23 Five\n",
167
+ "<class 'pandas.core.frame.DataFrame'>\n",
168
+ "RangeIndex: 1000 entries, 0 to 999\n",
169
+ "Data columns (total 3 columns):\n",
170
+ " # Column Non-Null Count Dtype \n",
171
+ "--- ------ -------------- ----- \n",
172
+ " 0 title 1000 non-null object \n",
173
+ " 1 price 1000 non-null float64\n",
174
+ " 2 rating 1000 non-null object \n",
175
+ "dtypes: float64(1), object(2)\n",
176
+ "memory usage: 23.6+ KB\n",
177
+ "None\n"
178
+ ]
179
+ }
180
+ ],
181
+ "source": [
182
+ "# Create DataFrame\n",
183
+ "df_books = pd.DataFrame({\n",
184
+ " \"title\": titles,\n",
185
+ " \"price\": prices,\n",
186
+ " \"rating\": ratings\n",
187
+ "})\n",
188
+ "\n",
189
+ "# Optional: preview first rows\n",
190
+ "print(df_books.head())\n",
191
+ "\n",
192
+ "# Optional: check structure\n",
193
+ "print(df_books.info())"
194
+ ]
195
+ },
196
+ {
197
+ "cell_type": "markdown",
198
+ "metadata": {
199
+ "id": "duI5dv3CZYvF"
200
+ },
201
+ "source": [
202
+ "### *d. Save web-scraped dataframe either as a CSV or Excel file*"
203
+ ]
204
+ },
205
+ {
206
+ "cell_type": "code",
207
+ "execution_count": null,
208
+ "metadata": {
209
+ "id": "lC1U_YHtZifh"
210
+ },
211
+ "outputs": [],
212
+ "source": [
213
+ "# 💾 Save to CSV\n",
214
+ "df_books.to_csv(\"books_data.csv\", index=False)\n",
215
+ "\n",
216
+ "# 💾 Or save to Excel\n",
217
+ "# df_books.to_excel(\"books_data.xlsx\", index=False)"
218
+ ]
219
+ },
220
+ {
221
+ "cell_type": "markdown",
222
+ "metadata": {
223
+ "id": "qMjRKMBQZlJi"
224
+ },
225
+ "source": [
226
+ "### *e. ✋🏻🛑⛔️ View first fiew lines*"
227
+ ]
228
+ },
229
+ {
230
+ "cell_type": "code",
231
+ "source": [
232
+ "df_books.head()"
233
+ ],
234
+ "metadata": {
235
+ "colab": {
236
+ "base_uri": "https://localhost:8080/",
237
+ "height": 204
238
+ },
239
+ "id": "0SuwZbE5HXQv",
240
+ "outputId": "b875f030-f3af-48f9-8c85-a1ee40378db5"
241
+ },
242
+ "execution_count": null,
243
+ "outputs": [
244
+ {
245
+ "output_type": "execute_result",
246
+ "data": {
247
+ "text/plain": [
248
+ " title price rating\n",
249
+ "0 A Light in the Attic 51.77 Three\n",
250
+ "1 Tipping the Velvet 53.74 One\n",
251
+ "2 Soumission 50.10 One\n",
252
+ "3 Sharp Objects 47.82 Four\n",
253
+ "4 Sapiens: A Brief History of Humankind 54.23 Five"
254
+ ],
255
+ "text/html": [
256
+ "\n",
257
+ " <div id=\"df-bb633879-69c9-42e7-a10c-ef90383c5f18\" class=\"colab-df-container\">\n",
258
+ " <div>\n",
259
+ "<style scoped>\n",
260
+ " .dataframe tbody tr th:only-of-type {\n",
261
+ " vertical-align: middle;\n",
262
+ " }\n",
263
+ "\n",
264
+ " .dataframe tbody tr th {\n",
265
+ " vertical-align: top;\n",
266
+ " }\n",
267
+ "\n",
268
+ " .dataframe thead th {\n",
269
+ " text-align: right;\n",
270
+ " }\n",
271
+ "</style>\n",
272
+ "<table border=\"1\" class=\"dataframe\">\n",
273
+ " <thead>\n",
274
+ " <tr style=\"text-align: right;\">\n",
275
+ " <th></th>\n",
276
+ " <th>title</th>\n",
277
+ " <th>price</th>\n",
278
+ " <th>rating</th>\n",
279
+ " </tr>\n",
280
+ " </thead>\n",
281
+ " <tbody>\n",
282
+ " <tr>\n",
283
+ " <th>0</th>\n",
284
+ " <td>A Light in the Attic</td>\n",
285
+ " <td>51.77</td>\n",
286
+ " <td>Three</td>\n",
287
+ " </tr>\n",
288
+ " <tr>\n",
289
+ " <th>1</th>\n",
290
+ " <td>Tipping the Velvet</td>\n",
291
+ " <td>53.74</td>\n",
292
+ " <td>One</td>\n",
293
+ " </tr>\n",
294
+ " <tr>\n",
295
+ " <th>2</th>\n",
296
+ " <td>Soumission</td>\n",
297
+ " <td>50.10</td>\n",
298
+ " <td>One</td>\n",
299
+ " </tr>\n",
300
+ " <tr>\n",
301
+ " <th>3</th>\n",
302
+ " <td>Sharp Objects</td>\n",
303
+ " <td>47.82</td>\n",
304
+ " <td>Four</td>\n",
305
+ " </tr>\n",
306
+ " <tr>\n",
307
+ " <th>4</th>\n",
308
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
309
+ " <td>54.23</td>\n",
310
+ " <td>Five</td>\n",
311
+ " </tr>\n",
312
+ " </tbody>\n",
313
+ "</table>\n",
314
+ "</div>\n",
315
+ " <div class=\"colab-df-buttons\">\n",
316
+ "\n",
317
+ " <div class=\"colab-df-container\">\n",
318
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bb633879-69c9-42e7-a10c-ef90383c5f18')\"\n",
319
+ " title=\"Convert this dataframe to an interactive table.\"\n",
320
+ " style=\"display:none;\">\n",
321
+ "\n",
322
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
323
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
324
+ " </svg>\n",
325
+ " </button>\n",
326
+ "\n",
327
+ " <style>\n",
328
+ " .colab-df-container {\n",
329
+ " display:flex;\n",
330
+ " gap: 12px;\n",
331
+ " }\n",
332
+ "\n",
333
+ " .colab-df-convert {\n",
334
+ " background-color: #E8F0FE;\n",
335
+ " border: none;\n",
336
+ " border-radius: 50%;\n",
337
+ " cursor: pointer;\n",
338
+ " display: none;\n",
339
+ " fill: #1967D2;\n",
340
+ " height: 32px;\n",
341
+ " padding: 0 0 0 0;\n",
342
+ " width: 32px;\n",
343
+ " }\n",
344
+ "\n",
345
+ " .colab-df-convert:hover {\n",
346
+ " background-color: #E2EBFA;\n",
347
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
348
+ " fill: #174EA6;\n",
349
+ " }\n",
350
+ "\n",
351
+ " .colab-df-buttons div {\n",
352
+ " margin-bottom: 4px;\n",
353
+ " }\n",
354
+ "\n",
355
+ " [theme=dark] .colab-df-convert {\n",
356
+ " background-color: #3B4455;\n",
357
+ " fill: #D2E3FC;\n",
358
+ " }\n",
359
+ "\n",
360
+ " [theme=dark] .colab-df-convert:hover {\n",
361
+ " background-color: #434B5C;\n",
362
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
363
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
364
+ " fill: #FFFFFF;\n",
365
+ " }\n",
366
+ " </style>\n",
367
+ "\n",
368
+ " <script>\n",
369
+ " const buttonEl =\n",
370
+ " document.querySelector('#df-bb633879-69c9-42e7-a10c-ef90383c5f18 button.colab-df-convert');\n",
371
+ " buttonEl.style.display =\n",
372
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
373
+ "\n",
374
+ " async function convertToInteractive(key) {\n",
375
+ " const element = document.querySelector('#df-bb633879-69c9-42e7-a10c-ef90383c5f18');\n",
376
+ " const dataTable =\n",
377
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
378
+ " [key], {});\n",
379
+ " if (!dataTable) return;\n",
380
+ "\n",
381
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
382
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
383
+ " + ' to learn more about interactive tables.';\n",
384
+ " element.innerHTML = '';\n",
385
+ " dataTable['output_type'] = 'display_data';\n",
386
+ " await google.colab.output.renderOutput(dataTable, element);\n",
387
+ " const docLink = document.createElement('div');\n",
388
+ " docLink.innerHTML = docLinkHtml;\n",
389
+ " element.appendChild(docLink);\n",
390
+ " }\n",
391
+ " </script>\n",
392
+ " </div>\n",
393
+ "\n",
394
+ "\n",
395
+ " </div>\n",
396
+ " </div>\n"
397
+ ],
398
+ "application/vnd.google.colaboratory.intrinsic+json": {
399
+ "type": "dataframe",
400
+ "variable_name": "df_books",
401
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
402
+ }
403
+ },
404
+ "metadata": {},
405
+ "execution_count": 29
406
+ }
407
+ ]
408
+ },
409
+ {
410
+ "cell_type": "code",
411
+ "execution_count": null,
412
+ "metadata": {
413
+ "id": "O_wIvTxYZqCK"
414
+ },
415
+ "outputs": [],
416
+ "source": []
417
+ },
418
+ {
419
+ "cell_type": "markdown",
420
+ "metadata": {
421
+ "id": "p-1Pr2szaqLk"
422
+ },
423
+ "source": [
424
+ "## **3.** 🧩 Create a meaningful connection between real & synthetic datasets"
425
+ ]
426
+ },
427
+ {
428
+ "cell_type": "markdown",
429
+ "metadata": {
430
+ "id": "SIaJUGIpaH4V"
431
+ },
432
+ "source": [
433
+ "### *a. Initial setup*"
434
+ ]
435
+ },
436
+ {
437
+ "cell_type": "code",
438
+ "execution_count": null,
439
+ "metadata": {
440
+ "id": "-gPXGcRPuV_9"
441
+ },
442
+ "outputs": [],
443
+ "source": [
444
+ "import numpy as np\n",
445
+ "import random\n",
446
+ "from datetime import datetime\n",
447
+ "import warnings\n",
448
+ "\n",
449
+ "warnings.filterwarnings(\"ignore\")\n",
450
+ "random.seed(2025)\n",
451
+ "np.random.seed(2025)"
452
+ ]
453
+ },
454
+ {
455
+ "cell_type": "markdown",
456
+ "metadata": {
457
+ "id": "pY4yCoIuaQqp"
458
+ },
459
+ "source": [
460
+ "### *b. Generate popularity scores based on rating (with some randomness) with a generate_popularity_score function*"
461
+ ]
462
+ },
463
+ {
464
+ "cell_type": "code",
465
+ "execution_count": null,
466
+ "metadata": {
467
+ "id": "mnd5hdAbaNjz"
468
+ },
469
+ "outputs": [],
470
+ "source": [
471
+ "def compute_popularity(rating):\n",
472
+ " base = {\"One\": 2, \"Two\": 3, \"Three\": 3, \"Four\": 4, \"Five\": 4}.get(rating, 3)\n",
473
+ " trend_factor = random.choices([-1, 0, 1], weights=[1, 3, 2])[0]\n",
474
+ " return int(np.clip(base + trend_factor, 1, 5))"
475
+ ]
476
+ },
477
+ {
478
+ "cell_type": "markdown",
479
+ "metadata": {
480
+ "id": "n4-TaNTFgPak"
481
+ },
482
+ "source": [
483
+ "### *c. ✋🏻🛑⛔️ Run the function to create a \"popularity_score\" column from \"rating\"*"
484
+ ]
485
+ },
486
+ {
487
+ "cell_type": "code",
488
+ "execution_count": null,
489
+ "metadata": {
490
+ "id": "V-G3OCUCgR07",
491
+ "colab": {
492
+ "base_uri": "https://localhost:8080/"
493
+ },
494
+ "outputId": "6ef307ec-1a1d-483a-9116-920627b8580c"
495
+ },
496
+ "outputs": [
497
+ {
498
+ "output_type": "stream",
499
+ "name": "stdout",
500
+ "text": [
501
+ " title price rating popularity_index\n",
502
+ "0 A Light in the Attic 51.77 Three 3\n",
503
+ "1 Tipping the Velvet 53.74 One 2\n",
504
+ "2 Soumission 50.10 One 2\n",
505
+ "3 Sharp Objects 47.82 Four 4\n",
506
+ "4 Sapiens: A Brief History of Humankind 54.23 Five 3\n"
507
+ ]
508
+ }
509
+ ],
510
+ "source": [
511
+ "# Create compute_popularity column based on rating\n",
512
+ "df_books[\"popularity_index\"] = df_books[\"rating\"].apply(compute_popularity)\n",
513
+ "\n",
514
+ "# Preview results\n",
515
+ "print(df_books.head())"
516
+ ]
517
+ },
518
+ {
519
+ "cell_type": "markdown",
520
+ "metadata": {
521
+ "id": "HnngRNTgacYt"
522
+ },
523
+ "source": [
524
+ "### *d. Decide on the sentiment_label based on the popularity score with a get_sentiment function*"
525
+ ]
526
+ },
527
+ {
528
+ "cell_type": "code",
529
+ "execution_count": null,
530
+ "metadata": {
531
+ "id": "kUtWmr8maZLZ"
532
+ },
533
+ "outputs": [],
534
+ "source": [
535
+ "def get_sentiment(popularity_index):\n",
536
+ " if popularity_index <= 2:\n",
537
+ " return \"negative\"\n",
538
+ " elif popularity_index == 3:\n",
539
+ " return \"neutral\"\n",
540
+ " else:\n",
541
+ " return \"positive\""
542
+ ]
543
+ },
544
+ {
545
+ "cell_type": "markdown",
546
+ "metadata": {
547
+ "id": "HF9F9HIzgT7Z"
548
+ },
549
+ "source": [
550
+ "### *e. ✋🏻🛑⛔️ Run the function to create a \"sentiment_label\" column from \"popularity_score\"*"
551
+ ]
552
+ },
553
+ {
554
+ "cell_type": "markdown",
555
+ "metadata": {
556
+ "id": "T8AdKkmASq9a"
557
+ },
558
+ "source": [
559
+ "## **4.** 📈 Generate synthetic book sales data of 18 months"
560
+ ]
561
+ },
562
+ {
563
+ "cell_type": "markdown",
564
+ "metadata": {
565
+ "id": "OhXbdGD5fH0c"
566
+ },
567
+ "source": [
568
+ "### *a. Create a generate_sales_profit function that would generate sales patterns based on sentiment_label (with some randomness)*"
569
+ ]
570
+ },
571
+ {
572
+ "cell_type": "code",
573
+ "execution_count": null,
574
+ "metadata": {
575
+ "id": "qkVhYPXGbgEn"
576
+ },
577
+ "outputs": [],
578
+ "source": [
579
+ "def generate_sales_profile(sentiment):\n",
580
+ " months = pd.date_range(end=datetime.today(), periods=18, freq=\"M\")\n",
581
+ "\n",
582
+ " if sentiment == \"positive\":\n",
583
+ " base = random.randint(200, 300)\n",
584
+ " trend = np.linspace(base, base + random.randint(20, 60), len(months))\n",
585
+ " elif sentiment == \"negative\":\n",
586
+ " base = random.randint(20, 80)\n",
587
+ " trend = np.linspace(base, base - random.randint(10, 30), len(months))\n",
588
+ " else: # neutral\n",
589
+ " base = random.randint(80, 160)\n",
590
+ " trend = np.full(len(months), base + random.randint(-10, 10))\n",
591
+ "\n",
592
+ " seasonality = 10 * np.sin(np.linspace(0, 3 * np.pi, len(months)))\n",
593
+ " noise = np.random.normal(0, 5, len(months))\n",
594
+ " monthly_sales = np.clip(trend + seasonality + noise, a_min=0, a_max=None).astype(int)\n",
595
+ "\n",
596
+ " return list(zip(months.strftime(\"%Y-%m\"), monthly_sales))"
597
+ ]
598
+ },
599
+ {
600
+ "cell_type": "markdown",
601
+ "metadata": {
602
+ "id": "L2ak1HlcgoTe"
603
+ },
604
+ "source": [
605
+ "### *b. Run the function as part of building sales_data*"
606
+ ]
607
+ },
608
+ {
609
+ "cell_type": "code",
610
+ "execution_count": null,
611
+ "metadata": {
612
+ "id": "SlJ24AUafoDB",
613
+ "outputId": "e1897a50-e820-45de-ba5e-1e597b516c3d",
614
+ "colab": {
615
+ "base_uri": "https://localhost:8080/",
616
+ "height": 537
617
+ }
618
+ },
619
+ "outputs": [
620
+ {
621
+ "output_type": "error",
622
+ "ename": "KeyError",
623
+ "evalue": "'sentiment_label'",
624
+ "traceback": [
625
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
626
+ "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
627
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mget_loc\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 3804\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3805\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcasted_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3806\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
628
+ "\u001b[0;32mindex.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[0;34m()\u001b[0m\n",
629
+ "\u001b[0;32mindex.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[0;34m()\u001b[0m\n",
630
+ "\u001b[0;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[0;34m()\u001b[0m\n",
631
+ "\u001b[0;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[0;34m()\u001b[0m\n",
632
+ "\u001b[0;31mKeyError\u001b[0m: 'sentiment_label'",
633
+ "\nThe above exception was the direct cause of the following exception:\n",
634
+ "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
635
+ "\u001b[0;32m/tmp/ipykernel_143/1498594504.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0msales_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0m_\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrow\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdf_books\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miterrows\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mrecords\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgenerate_sales_profile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrow\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"sentiment_label\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mmonth\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munits\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrecords\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m sales_data.append({\n",
636
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/core/series.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 1119\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1120\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mkey_is_scalar\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1121\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1122\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1123\u001b[0m \u001b[0;31m# Convert generator to list before going through hashable part\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
637
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/core/series.py\u001b[0m in \u001b[0;36m_get_value\u001b[0;34m(self, label, takeable)\u001b[0m\n\u001b[1;32m 1235\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1236\u001b[0m \u001b[0;31m# Similar to Index.get_value, but we do not fall back to positional\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1237\u001b[0;31m \u001b[0mloc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1238\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1239\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mis_integer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
638
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mget_loc\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m 3810\u001b[0m ):\n\u001b[1;32m 3811\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mInvalidIndexError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3812\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 3813\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3814\u001b[0m \u001b[0;31m# If we have a listlike key, _check_indexing_error will raise\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
639
+ "\u001b[0;31mKeyError\u001b[0m: 'sentiment_label'"
640
+ ]
641
+ }
642
+ ],
643
+ "source": [
644
+ "sales_data = []\n",
645
+ "for _, row in df_books.iterrows():\n",
646
+ " records = generate_sales_profile(row[\"sentiment_label\"])\n",
647
+ " for month, units in records:\n",
648
+ " sales_data.append({\n",
649
+ " \"title\": row[\"title\"],\n",
650
+ " \"month\": month,\n",
651
+ " \"units_sold\": units,\n",
652
+ " \"sentiment_label\": row[\"sentiment_label\"]\n",
653
+ " })"
654
+ ]
655
+ },
656
+ {
657
+ "cell_type": "markdown",
658
+ "metadata": {
659
+ "id": "4IXZKcCSgxnq"
660
+ },
661
+ "source": [
662
+ "### *c. ✋🏻🛑⛔️ Create a df_sales DataFrame from sales_data*"
663
+ ]
664
+ },
665
+ {
666
+ "cell_type": "code",
667
+ "execution_count": null,
668
+ "metadata": {
669
+ "id": "wcN6gtiZg-ws",
670
+ "colab": {
671
+ "base_uri": "https://localhost:8080/"
672
+ },
673
+ "outputId": "f9c860d2-37ed-4dbf-931e-49b0af5ec4e6"
674
+ },
675
+ "outputs": [
676
+ {
677
+ "output_type": "stream",
678
+ "name": "stdout",
679
+ "text": [
680
+ "Empty DataFrame\n",
681
+ "Columns: []\n",
682
+ "Index: []\n",
683
+ "<class 'pandas.core.frame.DataFrame'>\n",
684
+ "RangeIndex: 0 entries\n",
685
+ "Empty DataFrame\n",
686
+ "None\n"
687
+ ]
688
+ }
689
+ ],
690
+ "source": [
691
+ "# Create DataFrame from sales_data list of dictionaries\n",
692
+ "df_sales = pd.DataFrame(sales_data)\n",
693
+ "\n",
694
+ "# Optional: preview structure\n",
695
+ "print(df_sales.head())\n",
696
+ "print(df_sales.info())"
697
+ ]
698
+ },
699
+ {
700
+ "cell_type": "markdown",
701
+ "metadata": {
702
+ "id": "EhIjz9WohAmZ"
703
+ },
704
+ "source": [
705
+ "### *d. Save df_sales as synthetic_sales_data.csv & view first few lines*"
706
+ ]
707
+ },
708
+ {
709
+ "cell_type": "code",
710
+ "execution_count": null,
711
+ "metadata": {
712
+ "id": "MzbZvLcAhGaH"
713
+ },
714
+ "outputs": [],
715
+ "source": [
716
+ "df_sales.to_csv(\"synthetic_sales_data.csv\", index=False)\n",
717
+ "\n",
718
+ "print(df_sales.head())"
719
+ ]
720
+ },
721
+ {
722
+ "cell_type": "markdown",
723
+ "metadata": {
724
+ "id": "7g9gqBgQMtJn"
725
+ },
726
+ "source": [
727
+ "## **5.** 🎯 Generate synthetic customer reviews"
728
+ ]
729
+ },
730
+ {
731
+ "cell_type": "markdown",
732
+ "metadata": {
733
+ "id": "Gi4y9M9KuDWx"
734
+ },
735
+ "source": [
736
+ "### *a. ✋🏻🛑⛔️ Ask ChatGPT to create a list of 50 distinct generic book review texts for the sentiment labels \"positive\", \"neutral\", and \"negative\" called synthetic_reviews_by_sentiment*"
737
+ ]
738
+ },
739
+ {
740
+ "cell_type": "code",
741
+ "execution_count": null,
742
+ "metadata": {
743
+ "id": "b3cd2a50"
744
+ },
745
+ "outputs": [],
746
+ "source": [
747
+ "synthetic_reviews_by_sentiment = {\n",
748
+ " \"positive\": [\n",
749
+ " \"A beautifully written and engaging story.\",\n",
750
+ " \"Brilliantly written with unforgettable characters.\",\n",
751
+ " \"An inspiring story that exceeded expectations.\",\n",
752
+ " \"Beautiful prose and a deeply satisfying conclusion.\",\n",
753
+ " \"A captivating journey from start to finish.\",\n",
754
+ " \"Thought-provoking and emotionally powerful.\",\n",
755
+ " \"An absolute pleasure to read.\",\n",
756
+ " \"The storytelling was immersive and masterful.\",\n",
757
+ " \"A standout book I would gladly recommend.\",\n",
758
+ " \"Richly layered and intelligently crafted.\",\n",
759
+ " \"A refreshing and uplifting narrative.\",\n",
760
+ " \"The pacing was perfect and engaging.\",\n",
761
+ " \"A book that truly delivers on its promise.\",\n",
762
+ " \"Exceptionally well developed characters.\",\n",
763
+ " \"An unforgettable literary experience.\",\n",
764
+ " \"Creative, bold, and beautifully executed.\",\n",
765
+ " \"A rewarding read with real depth.\",\n",
766
+ " \"Smart, heartfelt, and memorable.\",\n",
767
+ " \"An impressive achievement in storytelling.\",\n",
768
+ " \"Elegant writing and strong thematic presence.\",\n",
769
+ " \"A delightful surprise from beginning to end.\",\n",
770
+ " \"Powerful themes handled with great care.\",\n",
771
+ " \"An engaging and meaningful story.\",\n",
772
+ " \"A book I couldn’t put down.\",\n",
773
+ " \"Expertly structured and emotionally resonant.\",\n",
774
+ " \"A remarkable blend of plot and character.\",\n",
775
+ " \"Truly enjoyable and well worth the time.\",\n",
776
+ " \"Compelling narrative with satisfying development.\",\n",
777
+ " \"Deeply moving and thoughtfully written.\",\n",
778
+ " \"A polished and confident piece of work.\",\n",
779
+ " \"The author’s voice felt authentic and strong.\",\n",
780
+ " \"A wonderfully immersive experience.\",\n",
781
+ " \"Smart storytelling with emotional payoff.\",\n",
782
+ " \"An uplifting and energizing read.\",\n",
783
+ " \"Memorable scenes that linger long after reading.\",\n",
784
+ " \"A thoroughly enjoyable literary journey.\",\n",
785
+ " \"Engaging from the very first page.\",\n",
786
+ " \"Well balanced, well paced, and well written.\",\n",
787
+ " \"A satisfying and cohesive narrative.\",\n",
788
+ " \"Inventive and beautifully described.\",\n",
789
+ " \"A refreshing take on a familiar theme.\",\n",
790
+ " \"Emotionally intelligent and compelling.\",\n",
791
+ " \"A book that truly stands out.\",\n",
792
+ " \"Strong execution and captivating storytelling.\",\n",
793
+ " \"One of the most engaging books I’ve read recently.\",\n",
794
+ " \"Confident writing with clear direction.\",\n",
795
+ " \"An impressive and polished work.\",\n",
796
+ " \"Rewarding, thoughtful, and engaging.\",\n",
797
+ " \"A literary experience worth revisiting.\",\n",
798
+ " \"Highly recommended for anyone who enjoys quality fiction.\"\n",
799
+ " ],\n",
800
+ " \"neutral\": [\n",
801
+ " \"it was an average reading experience.\",\n",
802
+ " \"Some parts were interesting, others less so.\",\n",
803
+ " \"It was fine overall, though not particularly memorable.\",\n",
804
+ " \"A decent read with a few strong moments.\",\n",
805
+ " \"Neither disappointing nor outstanding.\",\n",
806
+ " \"An okay story with moderate engagement.\",\n",
807
+ " \"Fairly predictable but readable.\",\n",
808
+ " \"A balanced mix of strengths and weaknesses.\",\n",
809
+ " \"It held my attention at times.\",\n",
810
+ " \"Competently written but not remarkable.\",\n",
811
+ " \"A standard reading experience.\",\n",
812
+ " \"Some characters worked better than others.\",\n",
813
+ " \"The pacing felt uneven in places.\",\n",
814
+ " \"It had potential that wasn’t fully realized.\",\n",
815
+ " \"Readable but not especially impactful.\",\n",
816
+ " \"An acceptable way to spend a few hours.\",\n",
817
+ " \"It met basic expectations.\",\n",
818
+ " \"Some scenes were stronger than the overall arc.\",\n",
819
+ " \"A somewhat forgettable read.\",\n",
820
+ " \"Solid structure, modest execution.\",\n",
821
+ " \"Interesting premise but average delivery.\",\n",
822
+ " \"The writing was serviceable.\",\n",
823
+ " \"Mixed feelings after finishing.\",\n",
824
+ " \"It had its moments.\",\n",
825
+ " \"Neither impressed nor disappointed.\",\n",
826
+ " \"A fairly standard narrative.\",\n",
827
+ " \"Moderately engaging throughout.\",\n",
828
+ " \"Not bad, not exceptional.\",\n",
829
+ " \"An ordinary but competent book.\",\n",
830
+ " \"Reasonably enjoyable at times.\",\n",
831
+ " \"The plot was straightforward.\",\n",
832
+ " \"Some elements stood out positively.\",\n",
833
+ " \"A mild but acceptable read.\",\n",
834
+ " \"Average storytelling overall.\",\n",
835
+ " \"It did what it set out to do.\",\n",
836
+ " \"The characters were adequate.\",\n",
837
+ " \"Some sections dragged slightly.\",\n",
838
+ " \"A balanced but unremarkable experience.\",\n",
839
+ " \"Entertaining in parts.\",\n",
840
+ " \"Nothing particularly groundbreaking.\",\n",
841
+ " \"A steady but unspectacular book.\",\n",
842
+ " \"It felt somewhat formulaic.\",\n",
843
+ " \"Serviceable and easy to follow.\",\n",
844
+ " \"An inoffensive reading experience.\",\n",
845
+ " \"Fair execution of a common theme.\",\n",
846
+ " \"Moderate enjoyment throughout.\",\n",
847
+ " \"The narrative was simple and clear.\",\n",
848
+ " \"A conventional reading journey.\",\n",
849
+ " \"Neither highly engaging nor dull.\",\n",
850
+ " \"Acceptable, though not memorable.\"\n",
851
+ " ],\n",
852
+ " \"negative\": [\n",
853
+ " \"I struggled to stay interested.\",\n",
854
+ " \"The plot felt confusing at times.\",\n",
855
+ " \"Disappointing execution of an interesting idea.\",\n",
856
+ " \"The characters lacked depth.\",\n",
857
+ " \"Difficult to connect with the story.\",\n",
858
+ " \"The pacing dragged significantly.\",\n",
859
+ " \"Confusing structure and uneven tone.\",\n",
860
+ " \"Not as compelling as expected.\",\n",
861
+ " \"The narrative felt forced.\",\n",
862
+ " \"Underwhelming overall experience.\",\n",
863
+ " \"It failed to hold my interest.\",\n",
864
+ " \"Predictable and lacking originality.\",\n",
865
+ " \"The dialogue felt unnatural.\",\n",
866
+ " \"An unsatisfying conclusion.\",\n",
867
+ " \"The story lacked cohesion.\",\n",
868
+ " \"Hard to stay invested.\",\n",
869
+ " \"The themes were poorly developed.\",\n",
870
+ " \"It felt longer than necessary.\",\n",
871
+ " \"A frustrating reading experience.\",\n",
872
+ " \"Missed opportunities throughout.\",\n",
873
+ " \"Flat characters and weak development.\",\n",
874
+ " \"The writing felt uninspired.\",\n",
875
+ " \"The plot twists were unconvincing.\",\n",
876
+ " \"Difficult to recommend.\",\n",
877
+ " \"Overall, not particularly enjoyable.\",\n",
878
+ " \"The story never fully came together.\",\n",
879
+ " \"Poor pacing undermined the narrative.\",\n",
880
+ " \"An underdeveloped storyline.\",\n",
881
+ " \"Repetitive and predictable.\",\n",
882
+ " \"Lacked emotional impact.\",\n",
883
+ " \"The execution didn’t match the premise.\",\n",
884
+ " \"Struggled to finish this one.\",\n",
885
+ " \"The storytelling felt scattered.\",\n",
886
+ " \"Minimal character growth.\",\n",
887
+ " \"An unsatisfying literary effort.\",\n",
888
+ " \"Too many loose ends.\",\n",
889
+ " \"The narrative lacked direction.\",\n",
890
+ " \"Not engaging enough to sustain interest.\",\n",
891
+ " \"A forgettable and uneven read.\",\n",
892
+ " \"The tone felt inconsistent.\",\n",
893
+ " \"The book lacked originality.\",\n",
894
+ " \"An overly simplistic approach.\",\n",
895
+ " \"Weak character motivations.\",\n",
896
+ " \"The story felt rushed in parts.\",\n",
897
+ " \"Did not meet expectations.\",\n",
898
+ " \"Shallow and underdeveloped themes.\",\n",
899
+ " \"An uneven and frustrating experience.\",\n",
900
+ " \"The pacing was problematic.\",\n",
901
+ " \"Not worth revisiting.\",\n",
902
+ " \"Ultimately a disappointing read.\"\n",
903
+ " ]\n",
904
+ "}"
905
+ ]
906
+ },
907
+ {
908
+ "cell_type": "markdown",
909
+ "metadata": {
910
+ "id": "fQhfVaDmuULT"
911
+ },
912
+ "source": [
913
+ "### *b. Generate 10 reviews per book using random sampling from the corresponding 50*"
914
+ ]
915
+ },
916
+ {
917
+ "cell_type": "code",
918
+ "execution_count": null,
919
+ "metadata": {
920
+ "id": "l2SRc3PjuTGM"
921
+ },
922
+ "outputs": [],
923
+ "source": [
924
+ "review_rows = []\n",
925
+ "for _, row in df_books.iterrows():\n",
926
+ " title = row['title']\n",
927
+ " sentiment_label = row['sentiment_label']\n",
928
+ " review_pool = synthetic_reviews_by_sentiment[sentiment_label]\n",
929
+ " sampled_reviews = random.sample(review_pool, 10)\n",
930
+ " for review_text in sampled_reviews:\n",
931
+ " review_rows.append({\n",
932
+ " \"title\": title,\n",
933
+ " \"sentiment_label\": sentiment_label,\n",
934
+ " \"review_text\": review_text,\n",
935
+ " \"rating\": row['rating'],\n",
936
+ " \"popularity_score\": row['popularity_score']\n",
937
+ " })"
938
+ ]
939
+ },
940
+ {
941
+ "cell_type": "markdown",
942
+ "metadata": {
943
+ "id": "bmJMXF-Bukdm"
944
+ },
945
+ "source": [
946
+ "### *c. Create the final dataframe df_reviews & save it as synthetic_book_reviews.csv*"
947
+ ]
948
+ },
949
+ {
950
+ "cell_type": "code",
951
+ "execution_count": null,
952
+ "metadata": {
953
+ "id": "ZUKUqZsuumsp"
954
+ },
955
+ "outputs": [],
956
+ "source": [
957
+ "df_reviews = pd.DataFrame(review_rows)\n",
958
+ "df_reviews.to_csv(\"synthetic_book_reviews.csv\", index=False)"
959
+ ]
960
+ },
961
+ {
962
+ "cell_type": "markdown",
963
+ "source": [
964
+ "### *c. inputs for R*"
965
+ ],
966
+ "metadata": {
967
+ "id": "_602pYUS3gY5"
968
+ }
969
+ },
970
+ {
971
+ "cell_type": "code",
972
+ "execution_count": null,
973
+ "metadata": {
974
+ "colab": {
975
+ "base_uri": "https://localhost:8080/"
976
+ },
977
+ "id": "3946e521",
978
+ "outputId": "2e008a28-1f94-4efa-9511-5f2429e761c9"
979
+ },
980
+ "outputs": [
981
+ {
982
+ "output_type": "stream",
983
+ "name": "stdout",
984
+ "text": [
985
+ "✅ Wrote synthetic_title_level_features.csv\n",
986
+ "✅ Wrote synthetic_monthly_revenue_series.csv\n"
987
+ ]
988
+ }
989
+ ],
990
+ "source": [
991
+ "import numpy as np\n",
992
+ "\n",
993
+ "def _safe_num(s):\n",
994
+ " return pd.to_numeric(\n",
995
+ " pd.Series(s).astype(str).str.replace(r\"[^0-9.]\", \"\", regex=True),\n",
996
+ " errors=\"coerce\"\n",
997
+ " )\n",
998
+ "\n",
999
+ "# --- Clean book metadata (price/rating) ---\n",
1000
+ "df_books_r = df_books.copy()\n",
1001
+ "if \"price\" in df_books_r.columns:\n",
1002
+ " df_books_r[\"price\"] = _safe_num(df_books_r[\"price\"])\n",
1003
+ "if \"rating\" in df_books_r.columns:\n",
1004
+ " df_books_r[\"rating\"] = _safe_num(df_books_r[\"rating\"])\n",
1005
+ "\n",
1006
+ "df_books_r[\"title\"] = df_books_r[\"title\"].astype(str).str.strip()\n",
1007
+ "\n",
1008
+ "# --- Clean sales ---\n",
1009
+ "df_sales_r = df_sales.copy()\n",
1010
+ "df_sales_r[\"title\"] = df_sales_r[\"title\"].astype(str).str.strip()\n",
1011
+ "df_sales_r[\"month\"] = pd.to_datetime(df_sales_r[\"month\"], errors=\"coerce\")\n",
1012
+ "df_sales_r[\"units_sold\"] = _safe_num(df_sales_r[\"units_sold\"])\n",
1013
+ "\n",
1014
+ "# --- Clean reviews ---\n",
1015
+ "df_reviews_r = df_reviews.copy()\n",
1016
+ "df_reviews_r[\"title\"] = df_reviews_r[\"title\"].astype(str).str.strip()\n",
1017
+ "df_reviews_r[\"sentiment_label\"] = df_reviews_r[\"sentiment_label\"].astype(str).str.lower().str.strip()\n",
1018
+ "if \"rating\" in df_reviews_r.columns:\n",
1019
+ " df_reviews_r[\"rating\"] = _safe_num(df_reviews_r[\"rating\"])\n",
1020
+ "if \"popularity_score\" in df_reviews_r.columns:\n",
1021
+ " df_reviews_r[\"popularity_score\"] = _safe_num(df_reviews_r[\"popularity_score\"])\n",
1022
+ "\n",
1023
+ "# --- Sentiment shares per title (from reviews) ---\n",
1024
+ "sent_counts = (\n",
1025
+ " df_reviews_r.groupby([\"title\", \"sentiment_label\"])\n",
1026
+ " .size()\n",
1027
+ " .unstack(fill_value=0)\n",
1028
+ ")\n",
1029
+ "for lab in [\"positive\", \"neutral\", \"negative\"]:\n",
1030
+ " if lab not in sent_counts.columns:\n",
1031
+ " sent_counts[lab] = 0\n",
1032
+ "\n",
1033
+ "sent_counts[\"total_reviews\"] = sent_counts[[\"positive\", \"neutral\", \"negative\"]].sum(axis=1)\n",
1034
+ "den = sent_counts[\"total_reviews\"].replace(0, np.nan)\n",
1035
+ "sent_counts[\"share_positive\"] = sent_counts[\"positive\"] / den\n",
1036
+ "sent_counts[\"share_neutral\"] = sent_counts[\"neutral\"] / den\n",
1037
+ "sent_counts[\"share_negative\"] = sent_counts[\"negative\"] / den\n",
1038
+ "sent_counts = sent_counts.reset_index()\n",
1039
+ "\n",
1040
+ "# --- Sales aggregation per title ---\n",
1041
+ "sales_by_title = (\n",
1042
+ " df_sales_r.dropna(subset=[\"title\"])\n",
1043
+ " .groupby(\"title\", as_index=False)\n",
1044
+ " .agg(\n",
1045
+ " months_observed=(\"month\", \"nunique\"),\n",
1046
+ " avg_units_sold=(\"units_sold\", \"mean\"),\n",
1047
+ " total_units_sold=(\"units_sold\", \"sum\"),\n",
1048
+ " )\n",
1049
+ ")\n",
1050
+ "\n",
1051
+ "# --- Title-level features (join sales + books + sentiment) ---\n",
1052
+ "df_title = (\n",
1053
+ " sales_by_title\n",
1054
+ " .merge(df_books_r[[\"title\", \"price\", \"rating\"]], on=\"title\", how=\"left\")\n",
1055
+ " .merge(sent_counts[[\"title\", \"share_positive\", \"share_neutral\", \"share_negative\", \"total_reviews\"]],\n",
1056
+ " on=\"title\", how=\"left\")\n",
1057
+ ")\n",
1058
+ "\n",
1059
+ "df_title[\"avg_revenue\"] = df_title[\"avg_units_sold\"] * df_title[\"price\"]\n",
1060
+ "df_title[\"total_revenue\"] = df_title[\"total_units_sold\"] * df_title[\"price\"]\n",
1061
+ "\n",
1062
+ "df_title.to_csv(\"synthetic_title_level_features.csv\", index=False)\n",
1063
+ "print(\"✅ Wrote synthetic_title_level_features.csv\")\n",
1064
+ "\n",
1065
+ "# --- Monthly revenue series (proxy: units_sold * price) ---\n",
1066
+ "monthly_rev = (\n",
1067
+ " df_sales_r.merge(df_books_r[[\"title\", \"price\"]], on=\"title\", how=\"left\")\n",
1068
+ ")\n",
1069
+ "monthly_rev[\"revenue\"] = monthly_rev[\"units_sold\"] * monthly_rev[\"price\"]\n",
1070
+ "\n",
1071
+ "df_monthly = (\n",
1072
+ " monthly_rev.dropna(subset=[\"month\"])\n",
1073
+ " .groupby(\"month\", as_index=False)[\"revenue\"]\n",
1074
+ " .sum()\n",
1075
+ " .rename(columns={\"revenue\": \"total_revenue\"})\n",
1076
+ " .sort_values(\"month\")\n",
1077
+ ")\n",
1078
+ "# if revenue is all NA (e.g., missing price), fallback to units_sold as a teaching proxy\n",
1079
+ "if df_monthly[\"total_revenue\"].notna().sum() == 0:\n",
1080
+ " df_monthly = (\n",
1081
+ " df_sales_r.dropna(subset=[\"month\"])\n",
1082
+ " .groupby(\"month\", as_index=False)[\"units_sold\"]\n",
1083
+ " .sum()\n",
1084
+ " .rename(columns={\"units_sold\": \"total_revenue\"})\n",
1085
+ " .sort_values(\"month\")\n",
1086
+ " )\n",
1087
+ "\n",
1088
+ "df_monthly[\"month\"] = pd.to_datetime(df_monthly[\"month\"], errors=\"coerce\").dt.strftime(\"%Y-%m-%d\")\n",
1089
+ "df_monthly.to_csv(\"synthetic_monthly_revenue_series.csv\", index=False)\n",
1090
+ "print(\"✅ Wrote synthetic_monthly_revenue_series.csv\")\n"
1091
+ ]
1092
+ },
1093
+ {
1094
+ "cell_type": "markdown",
1095
+ "metadata": {
1096
+ "id": "RYvGyVfXuo54"
1097
+ },
1098
+ "source": [
1099
+ "### *d. ✋🏻🛑⛔️ View the first few lines*"
1100
+ ]
1101
+ },
1102
+ {
1103
+ "cell_type": "code",
1104
+ "execution_count": null,
1105
+ "metadata": {
1106
+ "colab": {
1107
+ "base_uri": "https://localhost:8080/"
1108
+ },
1109
+ "id": "xfE8NMqOurKo",
1110
+ "outputId": "c2ef3d67-1a2b-4979-aac4-422dd6bf3ba9"
1111
+ },
1112
+ "outputs": [
1113
+ {
1114
+ "output_type": "execute_result",
1115
+ "data": {
1116
+ "text/plain": [
1117
+ " title sentiment_label review_text \\\n",
1118
+ "0 A Light in the Attic neutral The narrative was simple and clear. \n",
1119
+ "1 A Light in the Attic neutral Mixed feelings after finishing. \n",
1120
+ "2 A Light in the Attic neutral It did what it set out to do. \n",
1121
+ "3 A Light in the Attic neutral It held my attention at times. \n",
1122
+ "4 A Light in the Attic neutral Fair execution of a common theme. \n",
1123
+ "\n",
1124
+ " rating popularity_score \n",
1125
+ "0 Three 3 \n",
1126
+ "1 Three 3 \n",
1127
+ "2 Three 3 \n",
1128
+ "3 Three 3 \n",
1129
+ "4 Three 3 "
1130
+ ],
1131
+ "text/html": [
1132
+ "\n",
1133
+ " <div id=\"df-755c6f10-1500-4337-bb94-13c1993252a5\" class=\"colab-df-container\">\n",
1134
+ " <div>\n",
1135
+ "<style scoped>\n",
1136
+ " .dataframe tbody tr th:only-of-type {\n",
1137
+ " vertical-align: middle;\n",
1138
+ " }\n",
1139
+ "\n",
1140
+ " .dataframe tbody tr th {\n",
1141
+ " vertical-align: top;\n",
1142
+ " }\n",
1143
+ "\n",
1144
+ " .dataframe thead th {\n",
1145
+ " text-align: right;\n",
1146
+ " }\n",
1147
+ "</style>\n",
1148
+ "<table border=\"1\" class=\"dataframe\">\n",
1149
+ " <thead>\n",
1150
+ " <tr style=\"text-align: right;\">\n",
1151
+ " <th></th>\n",
1152
+ " <th>title</th>\n",
1153
+ " <th>sentiment_label</th>\n",
1154
+ " <th>review_text</th>\n",
1155
+ " <th>rating</th>\n",
1156
+ " <th>popularity_score</th>\n",
1157
+ " </tr>\n",
1158
+ " </thead>\n",
1159
+ " <tbody>\n",
1160
+ " <tr>\n",
1161
+ " <th>0</th>\n",
1162
+ " <td>A Light in the Attic</td>\n",
1163
+ " <td>neutral</td>\n",
1164
+ " <td>The narrative was simple and clear.</td>\n",
1165
+ " <td>Three</td>\n",
1166
+ " <td>3</td>\n",
1167
+ " </tr>\n",
1168
+ " <tr>\n",
1169
+ " <th>1</th>\n",
1170
+ " <td>A Light in the Attic</td>\n",
1171
+ " <td>neutral</td>\n",
1172
+ " <td>Mixed feelings after finishing.</td>\n",
1173
+ " <td>Three</td>\n",
1174
+ " <td>3</td>\n",
1175
+ " </tr>\n",
1176
+ " <tr>\n",
1177
+ " <th>2</th>\n",
1178
+ " <td>A Light in the Attic</td>\n",
1179
+ " <td>neutral</td>\n",
1180
+ " <td>It did what it set out to do.</td>\n",
1181
+ " <td>Three</td>\n",
1182
+ " <td>3</td>\n",
1183
+ " </tr>\n",
1184
+ " <tr>\n",
1185
+ " <th>3</th>\n",
1186
+ " <td>A Light in the Attic</td>\n",
1187
+ " <td>neutral</td>\n",
1188
+ " <td>It held my attention at times.</td>\n",
1189
+ " <td>Three</td>\n",
1190
+ " <td>3</td>\n",
1191
+ " </tr>\n",
1192
+ " <tr>\n",
1193
+ " <th>4</th>\n",
1194
+ " <td>A Light in the Attic</td>\n",
1195
+ " <td>neutral</td>\n",
1196
+ " <td>Fair execution of a common theme.</td>\n",
1197
+ " <td>Three</td>\n",
1198
+ " <td>3</td>\n",
1199
+ " </tr>\n",
1200
+ " </tbody>\n",
1201
+ "</table>\n",
1202
+ "</div>\n",
1203
+ " <div class=\"colab-df-buttons\">\n",
1204
+ "\n",
1205
+ " <div class=\"colab-df-container\">\n",
1206
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-755c6f10-1500-4337-bb94-13c1993252a5')\"\n",
1207
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1208
+ " style=\"display:none;\">\n",
1209
+ "\n",
1210
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1211
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1212
+ " </svg>\n",
1213
+ " </button>\n",
1214
+ "\n",
1215
+ " <style>\n",
1216
+ " .colab-df-container {\n",
1217
+ " display:flex;\n",
1218
+ " gap: 12px;\n",
1219
+ " }\n",
1220
+ "\n",
1221
+ " .colab-df-convert {\n",
1222
+ " background-color: #E8F0FE;\n",
1223
+ " border: none;\n",
1224
+ " border-radius: 50%;\n",
1225
+ " cursor: pointer;\n",
1226
+ " display: none;\n",
1227
+ " fill: #1967D2;\n",
1228
+ " height: 32px;\n",
1229
+ " padding: 0 0 0 0;\n",
1230
+ " width: 32px;\n",
1231
+ " }\n",
1232
+ "\n",
1233
+ " .colab-df-convert:hover {\n",
1234
+ " background-color: #E2EBFA;\n",
1235
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1236
+ " fill: #174EA6;\n",
1237
+ " }\n",
1238
+ "\n",
1239
+ " .colab-df-buttons div {\n",
1240
+ " margin-bottom: 4px;\n",
1241
+ " }\n",
1242
+ "\n",
1243
+ " [theme=dark] .colab-df-convert {\n",
1244
+ " background-color: #3B4455;\n",
1245
+ " fill: #D2E3FC;\n",
1246
+ " }\n",
1247
+ "\n",
1248
+ " [theme=dark] .colab-df-convert:hover {\n",
1249
+ " background-color: #434B5C;\n",
1250
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1251
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1252
+ " fill: #FFFFFF;\n",
1253
+ " }\n",
1254
+ " </style>\n",
1255
+ "\n",
1256
+ " <script>\n",
1257
+ " const buttonEl =\n",
1258
+ " document.querySelector('#df-755c6f10-1500-4337-bb94-13c1993252a5 button.colab-df-convert');\n",
1259
+ " buttonEl.style.display =\n",
1260
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1261
+ "\n",
1262
+ " async function convertToInteractive(key) {\n",
1263
+ " const element = document.querySelector('#df-755c6f10-1500-4337-bb94-13c1993252a5');\n",
1264
+ " const dataTable =\n",
1265
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1266
+ " [key], {});\n",
1267
+ " if (!dataTable) return;\n",
1268
+ "\n",
1269
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1270
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1271
+ " + ' to learn more about interactive tables.';\n",
1272
+ " element.innerHTML = '';\n",
1273
+ " dataTable['output_type'] = 'display_data';\n",
1274
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1275
+ " const docLink = document.createElement('div');\n",
1276
+ " docLink.innerHTML = docLinkHtml;\n",
1277
+ " element.appendChild(docLink);\n",
1278
+ " }\n",
1279
+ " </script>\n",
1280
+ " </div>\n",
1281
+ "\n",
1282
+ "\n",
1283
+ " </div>\n",
1284
+ " </div>\n"
1285
+ ],
1286
+ "application/vnd.google.colaboratory.intrinsic+json": {
1287
+ "type": "dataframe",
1288
+ "variable_name": "df_reviews",
1289
+ "summary": "{\n \"name\": \"df_reviews\",\n \"rows\": 10000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"review_text\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 150,\n \"samples\": [\n \"A delightful surprise from beginning to end.\",\n \"The story lacked cohesion.\",\n \"The author\\u2019s voice felt authentic and strong.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1290
+ }
1291
+ },
1292
+ "metadata": {},
1293
+ "execution_count": 20
1294
+ }
1295
+ ],
1296
+ "source": [
1297
+ "df_reviews.head()"
1298
+ ]
1299
+ }
1300
+ ],
1301
+ "metadata": {
1302
+ "colab": {
1303
+ "provenance": []
1304
+ },
1305
+ "kernelspec": {
1306
+ "display_name": "Python 3",
1307
+ "name": "python3"
1308
+ },
1309
+ "language_info": {
1310
+ "name": "python"
1311
+ }
1312
+ },
1313
+ "nbformat": 4,
1314
+ "nbformat_minor": 0
1315
+ }
Hands_on_activity_IV_(2).ipynb ADDED
@@ -0,0 +1,974 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "kz8lLSv6mVQo"
7
+ },
8
+ "source": [
9
+ "# **🤖 Data Analysis & Visualization**"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "jpASMyIQMaAq"
16
+ },
17
+ "source": [
18
+ "## **1.** 📦 Install required packages"
19
+ ]
20
+ },
21
+ {
22
+ "cell_type": "code",
23
+ "execution_count": null,
24
+ "metadata": {
25
+ "colab": {
26
+ "base_uri": "https://localhost:8080/"
27
+ },
28
+ "id": "f48c8f8c",
29
+ "outputId": "31fb8283-7f35-4fb0-f270-7c040da95ed4",
30
+ "collapsed": true
31
+ },
32
+ "outputs": [
33
+ {
34
+ "output_type": "stream",
35
+ "name": "stdout",
36
+ "text": [
37
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)\n",
38
+ "Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)\n",
39
+ "Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2)\n",
40
+ "Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)\n",
41
+ "Requirement already satisfied: textblob in /usr/local/lib/python3.12/dist-packages (0.19.0)\n",
42
+ "Collecting faker\n",
43
+ " Downloading faker-40.8.0-py3-none-any.whl.metadata (16 kB)\n",
44
+ "Requirement already satisfied: transformers in /usr/local/lib/python3.12/dist-packages (5.0.0)\n",
45
+ "Collecting vaderSentiment\n",
46
+ " Downloading vaderSentiment-3.3.2-py2.py3-none-any.whl.metadata (572 bytes)\n",
47
+ "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)\n",
48
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)\n",
49
+ "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.3)\n",
50
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)\n",
51
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)\n",
52
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.61.1)\n",
53
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.4.9)\n",
54
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.0)\n",
55
+ "Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)\n",
56
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)\n",
57
+ "Requirement already satisfied: nltk>=3.9 in /usr/local/lib/python3.12/dist-packages (from textblob) (3.9.1)\n",
58
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from transformers) (3.25.0)\n",
59
+ "Requirement already satisfied: huggingface-hub<2.0,>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (1.5.0)\n",
60
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/dist-packages (from transformers) (6.0.3)\n",
61
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.12/dist-packages (from transformers) (2025.11.3)\n",
62
+ "Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.22.2)\n",
63
+ "Requirement already satisfied: typer-slim in /usr/local/lib/python3.12/dist-packages (from transformers) (0.24.0)\n",
64
+ "Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.7.0)\n",
65
+ "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.12/dist-packages (from transformers) (4.67.3)\n",
66
+ "Requirement already satisfied: requests in /usr/local/lib/python3.12/dist-packages (from vaderSentiment) (2.32.4)\n",
67
+ "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (2025.3.0)\n",
68
+ "Requirement already satisfied: hf-xet<2.0.0,>=1.2.0 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (1.3.2)\n",
69
+ "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (0.28.1)\n",
70
+ "Requirement already satisfied: typer in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (0.24.1)\n",
71
+ "Requirement already satisfied: typing-extensions>=4.1.0 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<2.0,>=1.3.0->transformers) (4.15.0)\n",
72
+ "Requirement already satisfied: click in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (8.3.1)\n",
73
+ "Requirement already satisfied: joblib in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (1.5.3)\n",
74
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n",
75
+ "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests->vaderSentiment) (3.4.4)\n",
76
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.12/dist-packages (from requests->vaderSentiment) (3.11)\n",
77
+ "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests->vaderSentiment) (2.5.0)\n",
78
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.12/dist-packages (from requests->vaderSentiment) (2026.2.25)\n",
79
+ "Requirement already satisfied: anyio in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (4.12.1)\n",
80
+ "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.12/dist-packages (from httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (1.0.9)\n",
81
+ "Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.12/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->huggingface-hub<2.0,>=1.3.0->transformers) (0.16.0)\n",
82
+ "Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from typer->huggingface-hub<2.0,>=1.3.0->transformers) (1.5.4)\n",
83
+ "Requirement already satisfied: rich>=12.3.0 in /usr/local/lib/python3.12/dist-packages (from typer->huggingface-hub<2.0,>=1.3.0->transformers) (13.9.4)\n",
84
+ "Requirement already satisfied: annotated-doc>=0.0.2 in /usr/local/lib/python3.12/dist-packages (from typer->huggingface-hub<2.0,>=1.3.0->transformers) (0.0.4)\n",
85
+ "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.12/dist-packages (from rich>=12.3.0->typer->huggingface-hub<2.0,>=1.3.0->transformers) (4.0.0)\n",
86
+ "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.12/dist-packages (from rich>=12.3.0->typer->huggingface-hub<2.0,>=1.3.0->transformers) (2.19.2)\n",
87
+ "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.12/dist-packages (from markdown-it-py>=2.2.0->rich>=12.3.0->typer->huggingface-hub<2.0,>=1.3.0->transformers) (0.1.2)\n",
88
+ "Downloading faker-40.8.0-py3-none-any.whl (2.0 MB)\n",
89
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m20.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
90
+ "\u001b[?25hDownloading vaderSentiment-3.3.2-py2.py3-none-any.whl (125 kB)\n",
91
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m126.0/126.0 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
92
+ "\u001b[?25hInstalling collected packages: faker, vaderSentiment\n",
93
+ "Successfully installed faker-40.8.0 vaderSentiment-3.3.2\n"
94
+ ]
95
+ }
96
+ ],
97
+ "source": [
98
+ "!pip install pandas matplotlib seaborn numpy textblob faker transformers vaderSentiment\n"
99
+ ]
100
+ },
101
+ {
102
+ "cell_type": "markdown",
103
+ "metadata": {
104
+ "id": "NZd99NpKkKyp"
105
+ },
106
+ "source": [
107
+ "## **2.** ✅️ Load & inspect input datasets"
108
+ ]
109
+ },
110
+ {
111
+ "cell_type": "markdown",
112
+ "metadata": {
113
+ "id": "_JBLmm508Uq2"
114
+ },
115
+ "source": [
116
+ "### *a. Initial setup*"
117
+ ]
118
+ },
119
+ {
120
+ "cell_type": "code",
121
+ "execution_count": null,
122
+ "metadata": {
123
+ "id": "eBDXPQz18Xrs"
124
+ },
125
+ "outputs": [],
126
+ "source": [
127
+ "import pandas as pd\n",
128
+ "import numpy as np\n",
129
+ "import random"
130
+ ]
131
+ },
132
+ {
133
+ "cell_type": "markdown",
134
+ "metadata": {
135
+ "id": "IL8lZbMm8m3k"
136
+ },
137
+ "source": [
138
+ "### *b. ✋🏻🛑⛔️ Create the df_reviews dataframe from the synthetic_book_reviews.csv file*"
139
+ ]
140
+ },
141
+ {
142
+ "cell_type": "code",
143
+ "execution_count": null,
144
+ "metadata": {
145
+ "id": "fdgjghfO8uuq",
146
+ "colab": {
147
+ "base_uri": "https://localhost:8080/",
148
+ "height": 311
149
+ },
150
+ "outputId": "e14f27ca-3651-4e5f-e8d6-fcb1446e9ec9"
151
+ },
152
+ "outputs": [
153
+ {
154
+ "output_type": "error",
155
+ "ename": "FileNotFoundError",
156
+ "evalue": "[Errno 2] No such file or directory: 'synthetic_book_reviews.csv'",
157
+ "traceback": [
158
+ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
159
+ "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)",
160
+ "\u001b[0;32m/tmp/ipykernel_347/448808692.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf_reviews\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"synthetic_book_reviews.csv\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
161
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/io/parsers/readers.py\u001b[0m in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, date_format, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, encoding_errors, dialect, on_bad_lines, delim_whitespace, low_memory, memory_map, float_precision, storage_options, dtype_backend)\u001b[0m\n\u001b[1;32m 1024\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwds_defaults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1025\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1026\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1027\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1028\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
162
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/io/parsers/readers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 618\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 619\u001b[0m \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 620\u001b[0;31m \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 621\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 622\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
163
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/io/parsers/readers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m 1618\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1619\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhandles\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mIOHandles\u001b[0m \u001b[0;34m|\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1620\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1621\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1622\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
164
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/io/parsers/readers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, f, engine)\u001b[0m\n\u001b[1;32m 1878\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;34m\"b\"\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1879\u001b[0m \u001b[0mmode\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m\"b\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1880\u001b[0;31m self.handles = get_handle(\n\u001b[0m\u001b[1;32m 1881\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1882\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
165
+ "\u001b[0;32m/usr/local/lib/python3.12/dist-packages/pandas/io/common.py\u001b[0m in \u001b[0;36mget_handle\u001b[0;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001b[0m\n\u001b[1;32m 871\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencoding\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;34m\"b\"\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 872\u001b[0m \u001b[0;31m# Encoding\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 873\u001b[0;31m handle = open(\n\u001b[0m\u001b[1;32m 874\u001b[0m \u001b[0mhandle\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 875\u001b[0m \u001b[0mioargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
166
+ "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'synthetic_book_reviews.csv'"
167
+ ]
168
+ }
169
+ ],
170
+ "source": [
171
+ "df_reviews = pd.read_csv(\"synthetic_book_reviews.csv\")"
172
+ ]
173
+ },
174
+ {
175
+ "cell_type": "markdown",
176
+ "metadata": {
177
+ "id": "N-Dl37J0HLhU"
178
+ },
179
+ "source": [
180
+ "### *c. ✋🏻🛑⛔️ Create the df_sales dataframe from the synthetic_sales_data.csv file*"
181
+ ]
182
+ },
183
+ {
184
+ "cell_type": "code",
185
+ "execution_count": null,
186
+ "metadata": {
187
+ "id": "6XZs3P7fHgQe"
188
+ },
189
+ "outputs": [],
190
+ "source": [
191
+ "df_sales = pd.read_csv(\"synthetic_sales_data.csv\")"
192
+ ]
193
+ },
194
+ {
195
+ "cell_type": "markdown",
196
+ "metadata": {
197
+ "id": "MUI3SkmyrGQo"
198
+ },
199
+ "source": [
200
+ "### *d. ✋🏻🛑⛔️ Visualize the first few lines of the two final datasets: df_reviews and df_sales*"
201
+ ]
202
+ },
203
+ {
204
+ "cell_type": "code",
205
+ "execution_count": null,
206
+ "metadata": {
207
+ "id": "p8FdQFXErOqE",
208
+ "collapsed": true
209
+ },
210
+ "outputs": [],
211
+ "source": [
212
+ "df_reviews.head()"
213
+ ]
214
+ },
215
+ {
216
+ "cell_type": "code",
217
+ "source": [
218
+ "df_sales.head()"
219
+ ],
220
+ "metadata": {
221
+ "collapsed": true,
222
+ "id": "EDdSx5KMNjiB"
223
+ },
224
+ "execution_count": null,
225
+ "outputs": []
226
+ },
227
+ {
228
+ "cell_type": "markdown",
229
+ "metadata": {
230
+ "id": "Y3oqGHsmrQzx"
231
+ },
232
+ "source": [
233
+ "### *d. Run a quality check on the datasets*"
234
+ ]
235
+ },
236
+ {
237
+ "cell_type": "code",
238
+ "execution_count": null,
239
+ "metadata": {
240
+ "id": "VArQGPoKrfLm",
241
+ "collapsed": true
242
+ },
243
+ "outputs": [],
244
+ "source": [
245
+ "def quality_check(df, name=\"DataFrame\"):\n",
246
+ " print(f\"\\n🔍 Quality Check Report for: {name}\")\n",
247
+ " print(\"=\" * (25 + len(name)))\n",
248
+ "\n",
249
+ " # Basic info\n",
250
+ " print(f\"\\n📏 Shape: {df.shape}\")\n",
251
+ " print(\"\\n🔠 Column Types:\")\n",
252
+ " print(df.dtypes)\n",
253
+ "\n",
254
+ " # Missing values\n",
255
+ " print(\"\\n❓ Missing Values:\")\n",
256
+ " print(df.isnull().sum())\n",
257
+ "\n",
258
+ " # Duplicates\n",
259
+ " duplicate_count = df.duplicated().sum()\n",
260
+ " print(f\"\\n📋 Duplicate Rows: {duplicate_count}\")\n",
261
+ "\n",
262
+ " # Summary stats\n",
263
+ " print(\"\\n📊 Summary Statistics:\")\n",
264
+ " display(df.describe(include='all').transpose())\n",
265
+ "\n",
266
+ " # Sample rows\n",
267
+ " print(\"\\n👀 Sample Rows:\")\n",
268
+ " display(df.sample(5))\n",
269
+ "\n",
270
+ "# Run checks\n",
271
+ "quality_check(df_reviews, \"df_reviews\")\n",
272
+ "quality_check(df_sales, \"df_sales\")\n"
273
+ ]
274
+ },
275
+ {
276
+ "cell_type": "markdown",
277
+ "metadata": {
278
+ "id": "TTxUKDYINPxV"
279
+ },
280
+ "source": [
281
+ "## **3.** 🎭 Perform sentiment analysis using VADER"
282
+ ]
283
+ },
284
+ {
285
+ "cell_type": "markdown",
286
+ "metadata": {
287
+ "id": "OqhYU8rDxQRT"
288
+ },
289
+ "source": [
290
+ "### *a. Initial setup*"
291
+ ]
292
+ },
293
+ {
294
+ "cell_type": "code",
295
+ "execution_count": null,
296
+ "metadata": {
297
+ "id": "DNk5w8mNxSZ6"
298
+ },
299
+ "outputs": [],
300
+ "source": [
301
+ "from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer\n",
302
+ "\n",
303
+ "# 🤖 Initialize VADER analyzer\n",
304
+ "analyzer = SentimentIntensityAnalyzer()"
305
+ ]
306
+ },
307
+ {
308
+ "cell_type": "markdown",
309
+ "metadata": {
310
+ "id": "P123TwSWxVAr"
311
+ },
312
+ "source": [
313
+ "### *b. Create a function get_sentiment_label that will return the label negative, neutral, or positive based on the VADER analyzer's scoring of the text*"
314
+ ]
315
+ },
316
+ {
317
+ "cell_type": "code",
318
+ "execution_count": null,
319
+ "metadata": {
320
+ "id": "89809e6f"
321
+ },
322
+ "outputs": [],
323
+ "source": [
324
+ "def get_sentiment_label(text):\n",
325
+ " score = analyzer.polarity_scores(text)[\"compound\"]\n",
326
+ " if score >= 0.05:\n",
327
+ " return \"positive\"\n",
328
+ " elif score <= -0.05:\n",
329
+ " return \"negative\"\n",
330
+ " else:\n",
331
+ " return \"neutral\""
332
+ ]
333
+ },
334
+ {
335
+ "cell_type": "markdown",
336
+ "metadata": {
337
+ "id": "DS9eCZ95yQn3"
338
+ },
339
+ "source": [
340
+ "### *c. ✋🏻🛑⛔️ Apply get_sentiment_label to df_reviews column named review_text to get sentiment_label column*"
341
+ ]
342
+ },
343
+ {
344
+ "cell_type": "code",
345
+ "execution_count": null,
346
+ "metadata": {
347
+ "id": "SpXzFaDfyM7I"
348
+ },
349
+ "outputs": [],
350
+ "source": [
351
+ "# Create sentiment_label column using VADER sentiment analysis\n",
352
+ "df_reviews[\"sentiment_label\"] = df_reviews[\"review_text\"].apply(get_sentiment_label)\n",
353
+ "\n",
354
+ "# Preview results\n",
355
+ "df_reviews.head()"
356
+ ]
357
+ },
358
+ {
359
+ "cell_type": "markdown",
360
+ "metadata": {
361
+ "id": "5cnPCFFnyXN6"
362
+ },
363
+ "source": [
364
+ "### *d. ✋🏻🛑⛔️ View the first few lines of the resulting table df_reviews*"
365
+ ]
366
+ },
367
+ {
368
+ "cell_type": "code",
369
+ "execution_count": null,
370
+ "metadata": {
371
+ "id": "ODGyfjBSyZEO"
372
+ },
373
+ "outputs": [],
374
+ "source": [
375
+ "df_reviews.head()"
376
+ ]
377
+ },
378
+ {
379
+ "cell_type": "markdown",
380
+ "metadata": {
381
+ "id": "Qy3Hqm-FojvT"
382
+ },
383
+ "source": [
384
+ "## **4.** 📊 Use the following data visualization code snippets"
385
+ ]
386
+ },
387
+ {
388
+ "cell_type": "markdown",
389
+ "metadata": {
390
+ "id": "lcjGSw2bzqtZ"
391
+ },
392
+ "source": [
393
+ "### *a. Initial setup*"
394
+ ]
395
+ },
396
+ {
397
+ "cell_type": "code",
398
+ "execution_count": null,
399
+ "metadata": {
400
+ "id": "p5LV2o1rzsiC"
401
+ },
402
+ "outputs": [],
403
+ "source": [
404
+ "import matplotlib.pyplot as plt\n",
405
+ "import seaborn as sns\n",
406
+ "import matplotlib.dates as mdates"
407
+ ]
408
+ },
409
+ {
410
+ "cell_type": "code",
411
+ "execution_count": null,
412
+ "metadata": {
413
+ "id": "tvaBtswpGS__"
414
+ },
415
+ "outputs": [],
416
+ "source": [
417
+ "# ----------------------------\n",
418
+ "# Outputs (for Hugging Face app)\n",
419
+ "# ----------------------------\n",
420
+ "# In the notebook: you still SEE interactive tables/plots inline.\n",
421
+ "# For the Space dashboard: we also SAVE the same outputs as files.\n",
422
+ "\n",
423
+ "from pathlib import Path\n",
424
+ "\n",
425
+ "ART_DIR = Path(\"artifacts\")\n",
426
+ "PY_FIG = ART_DIR / \"py\" / \"figures\"\n",
427
+ "PY_TAB = ART_DIR / \"py\" / \"tables\"\n",
428
+ "\n",
429
+ "for p in [PY_FIG, PY_TAB]:\n",
430
+ " p.mkdir(parents=True, exist_ok=True)\n",
431
+ "\n",
432
+ "print(\"✅ Output folders:\")\n",
433
+ "print(\" -\", PY_FIG.resolve())\n",
434
+ "print(\" -\", PY_TAB.resolve())\n"
435
+ ]
436
+ },
437
+ {
438
+ "cell_type": "markdown",
439
+ "metadata": {
440
+ "id": "b9T1rkBe0AJU"
441
+ },
442
+ "source": [
443
+ "### *b. Sample of 5 books for each popularity level for visualizations*"
444
+ ]
445
+ },
446
+ {
447
+ "cell_type": "code",
448
+ "execution_count": null,
449
+ "metadata": {
450
+ "id": "sLdFmGqXqo_t"
451
+ },
452
+ "outputs": [],
453
+ "source": [
454
+ "sampled_titles = []\n",
455
+ "for pop_score in sorted(df_reviews[\"popularity_score\"].dropna().unique()):\n",
456
+ " all_titles = df_reviews[df_reviews[\"popularity_score\"] == pop_score][\"title\"].unique()\n",
457
+ " sampled = random.sample(list(all_titles), min(5, len(all_titles)))\n",
458
+ " sampled_titles.extend(sampled)"
459
+ ]
460
+ },
461
+ {
462
+ "cell_type": "markdown",
463
+ "metadata": {
464
+ "id": "xq7-C8m70mMH"
465
+ },
466
+ "source": [
467
+ "### *c. Copy relevant sales, reviews, and book names*"
468
+ ]
469
+ },
470
+ {
471
+ "cell_type": "code",
472
+ "execution_count": null,
473
+ "metadata": {
474
+ "id": "laDdMece0qrq"
475
+ },
476
+ "outputs": [],
477
+ "source": [
478
+ "sampled_sales = df_sales[df_sales[\"title\"].isin(sampled_titles)].copy()\n",
479
+ "sampled_reviews = df_reviews[df_reviews[\"title\"].isin(sampled_titles)].copy()\n",
480
+ "sampled_books = df_reviews[df_reviews[\"title\"].isin(sampled_titles)].copy()"
481
+ ]
482
+ },
483
+ {
484
+ "cell_type": "markdown",
485
+ "metadata": {
486
+ "id": "8YtfkG_A0wTy"
487
+ },
488
+ "source": [
489
+ "### *d. Plot sales trends over time for the sampled books*"
490
+ ]
491
+ },
492
+ {
493
+ "cell_type": "code",
494
+ "execution_count": null,
495
+ "metadata": {
496
+ "id": "1iTVzflW0Rkw"
497
+ },
498
+ "outputs": [],
499
+ "source": [
500
+ "# 🕒 Ensure datetime format\n",
501
+ "df_sales[\"month\"] = pd.to_datetime(df_sales[\"month\"])\n",
502
+ "# 🎨 Color mapping\n",
503
+ "popularity_colors = {\n",
504
+ " 1: \"darkred\", 2: \"orangered\", 3: \"gold\", 4: \"mediumseagreen\", 5: \"royalblue\"\n",
505
+ "}\n",
506
+ "\n",
507
+ "# 📈 Plot 1: Sales trends\n",
508
+ "plt.figure(figsize=(20, 8))\n",
509
+ "for title in sampled_titles:\n",
510
+ " row = sampled_books[sampled_books[\"title\"] == title].iloc[0]\n",
511
+ " color = popularity_colors.get(row[\"popularity_score\"], \"gray\")\n",
512
+ " subset = sampled_sales[sampled_sales[\"title\"] == title]\n",
513
+ " plt.plot(subset[\"month\"], subset[\"units_sold\"], label=f\"{title} (Pop. {row['popularity_score']})\", color=color)\n",
514
+ "\n",
515
+ "plt.title(\"📈 Sales Trends Over Time (5 per Popularity Level)\")\n",
516
+ "plt.xlabel(\"Month\")\n",
517
+ "plt.ylabel(\"Units Sold\")\n",
518
+ "plt.xticks(rotation=45)\n",
519
+ "plt.legend(loc='center left', bbox_to_anchor=(1, 0.5), fontsize='small')\n",
520
+ "plt.grid(True)\n",
521
+ "plt.tight_layout()\n",
522
+ "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%b %Y'))\n",
523
+ "plt.savefig(PY_FIG / 'sales_trends_sampled_titles.png', dpi=150)\n",
524
+ "plt.show()"
525
+ ]
526
+ },
527
+ {
528
+ "cell_type": "markdown",
529
+ "metadata": {
530
+ "id": "lDpMkjDP1K6j"
531
+ },
532
+ "source": [
533
+ "### *e. Plot sentiment_label distribution per book*"
534
+ ]
535
+ },
536
+ {
537
+ "cell_type": "code",
538
+ "execution_count": null,
539
+ "metadata": {
540
+ "id": "dn1Jgd5R1KLu"
541
+ },
542
+ "outputs": [],
543
+ "source": [
544
+ "# 🎨 Give a new name to each book that includes the rating together with the title\n",
545
+ "sampled_reviews[\"grouped_title\"] = sampled_reviews[\"rating\"].astype(str) + \"★ | \" + sampled_reviews[\"title\"]\n",
546
+ "\n",
547
+ "# 📊 Aggregate sentiment counts\n",
548
+ "sentiment_counts = (\n",
549
+ " sampled_reviews.groupby([\"grouped_title\", \"sentiment_label\"])\n",
550
+ " .size()\n",
551
+ " .unstack(fill_value=0)[[\"negative\", \"neutral\", \"positive\"]] # consistent order\n",
552
+ ")\n",
553
+ "\n",
554
+ "# 💾 Save table for HF dashboard\n",
555
+ "sentiment_counts.reset_index().to_csv(PY_TAB / 'sentiment_counts_sampled.csv', index=False)\n",
556
+ "\n",
557
+ "\n",
558
+ "# ✅ Plot stacked horizontal bars\n",
559
+ "fig, ax = plt.subplots(figsize=(12, 14))\n",
560
+ "sentiment_counts.plot.barh(\n",
561
+ " stacked=True,\n",
562
+ " ax=ax,\n",
563
+ " color={\"negative\": \"royalblue\", \"neutral\": \"lightgray\", \"positive\": \"crimson\"}\n",
564
+ ")\n",
565
+ "\n",
566
+ "plt.title(\"💬 Sentiment Distribution in Reviews (5 Books per Popularity Level)\", fontsize=14)\n",
567
+ "plt.xlabel(\"Number of Reviews\")\n",
568
+ "plt.ylabel(\"Book Title (Grouped by Popularity Score)\")\n",
569
+ "plt.legend(title=\"Sentiment\", loc=\"lower right\")\n",
570
+ "plt.grid(axis=\"x\", linestyle=\"--\", alpha=0.6)\n",
571
+ "plt.tight_layout()\n",
572
+ "plt.savefig(PY_FIG / 'sentiment_distribution_sampled_titles.png', dpi=150)\n",
573
+ "plt.show()"
574
+ ]
575
+ },
576
+ {
577
+ "cell_type": "markdown",
578
+ "metadata": {
579
+ "id": "rmgylC1ENCHy"
580
+ },
581
+ "source": [
582
+ "## **5.** 🔮 Forecast book sales with the following ARIMA code"
583
+ ]
584
+ },
585
+ {
586
+ "cell_type": "markdown",
587
+ "metadata": {
588
+ "id": "jFV4JE1R3FKH"
589
+ },
590
+ "source": [
591
+ "### *a. Initial setup*"
592
+ ]
593
+ },
594
+ {
595
+ "cell_type": "code",
596
+ "execution_count": null,
597
+ "metadata": {
598
+ "id": "Mh8Alha03H22"
599
+ },
600
+ "outputs": [],
601
+ "source": [
602
+ "import matplotlib.pyplot as plt\n",
603
+ "import matplotlib.dates as mdates\n",
604
+ "import statsmodels.api as sm\n",
605
+ "from itertools import product\n",
606
+ "import matplotlib.cm as cm\n",
607
+ "import warnings"
608
+ ]
609
+ },
610
+ {
611
+ "cell_type": "markdown",
612
+ "metadata": {
613
+ "id": "gHucD8OW3U0w"
614
+ },
615
+ "source": [
616
+ "### *b. Define function find_best_arima to try different ARIMA parameter values and return the best combination for each book's price forecast*"
617
+ ]
618
+ },
619
+ {
620
+ "cell_type": "code",
621
+ "execution_count": null,
622
+ "metadata": {
623
+ "id": "477fa43f"
624
+ },
625
+ "outputs": [],
626
+ "source": [
627
+ "def find_best_arima(series, p_range=(0, 5), d_range=(0, 2), q_range=(0, 1)):\n",
628
+ " best_aic = float(\"inf\")\n",
629
+ " best_order = None\n",
630
+ " best_model = None\n",
631
+ "\n",
632
+ " for p, d, q in product(range(p_range[0], p_range[1] + 1),\n",
633
+ " range(d_range[0], d_range[1] + 1),\n",
634
+ " range(q_range[0], q_range[1] + 1)):\n",
635
+ " try:\n",
636
+ " model = sm.tsa.ARIMA(series, order=(p, d, q))\n",
637
+ " results = model.fit()\n",
638
+ " if results.aic < best_aic:\n",
639
+ " best_aic = results.aic\n",
640
+ " best_order = (p, d, q)\n",
641
+ " best_model = results\n",
642
+ " except:\n",
643
+ " continue\n",
644
+ "\n",
645
+ " return best_order, best_model"
646
+ ]
647
+ },
648
+ {
649
+ "cell_type": "markdown",
650
+ "metadata": {
651
+ "id": "Rq5t1Hey3jkD"
652
+ },
653
+ "source": [
654
+ "### *c. Plot the figure*"
655
+ ]
656
+ },
657
+ {
658
+ "cell_type": "code",
659
+ "execution_count": null,
660
+ "metadata": {
661
+ "id": "DmxGdvLE3dHQ"
662
+ },
663
+ "outputs": [],
664
+ "source": [
665
+ "# 🎨 Generate 25 highly distinct colors using HUSL (hue-saturation-lightness)\n",
666
+ "colors = sns.color_palette(\"tab10\", len(sampled_titles))\n",
667
+ "\n",
668
+ "plt.figure(figsize=(16, 10))\n",
669
+ "\n",
670
+ "for i, title in enumerate(sampled_titles):\n",
671
+ " book_sales = sampled_sales[sampled_sales[\"title\"] == title].copy()\n",
672
+ " book_sales[\"month\"] = pd.to_datetime(book_sales[\"month\"])\n",
673
+ " book_sales = book_sales.sort_values(\"month\").set_index(\"month\")\n",
674
+ "\n",
675
+ " with warnings.catch_warnings():\n",
676
+ " warnings.simplefilter(\"ignore\")\n",
677
+ " best_order, best_model = find_best_arima(book_sales[\"units_sold\"])\n",
678
+ " if best_model is not None:\n",
679
+ " forecast = best_model.get_forecast(steps=6)\n",
680
+ " forecast_index = pd.date_range(start=book_sales.index[-1] + pd.DateOffset(months=1), periods=6, freq='MS')\n",
681
+ "\n",
682
+ " # 🟦 Plot observed sales (solid line)\n",
683
+ " plt.plot(book_sales.index, book_sales[\"units_sold\"], color=colors[i], label=title, linewidth=2)\n",
684
+ "\n",
685
+ " # 🟠 Plot forecast (dotted line, same color)\n",
686
+ " plt.plot(forecast_index, forecast.predicted_mean, linestyle=\"--\", color=colors[i], linewidth=2)\n",
687
+ "\n",
688
+ "# 📈 Final formatting\n",
689
+ "plt.title(\"📈 ARIMA Forecasts for Sampled Books (1 per Popularity Level)\", fontsize=14)\n",
690
+ "plt.xlabel(\"Month\")\n",
691
+ "plt.ylabel(\"Units Sold\")\n",
692
+ "plt.xticks(rotation=45)\n",
693
+ "plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%b %Y'))\n",
694
+ "plt.grid(True)\n",
695
+ "plt.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5), fontsize=\"small\")\n",
696
+ "plt.tight_layout()\n",
697
+ "plt.savefig(PY_FIG / 'arima_forecasts_sampled_titles.png', dpi=150)\n",
698
+ "plt.show()"
699
+ ]
700
+ },
701
+ {
702
+ "cell_type": "markdown",
703
+ "metadata": {
704
+ "id": "SKBcx3fyCFly"
705
+ },
706
+ "source": [
707
+ "## **6.** 🏷️ Decide on price changes with a rule-based approach based on sentiment and future revenue"
708
+ ]
709
+ },
710
+ {
711
+ "cell_type": "markdown",
712
+ "metadata": {
713
+ "id": "nY-vV2JJDZqu"
714
+ },
715
+ "source": [
716
+ "### *a. Calculate average sales per book*"
717
+ ]
718
+ },
719
+ {
720
+ "cell_type": "code",
721
+ "execution_count": null,
722
+ "metadata": {
723
+ "id": "nbDT_RHaDD2R"
724
+ },
725
+ "outputs": [],
726
+ "source": [
727
+ "avg_sales = df_sales.groupby(\"title\")[\"units_sold\"].mean().reset_index()\n",
728
+ "avg_sales.columns = [\"title\", \"avg_units_sold\"]"
729
+ ]
730
+ },
731
+ {
732
+ "cell_type": "markdown",
733
+ "metadata": {
734
+ "id": "94wi-RvkDf2z"
735
+ },
736
+ "source": [
737
+ "### *b. Calculate sentiment distribution per book*"
738
+ ]
739
+ },
740
+ {
741
+ "cell_type": "code",
742
+ "execution_count": null,
743
+ "metadata": {
744
+ "id": "fWjQ9IOXDk-M"
745
+ },
746
+ "outputs": [],
747
+ "source": [
748
+ "sentiment_counts = df_reviews.groupby([\"title\", \"sentiment_label\"]).size().unstack(fill_value=0)\n",
749
+ "sentiment_counts[\"total\"] = sentiment_counts.sum(axis=1)\n",
750
+ "sentiment_counts[\"positive_ratio\"] = sentiment_counts.get(\"positive\") / sentiment_counts[\"total\"]\n",
751
+ "sentiment_counts[\"negative_ratio\"] = sentiment_counts.get(\"negative\") / sentiment_counts[\"total\"]"
752
+ ]
753
+ },
754
+ {
755
+ "cell_type": "markdown",
756
+ "metadata": {
757
+ "id": "Vm10ym_iDtEW"
758
+ },
759
+ "source": [
760
+ "### *c. Merge the calculated sales and sentiment characteristics*"
761
+ ]
762
+ },
763
+ {
764
+ "cell_type": "code",
765
+ "execution_count": null,
766
+ "metadata": {
767
+ "id": "T-zlh6rBDpxg"
768
+ },
769
+ "outputs": [],
770
+ "source": [
771
+ "df_decision = avg_sales.merge(sentiment_counts, on=\"title\", how=\"left\")"
772
+ ]
773
+ },
774
+ {
775
+ "cell_type": "markdown",
776
+ "metadata": {
777
+ "id": "1WIWDojyD7fK"
778
+ },
779
+ "source": [
780
+ "### *d. ✋🏻🛑⛔️ Create the pricing_decision function as a basic rule-based pricing decider based on sentiment and revenue*\n",
781
+ "\n",
782
+ "\n",
783
+ "\n",
784
+ "\n"
785
+ ]
786
+ },
787
+ {
788
+ "cell_type": "markdown",
789
+ "metadata": {
790
+ "id": "b5qJCb46Dxfb"
791
+ },
792
+ "source": [
793
+ "* If there are 120 or more average units sold and 0.6 or higher positive ratio, the decision should be to increase price.\n",
794
+ "* If there are 60 or less average units sold and 0.4 or higher negative ratio, the decision should be to decrease price.\n",
795
+ "* Otherwise, the price should be kept the same."
796
+ ]
797
+ },
798
+ {
799
+ "cell_type": "code",
800
+ "execution_count": null,
801
+ "metadata": {
802
+ "id": "XBzozedwD6yx"
803
+ },
804
+ "outputs": [],
805
+ "source": [
806
+ "def pricing_decision(row):\n",
807
+ " avg_units = row[\"avg_units_sold\"]\n",
808
+ " positive_ratio = row.get(\"positive_ratio\", 0)\n",
809
+ " negative_ratio = row.get(\"negative_ratio\", 0)\n",
810
+ "\n",
811
+ " if avg_units >= 120 and positive_ratio >= 0.6:\n",
812
+ " return \"increase price\"\n",
813
+ " elif avg_units <= 60 and negative_ratio >= 0.4:\n",
814
+ " return \"decrease price\"\n",
815
+ " else:\n",
816
+ " return \"keep price\"\n",
817
+ "\n",
818
+ "df_decision[\"pricing_action\"] = df_decision.apply(pricing_decision, axis=1)\n",
819
+ "\n",
820
+ "# Preview results\n",
821
+ "print(df_decision.head())\n",
822
+ "\n",
823
+ "# Optional: distribution of decisions\n",
824
+ "print(df_decision[\"pricing_action\"].value_counts())"
825
+ ]
826
+ },
827
+ {
828
+ "cell_type": "markdown",
829
+ "metadata": {
830
+ "id": "xmLEdF14EPAA"
831
+ },
832
+ "source": [
833
+ "### *e. ✋🏻🛑⛔️ Run the pricing_decision function and check out the first few decisions*"
834
+ ]
835
+ },
836
+ {
837
+ "cell_type": "code",
838
+ "execution_count": null,
839
+ "metadata": {
840
+ "id": "TZ0ZhgHrEQJB"
841
+ },
842
+ "outputs": [],
843
+ "source": [
844
+ "# Apply the pricing decision rule to each row\n",
845
+ "df_decision[\"pricing_action\"] = df_decision.apply(pricing_decision, axis=1)\n",
846
+ "\n",
847
+ "# Display the first few pricing decisions\n",
848
+ "df_decision[[\"title\", \"avg_units_sold\", \"positive_ratio\", \"negative_ratio\", \"pricing_action\"]].head()"
849
+ ]
850
+ },
851
+ {
852
+ "cell_type": "markdown",
853
+ "metadata": {
854
+ "id": "WTkP2_-EApev"
855
+ },
856
+ "source": [
857
+ "\n",
858
+ "## **7.** 💾 Save Python outputs for the Hugging Face dashboard"
859
+ ]
860
+ },
861
+ {
862
+ "cell_type": "markdown",
863
+ "metadata": {
864
+ "id": "3EIjfnokGpJv"
865
+ },
866
+ "source": [
867
+ "\n",
868
+ "This section exports **HF-ready artifacts** into a consistent folder structure:\n",
869
+ "\n",
870
+ "- `(root folder)py/figures/` (Python-generated visuals)\n",
871
+ "- `(root folder)py/tables/` (tables/metrics)"
872
+ ]
873
+ },
874
+ {
875
+ "cell_type": "code",
876
+ "execution_count": null,
877
+ "metadata": {
878
+ "id": "ZJJ4PMgIApev"
879
+ },
880
+ "outputs": [],
881
+ "source": [
882
+ "\n",
883
+ "import json\n",
884
+ "\n",
885
+ "# -------------------------\n",
886
+ "# 1) Dashboard table (monthly) — reuse if already built\n",
887
+ "# -------------------------\n",
888
+ "if \"df_monthly\" in globals() and df_monthly is not None:\n",
889
+ " df_dashboard = df_monthly.copy()\n",
890
+ "else:\n",
891
+ " # fallback: monthly units sold only\n",
892
+ " df_dashboard = (\n",
893
+ " df_sales.groupby(\"month\", as_index=False)\n",
894
+ " .agg(total_units_sold=(\"units_sold\", \"sum\"))\n",
895
+ " .sort_values(\"month\")\n",
896
+ " )\n",
897
+ "\n",
898
+ "# Save the single overview dashboard table\n",
899
+ "df_dashboard.to_csv(PY_TAB / \"df_dashboard.csv\", index=False)\n",
900
+ "\n",
901
+ "# -------------------------\n",
902
+ "# 2) KPI summary (small json) — computed from raw df_sales + df_dashboard\n",
903
+ "# -------------------------\n",
904
+ "kpis = {\n",
905
+ " \"n_titles\": int(df_sales[\"title\"].nunique()),\n",
906
+ " \"n_months\": int(df_dashboard[\"month\"].nunique()),\n",
907
+ " \"total_units_sold\": float(df_sales[\"units_sold\"].sum()),\n",
908
+ "}\n",
909
+ "\n",
910
+ "# Only include revenue KPIs if df_dashboard contains it (since you said monthly revenue already exists)\n",
911
+ "if \"total_revenue\" in df_dashboard.columns and df_dashboard[\"total_revenue\"].notna().any():\n",
912
+ " kpis[\"total_revenue\"] = float(df_dashboard[\"total_revenue\"].sum())\n",
913
+ "\n",
914
+ "with open(PY_FIG / \"kpis.json\", \"w\", encoding=\"utf-8\") as f:\n",
915
+ " json.dump(kpis, f, indent=2)\n",
916
+ "\n",
917
+ "# -------------------------\n",
918
+ "# 3) Python tables (title-level quick inspection)\n",
919
+ "# -------------------------\n",
920
+ "df_by_title_units = (\n",
921
+ " df_sales.groupby(\"title\", as_index=False)\n",
922
+ " .agg(total_units_sold=(\"units_sold\", \"sum\"))\n",
923
+ " .sort_values(\"total_units_sold\", ascending=False)\n",
924
+ ")\n",
925
+ "df_by_title_units.head(10).to_csv(PY_TAB / \"top_titles_by_units_sold.csv\", index=False)\n",
926
+ "\n",
927
+ "# Optional: title-level revenue table ONLY if df_sales already has per-row revenue\n",
928
+ "if \"revenue\" in df_sales.columns and df_sales[\"revenue\"].notna().any():\n",
929
+ " df_by_title_rev = (\n",
930
+ " df_sales.groupby(\"title\", as_index=False)\n",
931
+ " .agg(total_revenue=(\"revenue\", \"sum\"))\n",
932
+ " .sort_values(\"total_revenue\", ascending=False)\n",
933
+ " )\n",
934
+ " df_by_title_rev.head(10).to_csv(PY_TAB / \"top_titles_by_revenue.csv\", index=False)\n",
935
+ "\n",
936
+ "print(\"✅ Exports written to artifacts/:\")\n",
937
+ "print(\" - common/: df_dashboard.csv, kpis.json\")\n",
938
+ "print(\" - py/tables/: top_titles_by_units_sold.csv (+ optional top_titles_by_revenue.csv)\")\n"
939
+ ]
940
+ },
941
+ {
942
+ "cell_type": "markdown",
943
+ "metadata": {
944
+ "id": "0b4e76d3"
945
+ },
946
+ "source": [
947
+ "✅ **Extra outputs for the R notebook**: `(root folder)common/r_input_title_level.csv` and `(root folder)common/r_input_monthly_revenue.csv` (these are the only two files the R portion needs)."
948
+ ]
949
+ }
950
+ ],
951
+ "metadata": {
952
+ "colab": {
953
+ "collapsed_sections": [
954
+ "jpASMyIQMaAq",
955
+ "NZd99NpKkKyp",
956
+ "TTxUKDYINPxV",
957
+ "Qy3Hqm-FojvT",
958
+ "rmgylC1ENCHy",
959
+ "SKBcx3fyCFly",
960
+ "WTkP2_-EApev"
961
+ ],
962
+ "provenance": []
963
+ },
964
+ "kernelspec": {
965
+ "display_name": "Python 3",
966
+ "name": "python3"
967
+ },
968
+ "language_info": {
969
+ "name": "python"
970
+ }
971
+ },
972
+ "nbformat": 4,
973
+ "nbformat_minor": 0
974
+ }
SE21_2526_Hands_On_Activity_II_(1) (1).ipynb ADDED
The diff for this file is too large to render. See raw diff