Hugging Face's logo

Spaces:
ESCP
/
RX12WorkshopTemplate
Running

App Files Community
24
Mateofr21 commited on
Commit
8ad8e09
·
verified ·
1 Parent(s): dd40b75

Upload 3 files

Browse files
Files changed (3) hide show
  1. 1_Data_Creation_Matéo_François.ipynb +1975 -0
  2. 2a_Python_Analysis_Matéo_François.ipynb +0 -0
  3. R_analysis_Matéo_FRANCOIS.ipynb +518 -0
1_Data_Creation_Matéo_François.ipynb ADDED
@@ -0,0 +1,1975 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "4ba6aba8"
7
+ },
8
+ "source": [
9
+ "# 🤖 **Data Collection, Creation, Storage, and Processing**\n"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "jpASMyIQMaAq"
16
+ },
17
+ "source": [
18
+ "## **1.** 📦 Install required packages"
19
+ ]
20
+ },
21
+ {
22
+ "cell_type": "code",
23
+ "execution_count": 1,
24
+ "metadata": {
25
+ "colab": {
26
+ "base_uri": "https://localhost:8080/"
27
+ },
28
+ "id": "f48c8f8c",
29
+ "outputId": "adcbf6a8-cd68-4670-e3b5-78aed002e480"
30
+ },
31
+ "outputs": [
32
+ {
33
+ "output_type": "stream",
34
+ "name": "stdout",
35
+ "text": [
36
+ "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.12/dist-packages (4.13.5)\n",
37
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)\n",
38
+ "Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)\n",
39
+ "Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2)\n",
40
+ "Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)\n",
41
+ "Requirement already satisfied: textblob in /usr/local/lib/python3.12/dist-packages (0.19.0)\n",
42
+ "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (2.8.3)\n",
43
+ "Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (4.15.0)\n",
44
+ "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)\n",
45
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)\n",
46
+ "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.3)\n",
47
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)\n",
48
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)\n",
49
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.61.1)\n",
50
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.4.9)\n",
51
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.0)\n",
52
+ "Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)\n",
53
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)\n",
54
+ "Requirement already satisfied: nltk>=3.9 in /usr/local/lib/python3.12/dist-packages (from textblob) (3.9.1)\n",
55
+ "Requirement already satisfied: click in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (8.3.1)\n",
56
+ "Requirement already satisfied: joblib in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (1.5.3)\n",
57
+ "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (2025.11.3)\n",
58
+ "Requirement already satisfied: tqdm in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (4.67.3)\n",
59
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n"
60
+ ]
61
+ }
62
+ ],
63
+ "source": [
64
+ "!pip install beautifulsoup4 pandas matplotlib seaborn numpy textblob"
65
+ ]
66
+ },
67
+ {
68
+ "cell_type": "markdown",
69
+ "metadata": {
70
+ "id": "lquNYCbfL9IM"
71
+ },
72
+ "source": [
73
+ "## **2.** ⛏ Web-scrape all book titles, prices, and ratings from books.toscrape.com"
74
+ ]
75
+ },
76
+ {
77
+ "cell_type": "markdown",
78
+ "metadata": {
79
+ "id": "0IWuNpxxYDJF"
80
+ },
81
+ "source": [
82
+ "### *a. Initial setup*\n",
83
+ "Define the base url of the website you will scrape as well as how and what you will scrape"
84
+ ]
85
+ },
86
+ {
87
+ "cell_type": "code",
88
+ "execution_count": 2,
89
+ "metadata": {
90
+ "id": "91d52125"
91
+ },
92
+ "outputs": [],
93
+ "source": [
94
+ "import requests\n",
95
+ "from bs4 import BeautifulSoup\n",
96
+ "import pandas as pd\n",
97
+ "import time\n",
98
+ "\n",
99
+ "base_url = \"https://books.toscrape.com/catalogue/page-{}.html\"\n",
100
+ "headers = {\"User-Agent\": \"Mozilla/5.0\"}\n",
101
+ "\n",
102
+ "titles, prices, ratings = [], [], []"
103
+ ]
104
+ },
105
+ {
106
+ "cell_type": "markdown",
107
+ "metadata": {
108
+ "id": "oCdTsin2Yfp3"
109
+ },
110
+ "source": [
111
+ "### *b. Fill titles, prices, and ratings from the web pages*"
112
+ ]
113
+ },
114
+ {
115
+ "cell_type": "code",
116
+ "execution_count": 3,
117
+ "metadata": {
118
+ "id": "xqO5Y3dnYhxt"
119
+ },
120
+ "outputs": [],
121
+ "source": [
122
+ "# Loop through all 50 pages\n",
123
+ "for page in range(1, 51):\n",
124
+ " url = base_url.format(page)\n",
125
+ " response = requests.get(url, headers=headers)\n",
126
+ " soup = BeautifulSoup(response.content, \"html.parser\")\n",
127
+ " books = soup.find_all(\"article\", class_=\"product_pod\")\n",
128
+ "\n",
129
+ " for book in books:\n",
130
+ " titles.append(book.h3.a[\"title\"])\n",
131
+ " prices.append(float(book.find(\"p\", class_=\"price_color\").text[1:]))\n",
132
+ " ratings.append(book.p.get(\"class\")[1])\n",
133
+ "\n",
134
+ " time.sleep(0.5) # polite scraping delay"
135
+ ]
136
+ },
137
+ {
138
+ "cell_type": "markdown",
139
+ "metadata": {
140
+ "id": "T0TOeRC4Yrnn"
141
+ },
142
+ "source": [
143
+ "### *c. ✋🏻🛑⛔️ Create a dataframe df_books that contains the now complete \"title\", \"price\", and \"rating\" objects*"
144
+ ]
145
+ },
146
+ {
147
+ "cell_type": "code",
148
+ "execution_count": 4,
149
+ "metadata": {
150
+ "id": "l5FkkNhUYTHh",
151
+ "colab": {
152
+ "base_uri": "https://localhost:8080/",
153
+ "height": 201
154
+ },
155
+ "outputId": "74d8c2c9-fe97-4c4e-9cdd-858dde7b4140"
156
+ },
157
+ "outputs": [
158
+ {
159
+ "output_type": "execute_result",
160
+ "data": {
161
+ "text/plain": [
162
+ " title price rating\n",
163
+ "0 A Light in the Attic 51.77 Three\n",
164
+ "1 Tipping the Velvet 53.74 One\n",
165
+ "2 Soumission 50.10 One\n",
166
+ "3 Sharp Objects 47.82 Four\n",
167
+ "4 Sapiens: A Brief History of Humankind 54.23 Five"
168
+ ],
169
+ "text/html": [
170
+ "\n",
171
+ " <div id=\"df-9abbc423-e6d8-4007-8c2d-6f00e62cbb32\" class=\"colab-df-container\">\n",
172
+ " <div>\n",
173
+ "<style scoped>\n",
174
+ " .dataframe tbody tr th:only-of-type {\n",
175
+ " vertical-align: middle;\n",
176
+ " }\n",
177
+ "\n",
178
+ " .dataframe tbody tr th {\n",
179
+ " vertical-align: top;\n",
180
+ " }\n",
181
+ "\n",
182
+ " .dataframe thead th {\n",
183
+ " text-align: right;\n",
184
+ " }\n",
185
+ "</style>\n",
186
+ "<table border=\"1\" class=\"dataframe\">\n",
187
+ " <thead>\n",
188
+ " <tr style=\"text-align: right;\">\n",
189
+ " <th></th>\n",
190
+ " <th>title</th>\n",
191
+ " <th>price</th>\n",
192
+ " <th>rating</th>\n",
193
+ " </tr>\n",
194
+ " </thead>\n",
195
+ " <tbody>\n",
196
+ " <tr>\n",
197
+ " <th>0</th>\n",
198
+ " <td>A Light in the Attic</td>\n",
199
+ " <td>51.77</td>\n",
200
+ " <td>Three</td>\n",
201
+ " </tr>\n",
202
+ " <tr>\n",
203
+ " <th>1</th>\n",
204
+ " <td>Tipping the Velvet</td>\n",
205
+ " <td>53.74</td>\n",
206
+ " <td>One</td>\n",
207
+ " </tr>\n",
208
+ " <tr>\n",
209
+ " <th>2</th>\n",
210
+ " <td>Soumission</td>\n",
211
+ " <td>50.10</td>\n",
212
+ " <td>One</td>\n",
213
+ " </tr>\n",
214
+ " <tr>\n",
215
+ " <th>3</th>\n",
216
+ " <td>Sharp Objects</td>\n",
217
+ " <td>47.82</td>\n",
218
+ " <td>Four</td>\n",
219
+ " </tr>\n",
220
+ " <tr>\n",
221
+ " <th>4</th>\n",
222
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
223
+ " <td>54.23</td>\n",
224
+ " <td>Five</td>\n",
225
+ " </tr>\n",
226
+ " </tbody>\n",
227
+ "</table>\n",
228
+ "</div>\n",
229
+ " <div class=\"colab-df-buttons\">\n",
230
+ "\n",
231
+ " <div class=\"colab-df-container\">\n",
232
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9abbc423-e6d8-4007-8c2d-6f00e62cbb32')\"\n",
233
+ " title=\"Convert this dataframe to an interactive table.\"\n",
234
+ " style=\"display:none;\">\n",
235
+ "\n",
236
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
237
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
238
+ " </svg>\n",
239
+ " </button>\n",
240
+ "\n",
241
+ " <style>\n",
242
+ " .colab-df-container {\n",
243
+ " display:flex;\n",
244
+ " gap: 12px;\n",
245
+ " }\n",
246
+ "\n",
247
+ " .colab-df-convert {\n",
248
+ " background-color: #E8F0FE;\n",
249
+ " border: none;\n",
250
+ " border-radius: 50%;\n",
251
+ " cursor: pointer;\n",
252
+ " display: none;\n",
253
+ " fill: #1967D2;\n",
254
+ " height: 32px;\n",
255
+ " padding: 0 0 0 0;\n",
256
+ " width: 32px;\n",
257
+ " }\n",
258
+ "\n",
259
+ " .colab-df-convert:hover {\n",
260
+ " background-color: #E2EBFA;\n",
261
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
262
+ " fill: #174EA6;\n",
263
+ " }\n",
264
+ "\n",
265
+ " .colab-df-buttons div {\n",
266
+ " margin-bottom: 4px;\n",
267
+ " }\n",
268
+ "\n",
269
+ " [theme=dark] .colab-df-convert {\n",
270
+ " background-color: #3B4455;\n",
271
+ " fill: #D2E3FC;\n",
272
+ " }\n",
273
+ "\n",
274
+ " [theme=dark] .colab-df-convert:hover {\n",
275
+ " background-color: #434B5C;\n",
276
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
277
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
278
+ " fill: #FFFFFF;\n",
279
+ " }\n",
280
+ " </style>\n",
281
+ "\n",
282
+ " <script>\n",
283
+ " const buttonEl =\n",
284
+ " document.querySelector('#df-9abbc423-e6d8-4007-8c2d-6f00e62cbb32 button.colab-df-convert');\n",
285
+ " buttonEl.style.display =\n",
286
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
287
+ "\n",
288
+ " async function convertToInteractive(key) {\n",
289
+ " const element = document.querySelector('#df-9abbc423-e6d8-4007-8c2d-6f00e62cbb32');\n",
290
+ " const dataTable =\n",
291
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
292
+ " [key], {});\n",
293
+ " if (!dataTable) return;\n",
294
+ "\n",
295
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
296
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
297
+ " + ' to learn more about interactive tables.';\n",
298
+ " element.innerHTML = '';\n",
299
+ " dataTable['output_type'] = 'display_data';\n",
300
+ " await google.colab.output.renderOutput(dataTable, element);\n",
301
+ " const docLink = document.createElement('div');\n",
302
+ " docLink.innerHTML = docLinkHtml;\n",
303
+ " element.appendChild(docLink);\n",
304
+ " }\n",
305
+ " </script>\n",
306
+ " </div>\n",
307
+ "\n",
308
+ "\n",
309
+ " </div>\n",
310
+ " </div>\n"
311
+ ],
312
+ "application/vnd.google.colaboratory.intrinsic+json": {
313
+ "type": "dataframe",
314
+ "variable_name": "df_books",
315
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
316
+ }
317
+ },
318
+ "metadata": {},
319
+ "execution_count": 4
320
+ }
321
+ ],
322
+ "source": [
323
+ "df_books = pd.DataFrame({\n",
324
+ " \"title\": titles,\n",
325
+ " \"price\": prices,\n",
326
+ " \"rating\": ratings\n",
327
+ "})\n",
328
+ "\n",
329
+ "df_books.head()"
330
+ ]
331
+ },
332
+ {
333
+ "cell_type": "markdown",
334
+ "metadata": {
335
+ "id": "duI5dv3CZYvF"
336
+ },
337
+ "source": [
338
+ "### *d. Save web-scraped dataframe either as a CSV or Excel file*"
339
+ ]
340
+ },
341
+ {
342
+ "cell_type": "code",
343
+ "execution_count": 5,
344
+ "metadata": {
345
+ "id": "lC1U_YHtZifh"
346
+ },
347
+ "outputs": [],
348
+ "source": [
349
+ "# 💾 Save to CSV\n",
350
+ "df_books.to_csv(\"books_data.csv\", index=False)\n",
351
+ "\n",
352
+ "# 💾 Or save to Excel\n",
353
+ "# df_books.to_excel(\"books_data.xlsx\", index=False)"
354
+ ]
355
+ },
356
+ {
357
+ "cell_type": "markdown",
358
+ "metadata": {
359
+ "id": "qMjRKMBQZlJi"
360
+ },
361
+ "source": [
362
+ "### *e. ✋🏻🛑⛔️ View first fiew lines*"
363
+ ]
364
+ },
365
+ {
366
+ "cell_type": "code",
367
+ "execution_count": 7,
368
+ "metadata": {
369
+ "colab": {
370
+ "base_uri": "https://localhost:8080/",
371
+ "height": 352
372
+ },
373
+ "id": "O_wIvTxYZqCK",
374
+ "outputId": "c5be92d3-fac0-4af0-993a-314ee714d8c4"
375
+ },
376
+ "outputs": [
377
+ {
378
+ "output_type": "execute_result",
379
+ "data": {
380
+ "text/plain": [
381
+ " title price rating\n",
382
+ "0 A Light in the Attic 51.77 Three\n",
383
+ "1 Tipping the Velvet 53.74 One\n",
384
+ "2 Soumission 50.10 One\n",
385
+ "3 Sharp Objects 47.82 Four\n",
386
+ "4 Sapiens: A Brief History of Humankind 54.23 Five\n",
387
+ "5 The Requiem Red 22.65 One\n",
388
+ "6 The Dirty Little Secrets of Getting Your Dream... 33.34 Four\n",
389
+ "7 The Coming Woman: A Novel Based on the Life of... 17.93 Three\n",
390
+ "8 The Boys in the Boat: Nine Americans and Their... 22.60 Four\n",
391
+ "9 The Black Maria 52.15 One"
392
+ ],
393
+ "text/html": [
394
+ "\n",
395
+ " <div id=\"df-f6f463fe-b2c3-426c-90b0-0336aad36d57\" class=\"colab-df-container\">\n",
396
+ " <div>\n",
397
+ "<style scoped>\n",
398
+ " .dataframe tbody tr th:only-of-type {\n",
399
+ " vertical-align: middle;\n",
400
+ " }\n",
401
+ "\n",
402
+ " .dataframe tbody tr th {\n",
403
+ " vertical-align: top;\n",
404
+ " }\n",
405
+ "\n",
406
+ " .dataframe thead th {\n",
407
+ " text-align: right;\n",
408
+ " }\n",
409
+ "</style>\n",
410
+ "<table border=\"1\" class=\"dataframe\">\n",
411
+ " <thead>\n",
412
+ " <tr style=\"text-align: right;\">\n",
413
+ " <th></th>\n",
414
+ " <th>title</th>\n",
415
+ " <th>price</th>\n",
416
+ " <th>rating</th>\n",
417
+ " </tr>\n",
418
+ " </thead>\n",
419
+ " <tbody>\n",
420
+ " <tr>\n",
421
+ " <th>0</th>\n",
422
+ " <td>A Light in the Attic</td>\n",
423
+ " <td>51.77</td>\n",
424
+ " <td>Three</td>\n",
425
+ " </tr>\n",
426
+ " <tr>\n",
427
+ " <th>1</th>\n",
428
+ " <td>Tipping the Velvet</td>\n",
429
+ " <td>53.74</td>\n",
430
+ " <td>One</td>\n",
431
+ " </tr>\n",
432
+ " <tr>\n",
433
+ " <th>2</th>\n",
434
+ " <td>Soumission</td>\n",
435
+ " <td>50.10</td>\n",
436
+ " <td>One</td>\n",
437
+ " </tr>\n",
438
+ " <tr>\n",
439
+ " <th>3</th>\n",
440
+ " <td>Sharp Objects</td>\n",
441
+ " <td>47.82</td>\n",
442
+ " <td>Four</td>\n",
443
+ " </tr>\n",
444
+ " <tr>\n",
445
+ " <th>4</th>\n",
446
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
447
+ " <td>54.23</td>\n",
448
+ " <td>Five</td>\n",
449
+ " </tr>\n",
450
+ " <tr>\n",
451
+ " <th>5</th>\n",
452
+ " <td>The Requiem Red</td>\n",
453
+ " <td>22.65</td>\n",
454
+ " <td>One</td>\n",
455
+ " </tr>\n",
456
+ " <tr>\n",
457
+ " <th>6</th>\n",
458
+ " <td>The Dirty Little Secrets of Getting Your Dream...</td>\n",
459
+ " <td>33.34</td>\n",
460
+ " <td>Four</td>\n",
461
+ " </tr>\n",
462
+ " <tr>\n",
463
+ " <th>7</th>\n",
464
+ " <td>The Coming Woman: A Novel Based on the Life of...</td>\n",
465
+ " <td>17.93</td>\n",
466
+ " <td>Three</td>\n",
467
+ " </tr>\n",
468
+ " <tr>\n",
469
+ " <th>8</th>\n",
470
+ " <td>The Boys in the Boat: Nine Americans and Their...</td>\n",
471
+ " <td>22.60</td>\n",
472
+ " <td>Four</td>\n",
473
+ " </tr>\n",
474
+ " <tr>\n",
475
+ " <th>9</th>\n",
476
+ " <td>The Black Maria</td>\n",
477
+ " <td>52.15</td>\n",
478
+ " <td>One</td>\n",
479
+ " </tr>\n",
480
+ " </tbody>\n",
481
+ "</table>\n",
482
+ "</div>\n",
483
+ " <div class=\"colab-df-buttons\">\n",
484
+ "\n",
485
+ " <div class=\"colab-df-container\">\n",
486
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f6f463fe-b2c3-426c-90b0-0336aad36d57')\"\n",
487
+ " title=\"Convert this dataframe to an interactive table.\"\n",
488
+ " style=\"display:none;\">\n",
489
+ "\n",
490
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
491
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
492
+ " </svg>\n",
493
+ " </button>\n",
494
+ "\n",
495
+ " <style>\n",
496
+ " .colab-df-container {\n",
497
+ " display:flex;\n",
498
+ " gap: 12px;\n",
499
+ " }\n",
500
+ "\n",
501
+ " .colab-df-convert {\n",
502
+ " background-color: #E8F0FE;\n",
503
+ " border: none;\n",
504
+ " border-radius: 50%;\n",
505
+ " cursor: pointer;\n",
506
+ " display: none;\n",
507
+ " fill: #1967D2;\n",
508
+ " height: 32px;\n",
509
+ " padding: 0 0 0 0;\n",
510
+ " width: 32px;\n",
511
+ " }\n",
512
+ "\n",
513
+ " .colab-df-convert:hover {\n",
514
+ " background-color: #E2EBFA;\n",
515
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
516
+ " fill: #174EA6;\n",
517
+ " }\n",
518
+ "\n",
519
+ " .colab-df-buttons div {\n",
520
+ " margin-bottom: 4px;\n",
521
+ " }\n",
522
+ "\n",
523
+ " [theme=dark] .colab-df-convert {\n",
524
+ " background-color: #3B4455;\n",
525
+ " fill: #D2E3FC;\n",
526
+ " }\n",
527
+ "\n",
528
+ " [theme=dark] .colab-df-convert:hover {\n",
529
+ " background-color: #434B5C;\n",
530
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
531
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
532
+ " fill: #FFFFFF;\n",
533
+ " }\n",
534
+ " </style>\n",
535
+ "\n",
536
+ " <script>\n",
537
+ " const buttonEl =\n",
538
+ " document.querySelector('#df-f6f463fe-b2c3-426c-90b0-0336aad36d57 button.colab-df-convert');\n",
539
+ " buttonEl.style.display =\n",
540
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
541
+ "\n",
542
+ " async function convertToInteractive(key) {\n",
543
+ " const element = document.querySelector('#df-f6f463fe-b2c3-426c-90b0-0336aad36d57');\n",
544
+ " const dataTable =\n",
545
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
546
+ " [key], {});\n",
547
+ " if (!dataTable) return;\n",
548
+ "\n",
549
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
550
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
551
+ " + ' to learn more about interactive tables.';\n",
552
+ " element.innerHTML = '';\n",
553
+ " dataTable['output_type'] = 'display_data';\n",
554
+ " await google.colab.output.renderOutput(dataTable, element);\n",
555
+ " const docLink = document.createElement('div');\n",
556
+ " docLink.innerHTML = docLinkHtml;\n",
557
+ " element.appendChild(docLink);\n",
558
+ " }\n",
559
+ " </script>\n",
560
+ " </div>\n",
561
+ "\n",
562
+ "\n",
563
+ " </div>\n",
564
+ " </div>\n"
565
+ ],
566
+ "application/vnd.google.colaboratory.intrinsic+json": {
567
+ "type": "dataframe",
568
+ "variable_name": "df_books",
569
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
570
+ }
571
+ },
572
+ "metadata": {},
573
+ "execution_count": 7
574
+ }
575
+ ],
576
+ "source": [
577
+ "df_books.head(10)"
578
+ ]
579
+ },
580
+ {
581
+ "cell_type": "markdown",
582
+ "metadata": {
583
+ "id": "p-1Pr2szaqLk"
584
+ },
585
+ "source": [
586
+ "## **3.** 🧩 Create a meaningful connection between real & synthetic datasets"
587
+ ]
588
+ },
589
+ {
590
+ "cell_type": "markdown",
591
+ "metadata": {
592
+ "id": "SIaJUGIpaH4V"
593
+ },
594
+ "source": [
595
+ "### *a. Initial setup*"
596
+ ]
597
+ },
598
+ {
599
+ "cell_type": "code",
600
+ "execution_count": 8,
601
+ "metadata": {
602
+ "id": "-gPXGcRPuV_9"
603
+ },
604
+ "outputs": [],
605
+ "source": [
606
+ "import numpy as np\n",
607
+ "import random\n",
608
+ "from datetime import datetime\n",
609
+ "import warnings\n",
610
+ "\n",
611
+ "warnings.filterwarnings(\"ignore\")\n",
612
+ "random.seed(2025)\n",
613
+ "np.random.seed(2025)"
614
+ ]
615
+ },
616
+ {
617
+ "cell_type": "markdown",
618
+ "metadata": {
619
+ "id": "pY4yCoIuaQqp"
620
+ },
621
+ "source": [
622
+ "### *b. Generate popularity scores based on rating (with some randomness) with a generate_popularity_score function*"
623
+ ]
624
+ },
625
+ {
626
+ "cell_type": "code",
627
+ "execution_count": 9,
628
+ "metadata": {
629
+ "id": "mnd5hdAbaNjz"
630
+ },
631
+ "outputs": [],
632
+ "source": [
633
+ "def generate_popularity_score(rating):\n",
634
+ " base = {\"One\": 2, \"Two\": 3, \"Three\": 3, \"Four\": 4, \"Five\": 4}.get(rating, 3)\n",
635
+ " trend_factor = random.choices([-1, 0, 1], weights=[1, 3, 2])[0]\n",
636
+ " return int(np.clip(base + trend_factor, 1, 5))"
637
+ ]
638
+ },
639
+ {
640
+ "cell_type": "markdown",
641
+ "metadata": {
642
+ "id": "n4-TaNTFgPak"
643
+ },
644
+ "source": [
645
+ "### *c. ✋🏻🛑⛔️ Run the function to create a \"popularity_score\" column from \"rating\"*"
646
+ ]
647
+ },
648
+ {
649
+ "cell_type": "code",
650
+ "execution_count": 10,
651
+ "metadata": {
652
+ "id": "V-G3OCUCgR07",
653
+ "colab": {
654
+ "base_uri": "https://localhost:8080/",
655
+ "height": 201
656
+ },
657
+ "outputId": "721c37fc-9478-4e8a-898a-b658215a8eed"
658
+ },
659
+ "outputs": [
660
+ {
661
+ "output_type": "execute_result",
662
+ "data": {
663
+ "text/plain": [
664
+ " title price rating popularity_score\n",
665
+ "0 A Light in the Attic 51.77 Three 3\n",
666
+ "1 Tipping the Velvet 53.74 One 2\n",
667
+ "2 Soumission 50.10 One 2\n",
668
+ "3 Sharp Objects 47.82 Four 4\n",
669
+ "4 Sapiens: A Brief History of Humankind 54.23 Five 3"
670
+ ],
671
+ "text/html": [
672
+ "\n",
673
+ " <div id=\"df-4cc5a181-6782-4b4b-940e-ccac508a41a6\" class=\"colab-df-container\">\n",
674
+ " <div>\n",
675
+ "<style scoped>\n",
676
+ " .dataframe tbody tr th:only-of-type {\n",
677
+ " vertical-align: middle;\n",
678
+ " }\n",
679
+ "\n",
680
+ " .dataframe tbody tr th {\n",
681
+ " vertical-align: top;\n",
682
+ " }\n",
683
+ "\n",
684
+ " .dataframe thead th {\n",
685
+ " text-align: right;\n",
686
+ " }\n",
687
+ "</style>\n",
688
+ "<table border=\"1\" class=\"dataframe\">\n",
689
+ " <thead>\n",
690
+ " <tr style=\"text-align: right;\">\n",
691
+ " <th></th>\n",
692
+ " <th>title</th>\n",
693
+ " <th>price</th>\n",
694
+ " <th>rating</th>\n",
695
+ " <th>popularity_score</th>\n",
696
+ " </tr>\n",
697
+ " </thead>\n",
698
+ " <tbody>\n",
699
+ " <tr>\n",
700
+ " <th>0</th>\n",
701
+ " <td>A Light in the Attic</td>\n",
702
+ " <td>51.77</td>\n",
703
+ " <td>Three</td>\n",
704
+ " <td>3</td>\n",
705
+ " </tr>\n",
706
+ " <tr>\n",
707
+ " <th>1</th>\n",
708
+ " <td>Tipping the Velvet</td>\n",
709
+ " <td>53.74</td>\n",
710
+ " <td>One</td>\n",
711
+ " <td>2</td>\n",
712
+ " </tr>\n",
713
+ " <tr>\n",
714
+ " <th>2</th>\n",
715
+ " <td>Soumission</td>\n",
716
+ " <td>50.10</td>\n",
717
+ " <td>One</td>\n",
718
+ " <td>2</td>\n",
719
+ " </tr>\n",
720
+ " <tr>\n",
721
+ " <th>3</th>\n",
722
+ " <td>Sharp Objects</td>\n",
723
+ " <td>47.82</td>\n",
724
+ " <td>Four</td>\n",
725
+ " <td>4</td>\n",
726
+ " </tr>\n",
727
+ " <tr>\n",
728
+ " <th>4</th>\n",
729
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
730
+ " <td>54.23</td>\n",
731
+ " <td>Five</td>\n",
732
+ " <td>3</td>\n",
733
+ " </tr>\n",
734
+ " </tbody>\n",
735
+ "</table>\n",
736
+ "</div>\n",
737
+ " <div class=\"colab-df-buttons\">\n",
738
+ "\n",
739
+ " <div class=\"colab-df-container\">\n",
740
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4cc5a181-6782-4b4b-940e-ccac508a41a6')\"\n",
741
+ " title=\"Convert this dataframe to an interactive table.\"\n",
742
+ " style=\"display:none;\">\n",
743
+ "\n",
744
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
745
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
746
+ " </svg>\n",
747
+ " </button>\n",
748
+ "\n",
749
+ " <style>\n",
750
+ " .colab-df-container {\n",
751
+ " display:flex;\n",
752
+ " gap: 12px;\n",
753
+ " }\n",
754
+ "\n",
755
+ " .colab-df-convert {\n",
756
+ " background-color: #E8F0FE;\n",
757
+ " border: none;\n",
758
+ " border-radius: 50%;\n",
759
+ " cursor: pointer;\n",
760
+ " display: none;\n",
761
+ " fill: #1967D2;\n",
762
+ " height: 32px;\n",
763
+ " padding: 0 0 0 0;\n",
764
+ " width: 32px;\n",
765
+ " }\n",
766
+ "\n",
767
+ " .colab-df-convert:hover {\n",
768
+ " background-color: #E2EBFA;\n",
769
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
770
+ " fill: #174EA6;\n",
771
+ " }\n",
772
+ "\n",
773
+ " .colab-df-buttons div {\n",
774
+ " margin-bottom: 4px;\n",
775
+ " }\n",
776
+ "\n",
777
+ " [theme=dark] .colab-df-convert {\n",
778
+ " background-color: #3B4455;\n",
779
+ " fill: #D2E3FC;\n",
780
+ " }\n",
781
+ "\n",
782
+ " [theme=dark] .colab-df-convert:hover {\n",
783
+ " background-color: #434B5C;\n",
784
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
785
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
786
+ " fill: #FFFFFF;\n",
787
+ " }\n",
788
+ " </style>\n",
789
+ "\n",
790
+ " <script>\n",
791
+ " const buttonEl =\n",
792
+ " document.querySelector('#df-4cc5a181-6782-4b4b-940e-ccac508a41a6 button.colab-df-convert');\n",
793
+ " buttonEl.style.display =\n",
794
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
795
+ "\n",
796
+ " async function convertToInteractive(key) {\n",
797
+ " const element = document.querySelector('#df-4cc5a181-6782-4b4b-940e-ccac508a41a6');\n",
798
+ " const dataTable =\n",
799
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
800
+ " [key], {});\n",
801
+ " if (!dataTable) return;\n",
802
+ "\n",
803
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
804
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
805
+ " + ' to learn more about interactive tables.';\n",
806
+ " element.innerHTML = '';\n",
807
+ " dataTable['output_type'] = 'display_data';\n",
808
+ " await google.colab.output.renderOutput(dataTable, element);\n",
809
+ " const docLink = document.createElement('div');\n",
810
+ " docLink.innerHTML = docLinkHtml;\n",
811
+ " element.appendChild(docLink);\n",
812
+ " }\n",
813
+ " </script>\n",
814
+ " </div>\n",
815
+ "\n",
816
+ "\n",
817
+ " </div>\n",
818
+ " </div>\n"
819
+ ],
820
+ "application/vnd.google.colaboratory.intrinsic+json": {
821
+ "type": "dataframe",
822
+ "variable_name": "df_books",
823
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
824
+ }
825
+ },
826
+ "metadata": {},
827
+ "execution_count": 10
828
+ }
829
+ ],
830
+ "source": [
831
+ "df_books[\"popularity_score\"] = df_books[\"rating\"].apply(generate_popularity_score)\n",
832
+ "df_books.head()"
833
+ ]
834
+ },
835
+ {
836
+ "cell_type": "markdown",
837
+ "metadata": {
838
+ "id": "HnngRNTgacYt"
839
+ },
840
+ "source": [
841
+ "### *d. Decide on the sentiment_label based on the popularity score with a get_sentiment function*"
842
+ ]
843
+ },
844
+ {
845
+ "cell_type": "code",
846
+ "execution_count": 11,
847
+ "metadata": {
848
+ "id": "kUtWmr8maZLZ"
849
+ },
850
+ "outputs": [],
851
+ "source": [
852
+ "def get_sentiment(popularity_score):\n",
853
+ " if popularity_score <= 2:\n",
854
+ " return \"negative\"\n",
855
+ " elif popularity_score == 3:\n",
856
+ " return \"neutral\"\n",
857
+ " else:\n",
858
+ " return \"positive\""
859
+ ]
860
+ },
861
+ {
862
+ "cell_type": "markdown",
863
+ "metadata": {
864
+ "id": "HF9F9HIzgT7Z"
865
+ },
866
+ "source": [
867
+ "### *e. ✋🏻🛑⛔️ Run the function to create a \"sentiment_label\" column from \"popularity_score\"*"
868
+ ]
869
+ },
870
+ {
871
+ "cell_type": "code",
872
+ "execution_count": 12,
873
+ "metadata": {
874
+ "id": "tafQj8_7gYCG",
875
+ "colab": {
876
+ "base_uri": "https://localhost:8080/",
877
+ "height": 201
878
+ },
879
+ "outputId": "fcda61c9-c0ca-4eae-fbfa-26490115406f"
880
+ },
881
+ "outputs": [
882
+ {
883
+ "output_type": "execute_result",
884
+ "data": {
885
+ "text/plain": [
886
+ " title price rating popularity_score \\\n",
887
+ "0 A Light in the Attic 51.77 Three 3 \n",
888
+ "1 Tipping the Velvet 53.74 One 2 \n",
889
+ "2 Soumission 50.10 One 2 \n",
890
+ "3 Sharp Objects 47.82 Four 4 \n",
891
+ "4 Sapiens: A Brief History of Humankind 54.23 Five 3 \n",
892
+ "\n",
893
+ " sentiment_label \n",
894
+ "0 neutral \n",
895
+ "1 negative \n",
896
+ "2 negative \n",
897
+ "3 positive \n",
898
+ "4 neutral "
899
+ ],
900
+ "text/html": [
901
+ "\n",
902
+ " <div id=\"df-00462008-2489-43a1-a170-bdb0e3aacbee\" class=\"colab-df-container\">\n",
903
+ " <div>\n",
904
+ "<style scoped>\n",
905
+ " .dataframe tbody tr th:only-of-type {\n",
906
+ " vertical-align: middle;\n",
907
+ " }\n",
908
+ "\n",
909
+ " .dataframe tbody tr th {\n",
910
+ " vertical-align: top;\n",
911
+ " }\n",
912
+ "\n",
913
+ " .dataframe thead th {\n",
914
+ " text-align: right;\n",
915
+ " }\n",
916
+ "</style>\n",
917
+ "<table border=\"1\" class=\"dataframe\">\n",
918
+ " <thead>\n",
919
+ " <tr style=\"text-align: right;\">\n",
920
+ " <th></th>\n",
921
+ " <th>title</th>\n",
922
+ " <th>price</th>\n",
923
+ " <th>rating</th>\n",
924
+ " <th>popularity_score</th>\n",
925
+ " <th>sentiment_label</th>\n",
926
+ " </tr>\n",
927
+ " </thead>\n",
928
+ " <tbody>\n",
929
+ " <tr>\n",
930
+ " <th>0</th>\n",
931
+ " <td>A Light in the Attic</td>\n",
932
+ " <td>51.77</td>\n",
933
+ " <td>Three</td>\n",
934
+ " <td>3</td>\n",
935
+ " <td>neutral</td>\n",
936
+ " </tr>\n",
937
+ " <tr>\n",
938
+ " <th>1</th>\n",
939
+ " <td>Tipping the Velvet</td>\n",
940
+ " <td>53.74</td>\n",
941
+ " <td>One</td>\n",
942
+ " <td>2</td>\n",
943
+ " <td>negative</td>\n",
944
+ " </tr>\n",
945
+ " <tr>\n",
946
+ " <th>2</th>\n",
947
+ " <td>Soumission</td>\n",
948
+ " <td>50.10</td>\n",
949
+ " <td>One</td>\n",
950
+ " <td>2</td>\n",
951
+ " <td>negative</td>\n",
952
+ " </tr>\n",
953
+ " <tr>\n",
954
+ " <th>3</th>\n",
955
+ " <td>Sharp Objects</td>\n",
956
+ " <td>47.82</td>\n",
957
+ " <td>Four</td>\n",
958
+ " <td>4</td>\n",
959
+ " <td>positive</td>\n",
960
+ " </tr>\n",
961
+ " <tr>\n",
962
+ " <th>4</th>\n",
963
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
964
+ " <td>54.23</td>\n",
965
+ " <td>Five</td>\n",
966
+ " <td>3</td>\n",
967
+ " <td>neutral</td>\n",
968
+ " </tr>\n",
969
+ " </tbody>\n",
970
+ "</table>\n",
971
+ "</div>\n",
972
+ " <div class=\"colab-df-buttons\">\n",
973
+ "\n",
974
+ " <div class=\"colab-df-container\">\n",
975
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-00462008-2489-43a1-a170-bdb0e3aacbee')\"\n",
976
+ " title=\"Convert this dataframe to an interactive table.\"\n",
977
+ " style=\"display:none;\">\n",
978
+ "\n",
979
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
980
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
981
+ " </svg>\n",
982
+ " </button>\n",
983
+ "\n",
984
+ " <style>\n",
985
+ " .colab-df-container {\n",
986
+ " display:flex;\n",
987
+ " gap: 12px;\n",
988
+ " }\n",
989
+ "\n",
990
+ " .colab-df-convert {\n",
991
+ " background-color: #E8F0FE;\n",
992
+ " border: none;\n",
993
+ " border-radius: 50%;\n",
994
+ " cursor: pointer;\n",
995
+ " display: none;\n",
996
+ " fill: #1967D2;\n",
997
+ " height: 32px;\n",
998
+ " padding: 0 0 0 0;\n",
999
+ " width: 32px;\n",
1000
+ " }\n",
1001
+ "\n",
1002
+ " .colab-df-convert:hover {\n",
1003
+ " background-color: #E2EBFA;\n",
1004
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1005
+ " fill: #174EA6;\n",
1006
+ " }\n",
1007
+ "\n",
1008
+ " .colab-df-buttons div {\n",
1009
+ " margin-bottom: 4px;\n",
1010
+ " }\n",
1011
+ "\n",
1012
+ " [theme=dark] .colab-df-convert {\n",
1013
+ " background-color: #3B4455;\n",
1014
+ " fill: #D2E3FC;\n",
1015
+ " }\n",
1016
+ "\n",
1017
+ " [theme=dark] .colab-df-convert:hover {\n",
1018
+ " background-color: #434B5C;\n",
1019
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1020
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1021
+ " fill: #FFFFFF;\n",
1022
+ " }\n",
1023
+ " </style>\n",
1024
+ "\n",
1025
+ " <script>\n",
1026
+ " const buttonEl =\n",
1027
+ " document.querySelector('#df-00462008-2489-43a1-a170-bdb0e3aacbee button.colab-df-convert');\n",
1028
+ " buttonEl.style.display =\n",
1029
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1030
+ "\n",
1031
+ " async function convertToInteractive(key) {\n",
1032
+ " const element = document.querySelector('#df-00462008-2489-43a1-a170-bdb0e3aacbee');\n",
1033
+ " const dataTable =\n",
1034
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1035
+ " [key], {});\n",
1036
+ " if (!dataTable) return;\n",
1037
+ "\n",
1038
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1039
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1040
+ " + ' to learn more about interactive tables.';\n",
1041
+ " element.innerHTML = '';\n",
1042
+ " dataTable['output_type'] = 'display_data';\n",
1043
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1044
+ " const docLink = document.createElement('div');\n",
1045
+ " docLink.innerHTML = docLinkHtml;\n",
1046
+ " element.appendChild(docLink);\n",
1047
+ " }\n",
1048
+ " </script>\n",
1049
+ " </div>\n",
1050
+ "\n",
1051
+ "\n",
1052
+ " </div>\n",
1053
+ " </div>\n"
1054
+ ],
1055
+ "application/vnd.google.colaboratory.intrinsic+json": {
1056
+ "type": "dataframe",
1057
+ "variable_name": "df_books",
1058
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1059
+ }
1060
+ },
1061
+ "metadata": {},
1062
+ "execution_count": 12
1063
+ }
1064
+ ],
1065
+ "source": [
1066
+ "df_books[\"sentiment_label\"] = df_books[\"popularity_score\"].apply(get_sentiment)\n",
1067
+ "df_books.head()"
1068
+ ]
1069
+ },
1070
+ {
1071
+ "cell_type": "markdown",
1072
+ "metadata": {
1073
+ "id": "T8AdKkmASq9a"
1074
+ },
1075
+ "source": [
1076
+ "## **4.** 📈 Generate synthetic book sales data of 18 months"
1077
+ ]
1078
+ },
1079
+ {
1080
+ "cell_type": "markdown",
1081
+ "metadata": {
1082
+ "id": "OhXbdGD5fH0c"
1083
+ },
1084
+ "source": [
1085
+ "### *a. Create a generate_sales_profit function that would generate sales patterns based on sentiment_label (with some randomness)*"
1086
+ ]
1087
+ },
1088
+ {
1089
+ "cell_type": "code",
1090
+ "execution_count": 13,
1091
+ "metadata": {
1092
+ "id": "qkVhYPXGbgEn"
1093
+ },
1094
+ "outputs": [],
1095
+ "source": [
1096
+ "def generate_sales_profile(sentiment):\n",
1097
+ " months = pd.date_range(end=datetime.today(), periods=18, freq=\"M\")\n",
1098
+ "\n",
1099
+ " if sentiment == \"positive\":\n",
1100
+ " base = random.randint(200, 300)\n",
1101
+ " trend = np.linspace(base, base + random.randint(20, 60), len(months))\n",
1102
+ " elif sentiment == \"negative\":\n",
1103
+ " base = random.randint(20, 80)\n",
1104
+ " trend = np.linspace(base, base - random.randint(10, 30), len(months))\n",
1105
+ " else: # neutral\n",
1106
+ " base = random.randint(80, 160)\n",
1107
+ " trend = np.full(len(months), base + random.randint(-10, 10))\n",
1108
+ "\n",
1109
+ " seasonality = 10 * np.sin(np.linspace(0, 3 * np.pi, len(months)))\n",
1110
+ " noise = np.random.normal(0, 5, len(months))\n",
1111
+ " monthly_sales = np.clip(trend + seasonality + noise, a_min=0, a_max=None).astype(int)\n",
1112
+ "\n",
1113
+ " return list(zip(months.strftime(\"%Y-%m\"), monthly_sales))"
1114
+ ]
1115
+ },
1116
+ {
1117
+ "cell_type": "markdown",
1118
+ "metadata": {
1119
+ "id": "L2ak1HlcgoTe"
1120
+ },
1121
+ "source": [
1122
+ "### *b. Run the function as part of building sales_data*"
1123
+ ]
1124
+ },
1125
+ {
1126
+ "cell_type": "code",
1127
+ "execution_count": 14,
1128
+ "metadata": {
1129
+ "id": "SlJ24AUafoDB"
1130
+ },
1131
+ "outputs": [],
1132
+ "source": [
1133
+ "sales_data = []\n",
1134
+ "for _, row in df_books.iterrows():\n",
1135
+ " records = generate_sales_profile(row[\"sentiment_label\"])\n",
1136
+ " for month, units in records:\n",
1137
+ " sales_data.append({\n",
1138
+ " \"title\": row[\"title\"],\n",
1139
+ " \"month\": month,\n",
1140
+ " \"units_sold\": units,\n",
1141
+ " \"sentiment_label\": row[\"sentiment_label\"]\n",
1142
+ " })"
1143
+ ]
1144
+ },
1145
+ {
1146
+ "cell_type": "markdown",
1147
+ "metadata": {
1148
+ "id": "4IXZKcCSgxnq"
1149
+ },
1150
+ "source": [
1151
+ "### *c. ✋🏻🛑⛔️ Create a df_sales DataFrame from sales_data*"
1152
+ ]
1153
+ },
1154
+ {
1155
+ "cell_type": "code",
1156
+ "execution_count": 15,
1157
+ "metadata": {
1158
+ "id": "wcN6gtiZg-ws",
1159
+ "colab": {
1160
+ "base_uri": "https://localhost:8080/",
1161
+ "height": 201
1162
+ },
1163
+ "outputId": "27be0a8c-7b10-402d-8bec-7c24cf9bac75"
1164
+ },
1165
+ "outputs": [
1166
+ {
1167
+ "output_type": "execute_result",
1168
+ "data": {
1169
+ "text/plain": [
1170
+ " title month units_sold sentiment_label\n",
1171
+ "0 A Light in the Attic 2024-08 100 neutral\n",
1172
+ "1 A Light in the Attic 2024-09 109 neutral\n",
1173
+ "2 A Light in the Attic 2024-10 102 neutral\n",
1174
+ "3 A Light in the Attic 2024-11 107 neutral\n",
1175
+ "4 A Light in the Attic 2024-12 108 neutral"
1176
+ ],
1177
+ "text/html": [
1178
+ "\n",
1179
+ " <div id=\"df-a779c3b7-b0ce-4b69-8e05-9f738e4bff9c\" class=\"colab-df-container\">\n",
1180
+ " <div>\n",
1181
+ "<style scoped>\n",
1182
+ " .dataframe tbody tr th:only-of-type {\n",
1183
+ " vertical-align: middle;\n",
1184
+ " }\n",
1185
+ "\n",
1186
+ " .dataframe tbody tr th {\n",
1187
+ " vertical-align: top;\n",
1188
+ " }\n",
1189
+ "\n",
1190
+ " .dataframe thead th {\n",
1191
+ " text-align: right;\n",
1192
+ " }\n",
1193
+ "</style>\n",
1194
+ "<table border=\"1\" class=\"dataframe\">\n",
1195
+ " <thead>\n",
1196
+ " <tr style=\"text-align: right;\">\n",
1197
+ " <th></th>\n",
1198
+ " <th>title</th>\n",
1199
+ " <th>month</th>\n",
1200
+ " <th>units_sold</th>\n",
1201
+ " <th>sentiment_label</th>\n",
1202
+ " </tr>\n",
1203
+ " </thead>\n",
1204
+ " <tbody>\n",
1205
+ " <tr>\n",
1206
+ " <th>0</th>\n",
1207
+ " <td>A Light in the Attic</td>\n",
1208
+ " <td>2024-08</td>\n",
1209
+ " <td>100</td>\n",
1210
+ " <td>neutral</td>\n",
1211
+ " </tr>\n",
1212
+ " <tr>\n",
1213
+ " <th>1</th>\n",
1214
+ " <td>A Light in the Attic</td>\n",
1215
+ " <td>2024-09</td>\n",
1216
+ " <td>109</td>\n",
1217
+ " <td>neutral</td>\n",
1218
+ " </tr>\n",
1219
+ " <tr>\n",
1220
+ " <th>2</th>\n",
1221
+ " <td>A Light in the Attic</td>\n",
1222
+ " <td>2024-10</td>\n",
1223
+ " <td>102</td>\n",
1224
+ " <td>neutral</td>\n",
1225
+ " </tr>\n",
1226
+ " <tr>\n",
1227
+ " <th>3</th>\n",
1228
+ " <td>A Light in the Attic</td>\n",
1229
+ " <td>2024-11</td>\n",
1230
+ " <td>107</td>\n",
1231
+ " <td>neutral</td>\n",
1232
+ " </tr>\n",
1233
+ " <tr>\n",
1234
+ " <th>4</th>\n",
1235
+ " <td>A Light in the Attic</td>\n",
1236
+ " <td>2024-12</td>\n",
1237
+ " <td>108</td>\n",
1238
+ " <td>neutral</td>\n",
1239
+ " </tr>\n",
1240
+ " </tbody>\n",
1241
+ "</table>\n",
1242
+ "</div>\n",
1243
+ " <div class=\"colab-df-buttons\">\n",
1244
+ "\n",
1245
+ " <div class=\"colab-df-container\">\n",
1246
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a779c3b7-b0ce-4b69-8e05-9f738e4bff9c')\"\n",
1247
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1248
+ " style=\"display:none;\">\n",
1249
+ "\n",
1250
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1251
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1252
+ " </svg>\n",
1253
+ " </button>\n",
1254
+ "\n",
1255
+ " <style>\n",
1256
+ " .colab-df-container {\n",
1257
+ " display:flex;\n",
1258
+ " gap: 12px;\n",
1259
+ " }\n",
1260
+ "\n",
1261
+ " .colab-df-convert {\n",
1262
+ " background-color: #E8F0FE;\n",
1263
+ " border: none;\n",
1264
+ " border-radius: 50%;\n",
1265
+ " cursor: pointer;\n",
1266
+ " display: none;\n",
1267
+ " fill: #1967D2;\n",
1268
+ " height: 32px;\n",
1269
+ " padding: 0 0 0 0;\n",
1270
+ " width: 32px;\n",
1271
+ " }\n",
1272
+ "\n",
1273
+ " .colab-df-convert:hover {\n",
1274
+ " background-color: #E2EBFA;\n",
1275
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1276
+ " fill: #174EA6;\n",
1277
+ " }\n",
1278
+ "\n",
1279
+ " .colab-df-buttons div {\n",
1280
+ " margin-bottom: 4px;\n",
1281
+ " }\n",
1282
+ "\n",
1283
+ " [theme=dark] .colab-df-convert {\n",
1284
+ " background-color: #3B4455;\n",
1285
+ " fill: #D2E3FC;\n",
1286
+ " }\n",
1287
+ "\n",
1288
+ " [theme=dark] .colab-df-convert:hover {\n",
1289
+ " background-color: #434B5C;\n",
1290
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1291
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1292
+ " fill: #FFFFFF;\n",
1293
+ " }\n",
1294
+ " </style>\n",
1295
+ "\n",
1296
+ " <script>\n",
1297
+ " const buttonEl =\n",
1298
+ " document.querySelector('#df-a779c3b7-b0ce-4b69-8e05-9f738e4bff9c button.colab-df-convert');\n",
1299
+ " buttonEl.style.display =\n",
1300
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1301
+ "\n",
1302
+ " async function convertToInteractive(key) {\n",
1303
+ " const element = document.querySelector('#df-a779c3b7-b0ce-4b69-8e05-9f738e4bff9c');\n",
1304
+ " const dataTable =\n",
1305
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1306
+ " [key], {});\n",
1307
+ " if (!dataTable) return;\n",
1308
+ "\n",
1309
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1310
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1311
+ " + ' to learn more about interactive tables.';\n",
1312
+ " element.innerHTML = '';\n",
1313
+ " dataTable['output_type'] = 'display_data';\n",
1314
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1315
+ " const docLink = document.createElement('div');\n",
1316
+ " docLink.innerHTML = docLinkHtml;\n",
1317
+ " element.appendChild(docLink);\n",
1318
+ " }\n",
1319
+ " </script>\n",
1320
+ " </div>\n",
1321
+ "\n",
1322
+ "\n",
1323
+ " </div>\n",
1324
+ " </div>\n"
1325
+ ],
1326
+ "application/vnd.google.colaboratory.intrinsic+json": {
1327
+ "type": "dataframe",
1328
+ "variable_name": "df_sales",
1329
+ "summary": "{\n \"name\": \"df_sales\",\n \"rows\": 18000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"month\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 18,\n \"samples\": [\n \"2024-08\",\n \"2024-09\",\n \"2025-04\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"units_sold\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 98,\n \"min\": 0,\n \"max\": 362,\n \"num_unique_values\": 354,\n \"samples\": [\n 214,\n 289,\n 205\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1330
+ }
1331
+ },
1332
+ "metadata": {},
1333
+ "execution_count": 15
1334
+ }
1335
+ ],
1336
+ "source": [
1337
+ "df_sales = pd.DataFrame(sales_data)\n",
1338
+ "df_sales.head()"
1339
+ ]
1340
+ },
1341
+ {
1342
+ "cell_type": "markdown",
1343
+ "metadata": {
1344
+ "id": "EhIjz9WohAmZ"
1345
+ },
1346
+ "source": [
1347
+ "### *d. Save df_sales as synthetic_sales_data.csv & view first few lines*"
1348
+ ]
1349
+ },
1350
+ {
1351
+ "cell_type": "code",
1352
+ "execution_count": 16,
1353
+ "metadata": {
1354
+ "colab": {
1355
+ "base_uri": "https://localhost:8080/"
1356
+ },
1357
+ "id": "MzbZvLcAhGaH",
1358
+ "outputId": "5eb66f96-a814-4cc4-964d-28e620fd78bb"
1359
+ },
1360
+ "outputs": [
1361
+ {
1362
+ "output_type": "stream",
1363
+ "name": "stdout",
1364
+ "text": [
1365
+ " title month units_sold sentiment_label\n",
1366
+ "0 A Light in the Attic 2024-08 100 neutral\n",
1367
+ "1 A Light in the Attic 2024-09 109 neutral\n",
1368
+ "2 A Light in the Attic 2024-10 102 neutral\n",
1369
+ "3 A Light in the Attic 2024-11 107 neutral\n",
1370
+ "4 A Light in the Attic 2024-12 108 neutral\n"
1371
+ ]
1372
+ }
1373
+ ],
1374
+ "source": [
1375
+ "df_sales.to_csv(\"synthetic_sales_data.csv\", index=False)\n",
1376
+ "\n",
1377
+ "print(df_sales.head())"
1378
+ ]
1379
+ },
1380
+ {
1381
+ "cell_type": "markdown",
1382
+ "metadata": {
1383
+ "id": "7g9gqBgQMtJn"
1384
+ },
1385
+ "source": [
1386
+ "## **5.** 🎯 Generate synthetic customer reviews"
1387
+ ]
1388
+ },
1389
+ {
1390
+ "cell_type": "markdown",
1391
+ "metadata": {
1392
+ "id": "Gi4y9M9KuDWx"
1393
+ },
1394
+ "source": [
1395
+ "### *a. ✋🏻🛑⛔️ Ask ChatGPT to create a list of 50 distinct generic book review texts for the sentiment labels \"positive\", \"neutral\", and \"negative\" called synthetic_reviews_by_sentiment*"
1396
+ ]
1397
+ },
1398
+ {
1399
+ "cell_type": "code",
1400
+ "execution_count": 19,
1401
+ "metadata": {
1402
+ "id": "b3cd2a50"
1403
+ },
1404
+ "outputs": [],
1405
+ "source": [
1406
+ "synthetic_reviews_by_sentiment = {\n",
1407
+ " \"positive\": [\n",
1408
+ " \"A compelling and heartwarming read that stayed with me long after I finished.\",\n",
1409
+ " \"Brilliantly written! The characters were unforgettable and the plot was engaging.\",\n",
1410
+ " \"One of the best books I've read this year — inspiring and emotionally rich.\",\n",
1411
+ " \"An absolute masterpiece with beautiful storytelling.\",\n",
1412
+ " \"I couldn’t put it down — gripping from start to finish.\",\n",
1413
+ " \"A powerful narrative that exceeded my expectations.\",\n",
1414
+ " \"The author’s writing style is captivating and immersive.\",\n",
1415
+ " \"An inspiring story filled with depth and meaning.\",\n",
1416
+ " \"A beautifully crafted tale that felt authentic and moving.\",\n",
1417
+ " \"The character development was exceptional.\",\n",
1418
+ " \"A truly uplifting and memorable experience.\",\n",
1419
+ " \"Rich in detail and emotionally satisfying.\",\n",
1420
+ " \"An engaging plot with well-timed twists.\",\n",
1421
+ " \"Thoroughly enjoyable and thoughtfully written.\",\n",
1422
+ " \"A refreshing and original storyline.\",\n",
1423
+ " \"Deeply touching and wonderfully executed.\",\n",
1424
+ " \"An instant favorite that I would gladly reread.\",\n",
1425
+ " \"The pacing was perfect and the ending satisfying.\",\n",
1426
+ " \"Creative, engaging, and beautifully structured.\",\n",
1427
+ " \"An outstanding book that left a lasting impression.\",\n",
1428
+ " \"Full of heart and brilliantly delivered.\",\n",
1429
+ " \"A delightful and immersive literary journey.\",\n",
1430
+ " \"Smart, compelling, and emotionally resonant.\",\n",
1431
+ " \"A must-read for fans of the genre.\",\n",
1432
+ " \"Incredibly well-written with vivid imagery.\",\n",
1433
+ " \"The dialogue felt natural and impactful.\",\n",
1434
+ " \"An engaging and thoughtfully constructed plot.\",\n",
1435
+ " \"A rewarding and meaningful reading experience.\",\n",
1436
+ " \"The storytelling was vivid and captivating.\",\n",
1437
+ " \"A gripping narrative that kept me invested.\",\n",
1438
+ " \"An emotionally rich and satisfying read.\",\n",
1439
+ " \"A powerful and beautifully told story.\",\n",
1440
+ " \"Thoroughly captivating from beginning to end.\",\n",
1441
+ " \"A wonderfully imaginative and engaging book.\",\n",
1442
+ " \"An inspiring story with strong characters.\",\n",
1443
+ " \"A compelling blend of drama and insight.\",\n",
1444
+ " \"Exceptionally well paced and structured.\",\n",
1445
+ " \"A heartfelt and beautifully written novel.\",\n",
1446
+ " \"The themes were handled with depth and care.\",\n",
1447
+ " \"A standout book that exceeded expectations.\",\n",
1448
+ " \"Engaging, thoughtful, and expertly written.\",\n",
1449
+ " \"A remarkable and moving literary work.\",\n",
1450
+ " \"The emotional arc was deeply satisfying.\",\n",
1451
+ " \"An unforgettable reading experience.\",\n",
1452
+ " \"Cleverly written with strong narrative flow.\",\n",
1453
+ " \"A book that truly shines in every way.\",\n",
1454
+ " \"An enriching and thoroughly enjoyable read.\",\n",
1455
+ " \"Skillfully written and emotionally impactful.\",\n",
1456
+ " \"A dynamic story that held my attention.\",\n",
1457
+ " \"An exceptional and inspiring novel.\"\n",
1458
+ " ],\n",
1459
+ " \"neutral\": [\n",
1460
+ " \"An average book — not great, but not bad either.\",\n",
1461
+ " \"Some parts really stood out, others felt a bit flat.\",\n",
1462
+ " \"It was okay overall. A decent way to pass the time.\",\n",
1463
+ " \"A fairly standard story with predictable moments.\",\n",
1464
+ " \"Neither impressive nor disappointing.\",\n",
1465
+ " \"It had its strengths, but also noticeable flaws.\",\n",
1466
+ " \"A middle-of-the-road reading experience.\",\n",
1467
+ " \"The pacing was uneven at times.\",\n",
1468
+ " \"An acceptable but not remarkable read.\",\n",
1469
+ " \"Some chapters were engaging, others less so.\",\n",
1470
+ " \"The characters were fine but not memorable.\",\n",
1471
+ " \"A straightforward and simple storyline.\",\n",
1472
+ " \"It delivered what was expected, nothing more.\",\n",
1473
+ " \"A mildly entertaining book.\",\n",
1474
+ " \"The plot was serviceable but not exciting.\",\n",
1475
+ " \"An easy read with limited depth.\",\n",
1476
+ " \"Competently written but lacked spark.\",\n",
1477
+ " \"A decent effort overall.\",\n",
1478
+ " \"The story progressed steadily but predictably.\",\n",
1479
+ " \"Not bad, but it didn’t fully captivate me.\",\n",
1480
+ " \"Some interesting ideas, though not fully explored.\",\n",
1481
+ " \"A readable but somewhat forgettable book.\",\n",
1482
+ " \"It met expectations without exceeding them.\",\n",
1483
+ " \"The writing was solid but unremarkable.\",\n",
1484
+ " \"A standard entry in its genre.\",\n",
1485
+ " \"It had potential but felt restrained.\",\n",
1486
+ " \"The ending was satisfactory but not surprising.\",\n",
1487
+ " \"An average literary experience.\",\n",
1488
+ " \"The themes were present but not deeply developed.\",\n",
1489
+ " \"A book that sits comfortably in the middle.\",\n",
1490
+ " \"Moderately enjoyable with room for improvement.\",\n",
1491
+ " \"A simple and straightforward narrative.\",\n",
1492
+ " \"Nothing particularly groundbreaking here.\",\n",
1493
+ " \"An ordinary yet readable novel.\",\n",
1494
+ " \"Some compelling moments scattered throughout.\",\n",
1495
+ " \"The structure was conventional and safe.\",\n",
1496
+ " \"A balanced mix of strengths and weaknesses.\",\n",
1497
+ " \"Engaging at times, but inconsistent.\",\n",
1498
+ " \"A passable story with limited emotional impact.\",\n",
1499
+ " \"Adequate character development overall.\",\n",
1500
+ " \"It maintained interest without fully immersing me.\",\n",
1501
+ " \"A neutral reading experience overall.\",\n",
1502
+ " \"Predictable but not unpleasant.\",\n",
1503
+ " \"Readable and competently executed.\",\n",
1504
+ " \"An average contribution to the genre.\",\n",
1505
+ " \"A calm and steady narrative.\",\n",
1506
+ " \"Interesting premise, moderate execution.\",\n",
1507
+ " \"A fair but unremarkable book.\",\n",
1508
+ " \"It was fine, though not memorable.\",\n",
1509
+ " \"Neither particularly good nor particularly bad.\"\n",
1510
+ " ],\n",
1511
+ " \"negative\": [\n",
1512
+ " \"I struggled to get through this one — it just didn’t grab me.\",\n",
1513
+ " \"The plot was confusing and the characters felt underdeveloped.\",\n",
1514
+ " \"Disappointing. I had high hopes, but they weren't met.\",\n",
1515
+ " \"The pacing was painfully slow.\",\n",
1516
+ " \"The story felt disjointed and unfocused.\",\n",
1517
+ " \"I found it difficult to stay engaged.\",\n",
1518
+ " \"The characters lacked depth and realism.\",\n",
1519
+ " \"A frustrating and unsatisfying read.\",\n",
1520
+ " \"The dialogue felt forced and unnatural.\",\n",
1521
+ " \"The plot twists were predictable and weak.\",\n",
1522
+ " \"It failed to hold my attention.\",\n",
1523
+ " \"The writing style didn’t resonate with me.\",\n",
1524
+ " \"An underwhelming and forgettable book.\",\n",
1525
+ " \"The storyline lacked coherence.\",\n",
1526
+ " \"The ending felt rushed and incomplete.\",\n",
1527
+ " \"I expected more substance and clarity.\",\n",
1528
+ " \"The themes were poorly executed.\",\n",
1529
+ " \"The narrative felt repetitive.\",\n",
1530
+ " \"I found the characters unrelatable.\",\n",
1531
+ " \"A disappointing reading experience.\",\n",
1532
+ " \"The book lacked emotional impact.\",\n",
1533
+ " \"The structure was messy and confusing.\",\n",
1534
+ " \"It struggled to maintain momentum.\",\n",
1535
+ " \"The writing felt flat and uninspired.\",\n",
1536
+ " \"The premise had promise but fell short.\",\n",
1537
+ " \"An uneven and poorly paced novel.\",\n",
1538
+ " \"The conflicts felt artificial.\",\n",
1539
+ " \"I was bored for most of the book.\",\n",
1540
+ " \"The storytelling lacked clarity.\",\n",
1541
+ " \"The character arcs felt incomplete.\",\n",
1542
+ " \"An unsatisfying and forgettable read.\",\n",
1543
+ " \"The tone felt inconsistent throughout.\",\n",
1544
+ " \"The plot progression was weak.\",\n",
1545
+ " \"The book felt longer than it needed to be.\",\n",
1546
+ " \"A bland and uninspired story.\",\n",
1547
+ " \"The execution did not match the ambition.\",\n",
1548
+ " \"I had trouble connecting with the narrative.\",\n",
1549
+ " \"The suspense never truly developed.\",\n",
1550
+ " \"The dialogue lacked authenticity.\",\n",
1551
+ " \"The story felt predictable and stale.\",\n",
1552
+ " \"A confusing and cluttered plot.\",\n",
1553
+ " \"The characters felt one-dimensional.\",\n",
1554
+ " \"The emotional beats didn’t land.\",\n",
1555
+ " \"An overly complicated storyline.\",\n",
1556
+ " \"The pacing dragged significantly.\",\n",
1557
+ " \"The resolution was unsatisfying.\",\n",
1558
+ " \"The writing style felt awkward.\",\n",
1559
+ " \"A lackluster and disappointing novel.\",\n",
1560
+ " \"The narrative failed to engage me.\",\n",
1561
+ " \"Overall, it simply didn’t work for me.\"\n",
1562
+ " ]\n",
1563
+ "}"
1564
+ ]
1565
+ },
1566
+ {
1567
+ "cell_type": "markdown",
1568
+ "metadata": {
1569
+ "id": "fQhfVaDmuULT"
1570
+ },
1571
+ "source": [
1572
+ "### *b. Generate 10 reviews per book using random sampling from the corresponding 50*"
1573
+ ]
1574
+ },
1575
+ {
1576
+ "cell_type": "code",
1577
+ "execution_count": 20,
1578
+ "metadata": {
1579
+ "id": "l2SRc3PjuTGM"
1580
+ },
1581
+ "outputs": [],
1582
+ "source": [
1583
+ "review_rows = []\n",
1584
+ "for _, row in df_books.iterrows():\n",
1585
+ " title = row['title']\n",
1586
+ " sentiment_label = row['sentiment_label']\n",
1587
+ " review_pool = synthetic_reviews_by_sentiment[sentiment_label]\n",
1588
+ " sampled_reviews = random.sample(review_pool, 10)\n",
1589
+ " for review_text in sampled_reviews:\n",
1590
+ " review_rows.append({\n",
1591
+ " \"title\": title,\n",
1592
+ " \"sentiment_label\": sentiment_label,\n",
1593
+ " \"review_text\": review_text,\n",
1594
+ " \"rating\": row['rating'],\n",
1595
+ " \"popularity_score\": row['popularity_score']\n",
1596
+ " })"
1597
+ ]
1598
+ },
1599
+ {
1600
+ "cell_type": "markdown",
1601
+ "metadata": {
1602
+ "id": "bmJMXF-Bukdm"
1603
+ },
1604
+ "source": [
1605
+ "### *c. Create the final dataframe df_reviews & save it as synthetic_book_reviews.csv*"
1606
+ ]
1607
+ },
1608
+ {
1609
+ "cell_type": "code",
1610
+ "execution_count": 21,
1611
+ "metadata": {
1612
+ "id": "ZUKUqZsuumsp"
1613
+ },
1614
+ "outputs": [],
1615
+ "source": [
1616
+ "df_reviews = pd.DataFrame(review_rows)\n",
1617
+ "df_reviews.to_csv(\"synthetic_book_reviews.csv\", index=False)"
1618
+ ]
1619
+ },
1620
+ {
1621
+ "cell_type": "markdown",
1622
+ "source": [
1623
+ "### *c. inputs for R*"
1624
+ ],
1625
+ "metadata": {
1626
+ "id": "_602pYUS3gY5"
1627
+ }
1628
+ },
1629
+ {
1630
+ "cell_type": "code",
1631
+ "execution_count": 22,
1632
+ "metadata": {
1633
+ "colab": {
1634
+ "base_uri": "https://localhost:8080/"
1635
+ },
1636
+ "id": "3946e521",
1637
+ "outputId": "6bb59b6b-1468-43c2-afad-7987d46ee3c8"
1638
+ },
1639
+ "outputs": [
1640
+ {
1641
+ "output_type": "stream",
1642
+ "name": "stdout",
1643
+ "text": [
1644
+ "✅ Wrote synthetic_title_level_features.csv\n",
1645
+ "✅ Wrote synthetic_monthly_revenue_series.csv\n"
1646
+ ]
1647
+ }
1648
+ ],
1649
+ "source": [
1650
+ "import numpy as np\n",
1651
+ "\n",
1652
+ "def _safe_num(s):\n",
1653
+ " return pd.to_numeric(\n",
1654
+ " pd.Series(s).astype(str).str.replace(r\"[^0-9.]\", \"\", regex=True),\n",
1655
+ " errors=\"coerce\"\n",
1656
+ " )\n",
1657
+ "\n",
1658
+ "# --- Clean book metadata (price/rating) ---\n",
1659
+ "df_books_r = df_books.copy()\n",
1660
+ "if \"price\" in df_books_r.columns:\n",
1661
+ " df_books_r[\"price\"] = _safe_num(df_books_r[\"price\"])\n",
1662
+ "if \"rating\" in df_books_r.columns:\n",
1663
+ " df_books_r[\"rating\"] = _safe_num(df_books_r[\"rating\"])\n",
1664
+ "\n",
1665
+ "df_books_r[\"title\"] = df_books_r[\"title\"].astype(str).str.strip()\n",
1666
+ "\n",
1667
+ "# --- Clean sales ---\n",
1668
+ "df_sales_r = df_sales.copy()\n",
1669
+ "df_sales_r[\"title\"] = df_sales_r[\"title\"].astype(str).str.strip()\n",
1670
+ "df_sales_r[\"month\"] = pd.to_datetime(df_sales_r[\"month\"], errors=\"coerce\")\n",
1671
+ "df_sales_r[\"units_sold\"] = _safe_num(df_sales_r[\"units_sold\"])\n",
1672
+ "\n",
1673
+ "# --- Clean reviews ---\n",
1674
+ "df_reviews_r = df_reviews.copy()\n",
1675
+ "df_reviews_r[\"title\"] = df_reviews_r[\"title\"].astype(str).str.strip()\n",
1676
+ "df_reviews_r[\"sentiment_label\"] = df_reviews_r[\"sentiment_label\"].astype(str).str.lower().str.strip()\n",
1677
+ "if \"rating\" in df_reviews_r.columns:\n",
1678
+ " df_reviews_r[\"rating\"] = _safe_num(df_reviews_r[\"rating\"])\n",
1679
+ "if \"popularity_score\" in df_reviews_r.columns:\n",
1680
+ " df_reviews_r[\"popularity_score\"] = _safe_num(df_reviews_r[\"popularity_score\"])\n",
1681
+ "\n",
1682
+ "# --- Sentiment shares per title (from reviews) ---\n",
1683
+ "sent_counts = (\n",
1684
+ " df_reviews_r.groupby([\"title\", \"sentiment_label\"])\n",
1685
+ " .size()\n",
1686
+ " .unstack(fill_value=0)\n",
1687
+ ")\n",
1688
+ "for lab in [\"positive\", \"neutral\", \"negative\"]:\n",
1689
+ " if lab not in sent_counts.columns:\n",
1690
+ " sent_counts[lab] = 0\n",
1691
+ "\n",
1692
+ "sent_counts[\"total_reviews\"] = sent_counts[[\"positive\", \"neutral\", \"negative\"]].sum(axis=1)\n",
1693
+ "den = sent_counts[\"total_reviews\"].replace(0, np.nan)\n",
1694
+ "sent_counts[\"share_positive\"] = sent_counts[\"positive\"] / den\n",
1695
+ "sent_counts[\"share_neutral\"] = sent_counts[\"neutral\"] / den\n",
1696
+ "sent_counts[\"share_negative\"] = sent_counts[\"negative\"] / den\n",
1697
+ "sent_counts = sent_counts.reset_index()\n",
1698
+ "\n",
1699
+ "# --- Sales aggregation per title ---\n",
1700
+ "sales_by_title = (\n",
1701
+ " df_sales_r.dropna(subset=[\"title\"])\n",
1702
+ " .groupby(\"title\", as_index=False)\n",
1703
+ " .agg(\n",
1704
+ " months_observed=(\"month\", \"nunique\"),\n",
1705
+ " avg_units_sold=(\"units_sold\", \"mean\"),\n",
1706
+ " total_units_sold=(\"units_sold\", \"sum\"),\n",
1707
+ " )\n",
1708
+ ")\n",
1709
+ "\n",
1710
+ "# --- Title-level features (join sales + books + sentiment) ---\n",
1711
+ "df_title = (\n",
1712
+ " sales_by_title\n",
1713
+ " .merge(df_books_r[[\"title\", \"price\", \"rating\"]], on=\"title\", how=\"left\")\n",
1714
+ " .merge(sent_counts[[\"title\", \"share_positive\", \"share_neutral\", \"share_negative\", \"total_reviews\"]],\n",
1715
+ " on=\"title\", how=\"left\")\n",
1716
+ ")\n",
1717
+ "\n",
1718
+ "df_title[\"avg_revenue\"] = df_title[\"avg_units_sold\"] * df_title[\"price\"]\n",
1719
+ "df_title[\"total_revenue\"] = df_title[\"total_units_sold\"] * df_title[\"price\"]\n",
1720
+ "\n",
1721
+ "df_title.to_csv(\"synthetic_title_level_features.csv\", index=False)\n",
1722
+ "print(\"✅ Wrote synthetic_title_level_features.csv\")\n",
1723
+ "\n",
1724
+ "# --- Monthly revenue series (proxy: units_sold * price) ---\n",
1725
+ "monthly_rev = (\n",
1726
+ " df_sales_r.merge(df_books_r[[\"title\", \"price\"]], on=\"title\", how=\"left\")\n",
1727
+ ")\n",
1728
+ "monthly_rev[\"revenue\"] = monthly_rev[\"units_sold\"] * monthly_rev[\"price\"]\n",
1729
+ "\n",
1730
+ "df_monthly = (\n",
1731
+ " monthly_rev.dropna(subset=[\"month\"])\n",
1732
+ " .groupby(\"month\", as_index=False)[\"revenue\"]\n",
1733
+ " .sum()\n",
1734
+ " .rename(columns={\"revenue\": \"total_revenue\"})\n",
1735
+ " .sort_values(\"month\")\n",
1736
+ ")\n",
1737
+ "# if revenue is all NA (e.g., missing price), fallback to units_sold as a teaching proxy\n",
1738
+ "if df_monthly[\"total_revenue\"].notna().sum() == 0:\n",
1739
+ " df_monthly = (\n",
1740
+ " df_sales_r.dropna(subset=[\"month\"])\n",
1741
+ " .groupby(\"month\", as_index=False)[\"units_sold\"]\n",
1742
+ " .sum()\n",
1743
+ " .rename(columns={\"units_sold\": \"total_revenue\"})\n",
1744
+ " .sort_values(\"month\")\n",
1745
+ " )\n",
1746
+ "\n",
1747
+ "df_monthly[\"month\"] = pd.to_datetime(df_monthly[\"month\"], errors=\"coerce\").dt.strftime(\"%Y-%m-%d\")\n",
1748
+ "df_monthly.to_csv(\"synthetic_monthly_revenue_series.csv\", index=False)\n",
1749
+ "print(\"✅ Wrote synthetic_monthly_revenue_series.csv\")\n"
1750
+ ]
1751
+ },
1752
+ {
1753
+ "cell_type": "markdown",
1754
+ "metadata": {
1755
+ "id": "RYvGyVfXuo54"
1756
+ },
1757
+ "source": [
1758
+ "### *d. ✋🏻🛑⛔️ View the first few lines*"
1759
+ ]
1760
+ },
1761
+ {
1762
+ "cell_type": "code",
1763
+ "execution_count": 25,
1764
+ "metadata": {
1765
+ "colab": {
1766
+ "base_uri": "https://localhost:8080/",
1767
+ "height": 201
1768
+ },
1769
+ "id": "xfE8NMqOurKo",
1770
+ "outputId": "370035c0-d9c0-4efc-e5c9-0becc2d0f697"
1771
+ },
1772
+ "outputs": [
1773
+ {
1774
+ "output_type": "execute_result",
1775
+ "data": {
1776
+ "text/plain": [
1777
+ " title sentiment_label \\\n",
1778
+ "0 A Light in the Attic neutral \n",
1779
+ "1 A Light in the Attic neutral \n",
1780
+ "2 A Light in the Attic neutral \n",
1781
+ "3 A Light in the Attic neutral \n",
1782
+ "4 A Light in the Attic neutral \n",
1783
+ "\n",
1784
+ " review_text rating popularity_score \n",
1785
+ "0 Interesting premise, moderate execution. Three 3 \n",
1786
+ "1 It met expectations without exceeding them. Three 3 \n",
1787
+ "2 Some compelling moments scattered throughout. Three 3 \n",
1788
+ "3 An acceptable but not remarkable read. Three 3 \n",
1789
+ "4 An average contribution to the genre. Three 3 "
1790
+ ],
1791
+ "text/html": [
1792
+ "\n",
1793
+ " <div id=\"df-927bd57a-a757-4663-87e3-599a0a7c0238\" class=\"colab-df-container\">\n",
1794
+ " <div>\n",
1795
+ "<style scoped>\n",
1796
+ " .dataframe tbody tr th:only-of-type {\n",
1797
+ " vertical-align: middle;\n",
1798
+ " }\n",
1799
+ "\n",
1800
+ " .dataframe tbody tr th {\n",
1801
+ " vertical-align: top;\n",
1802
+ " }\n",
1803
+ "\n",
1804
+ " .dataframe thead th {\n",
1805
+ " text-align: right;\n",
1806
+ " }\n",
1807
+ "</style>\n",
1808
+ "<table border=\"1\" class=\"dataframe\">\n",
1809
+ " <thead>\n",
1810
+ " <tr style=\"text-align: right;\">\n",
1811
+ " <th></th>\n",
1812
+ " <th>title</th>\n",
1813
+ " <th>sentiment_label</th>\n",
1814
+ " <th>review_text</th>\n",
1815
+ " <th>rating</th>\n",
1816
+ " <th>popularity_score</th>\n",
1817
+ " </tr>\n",
1818
+ " </thead>\n",
1819
+ " <tbody>\n",
1820
+ " <tr>\n",
1821
+ " <th>0</th>\n",
1822
+ " <td>A Light in the Attic</td>\n",
1823
+ " <td>neutral</td>\n",
1824
+ " <td>Interesting premise, moderate execution.</td>\n",
1825
+ " <td>Three</td>\n",
1826
+ " <td>3</td>\n",
1827
+ " </tr>\n",
1828
+ " <tr>\n",
1829
+ " <th>1</th>\n",
1830
+ " <td>A Light in the Attic</td>\n",
1831
+ " <td>neutral</td>\n",
1832
+ " <td>It met expectations without exceeding them.</td>\n",
1833
+ " <td>Three</td>\n",
1834
+ " <td>3</td>\n",
1835
+ " </tr>\n",
1836
+ " <tr>\n",
1837
+ " <th>2</th>\n",
1838
+ " <td>A Light in the Attic</td>\n",
1839
+ " <td>neutral</td>\n",
1840
+ " <td>Some compelling moments scattered throughout.</td>\n",
1841
+ " <td>Three</td>\n",
1842
+ " <td>3</td>\n",
1843
+ " </tr>\n",
1844
+ " <tr>\n",
1845
+ " <th>3</th>\n",
1846
+ " <td>A Light in the Attic</td>\n",
1847
+ " <td>neutral</td>\n",
1848
+ " <td>An acceptable but not remarkable read.</td>\n",
1849
+ " <td>Three</td>\n",
1850
+ " <td>3</td>\n",
1851
+ " </tr>\n",
1852
+ " <tr>\n",
1853
+ " <th>4</th>\n",
1854
+ " <td>A Light in the Attic</td>\n",
1855
+ " <td>neutral</td>\n",
1856
+ " <td>An average contribution to the genre.</td>\n",
1857
+ " <td>Three</td>\n",
1858
+ " <td>3</td>\n",
1859
+ " </tr>\n",
1860
+ " </tbody>\n",
1861
+ "</table>\n",
1862
+ "</div>\n",
1863
+ " <div class=\"colab-df-buttons\">\n",
1864
+ "\n",
1865
+ " <div class=\"colab-df-container\">\n",
1866
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-927bd57a-a757-4663-87e3-599a0a7c0238')\"\n",
1867
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1868
+ " style=\"display:none;\">\n",
1869
+ "\n",
1870
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1871
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1872
+ " </svg>\n",
1873
+ " </button>\n",
1874
+ "\n",
1875
+ " <style>\n",
1876
+ " .colab-df-container {\n",
1877
+ " display:flex;\n",
1878
+ " gap: 12px;\n",
1879
+ " }\n",
1880
+ "\n",
1881
+ " .colab-df-convert {\n",
1882
+ " background-color: #E8F0FE;\n",
1883
+ " border: none;\n",
1884
+ " border-radius: 50%;\n",
1885
+ " cursor: pointer;\n",
1886
+ " display: none;\n",
1887
+ " fill: #1967D2;\n",
1888
+ " height: 32px;\n",
1889
+ " padding: 0 0 0 0;\n",
1890
+ " width: 32px;\n",
1891
+ " }\n",
1892
+ "\n",
1893
+ " .colab-df-convert:hover {\n",
1894
+ " background-color: #E2EBFA;\n",
1895
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1896
+ " fill: #174EA6;\n",
1897
+ " }\n",
1898
+ "\n",
1899
+ " .colab-df-buttons div {\n",
1900
+ " margin-bottom: 4px;\n",
1901
+ " }\n",
1902
+ "\n",
1903
+ " [theme=dark] .colab-df-convert {\n",
1904
+ " background-color: #3B4455;\n",
1905
+ " fill: #D2E3FC;\n",
1906
+ " }\n",
1907
+ "\n",
1908
+ " [theme=dark] .colab-df-convert:hover {\n",
1909
+ " background-color: #434B5C;\n",
1910
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1911
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1912
+ " fill: #FFFFFF;\n",
1913
+ " }\n",
1914
+ " </style>\n",
1915
+ "\n",
1916
+ " <script>\n",
1917
+ " const buttonEl =\n",
1918
+ " document.querySelector('#df-927bd57a-a757-4663-87e3-599a0a7c0238 button.colab-df-convert');\n",
1919
+ " buttonEl.style.display =\n",
1920
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1921
+ "\n",
1922
+ " async function convertToInteractive(key) {\n",
1923
+ " const element = document.querySelector('#df-927bd57a-a757-4663-87e3-599a0a7c0238');\n",
1924
+ " const dataTable =\n",
1925
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1926
+ " [key], {});\n",
1927
+ " if (!dataTable) return;\n",
1928
+ "\n",
1929
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1930
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1931
+ " + ' to learn more about interactive tables.';\n",
1932
+ " element.innerHTML = '';\n",
1933
+ " dataTable['output_type'] = 'display_data';\n",
1934
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1935
+ " const docLink = document.createElement('div');\n",
1936
+ " docLink.innerHTML = docLinkHtml;\n",
1937
+ " element.appendChild(docLink);\n",
1938
+ " }\n",
1939
+ " </script>\n",
1940
+ " </div>\n",
1941
+ "\n",
1942
+ "\n",
1943
+ " </div>\n",
1944
+ " </div>\n"
1945
+ ],
1946
+ "application/vnd.google.colaboratory.intrinsic+json": {
1947
+ "type": "dataframe",
1948
+ "variable_name": "df_reviews",
1949
+ "summary": "{\n \"name\": \"df_reviews\",\n \"rows\": 10000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"review_text\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 150,\n \"samples\": [\n \"Full of heart and brilliantly delivered.\",\n \"The ending felt rushed and incomplete.\",\n \"An emotionally rich and satisfying read.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1950
+ }
1951
+ },
1952
+ "metadata": {},
1953
+ "execution_count": 25
1954
+ }
1955
+ ],
1956
+ "source": [
1957
+ "df_reviews.head()"
1958
+ ]
1959
+ }
1960
+ ],
1961
+ "metadata": {
1962
+ "colab": {
1963
+ "provenance": []
1964
+ },
1965
+ "kernelspec": {
1966
+ "display_name": "Python 3",
1967
+ "name": "python3"
1968
+ },
1969
+ "language_info": {
1970
+ "name": "python"
1971
+ }
1972
+ },
1973
+ "nbformat": 4,
1974
+ "nbformat_minor": 0
1975
+ }
2a_Python_Analysis_Matéo_François.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
R_analysis_Matéo_FRANCOIS.ipynb ADDED
@@ -0,0 +1,518 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "id": "75fd9cc6",
6
+ "metadata": {
7
+ "id": "75fd9cc6"
8
+ },
9
+ "source": [
10
+ "# **🤖 Benchmarking & Modeling**"
11
+ ]
12
+ },
13
+ {
14
+ "cell_type": "markdown",
15
+ "id": "fb807724",
16
+ "metadata": {
17
+ "id": "fb807724"
18
+ },
19
+ "source": [
20
+ "## **1.** 📦 Setup"
21
+ ]
22
+ },
23
+ {
24
+ "cell_type": "code",
25
+ "execution_count": 6,
26
+ "id": "d40cd131",
27
+ "metadata": {
28
+ "id": "d40cd131",
29
+ "outputId": "f520f27c-3e9c-470d-c36b-0267ff1c4949",
30
+ "colab": {
31
+ "base_uri": "https://localhost:8080/"
32
+ }
33
+ },
34
+ "outputs": [
35
+ {
36
+ "output_type": "stream",
37
+ "name": "stderr",
38
+ "text": [
39
+ "Installing packages into ‘/usr/local/lib/R/site-library’\n",
40
+ "(as ‘lib’ is unspecified)\n",
41
+ "\n",
42
+ "Warning message in install.packages(c(\"readr\", \"dplyr\", \"stringr\", \"tidyr\", \"lubridate\", :\n",
43
+ "“installation of package ‘forecast’ had non-zero exit status”\n"
44
+ ]
45
+ }
46
+ ],
47
+ "source": [
48
+ "install.packages(c(\n",
49
+ " \"readr\",\"dplyr\",\"stringr\",\"tidyr\",\"lubridate\",\n",
50
+ " \"ggplot2\",\"forecast\",\"broom\",\"jsonlite\"\n",
51
+ "), repos=\"https://cloud.r-project.org\")"
52
+ ]
53
+ },
54
+ {
55
+ "cell_type": "code",
56
+ "source": [
57
+ "#do above first and then Runtime → Restart session and then do:\n",
58
+ "library(readr)\n",
59
+ "library(dplyr)\n",
60
+ "library(stringr)\n",
61
+ "library(tidyr)\n",
62
+ "library(lubridate)\n",
63
+ "library(ggplot2)\n",
64
+ "library(forecast)\n",
65
+ "library(broom)\n",
66
+ "library(jsonlite)"
67
+ ],
68
+ "metadata": {
69
+ "colab": {
70
+ "base_uri": "https://localhost:8080/"
71
+ },
72
+ "id": "FR_YZUD9J_Py",
73
+ "outputId": "6b801e29-0053-469f-b4c9-05d8ead19904"
74
+ },
75
+ "id": "FR_YZUD9J_Py",
76
+ "execution_count": 1,
77
+ "outputs": [
78
+ {
79
+ "output_type": "stream",
80
+ "name": "stderr",
81
+ "text": [
82
+ "\n",
83
+ "Attaching package: ‘dplyr’\n",
84
+ "\n",
85
+ "\n",
86
+ "The following objects are masked from ‘package:stats’:\n",
87
+ "\n",
88
+ " filter, lag\n",
89
+ "\n",
90
+ "\n",
91
+ "The following objects are masked from ‘package:base’:\n",
92
+ "\n",
93
+ " intersect, setdiff, setequal, union\n",
94
+ "\n",
95
+ "\n",
96
+ "\n",
97
+ "Attaching package: ‘lubridate’\n",
98
+ "\n",
99
+ "\n",
100
+ "The following objects are masked from ‘package:base’:\n",
101
+ "\n",
102
+ " date, intersect, setdiff, union\n",
103
+ "\n",
104
+ "\n"
105
+ ]
106
+ }
107
+ ]
108
+ },
109
+ {
110
+ "cell_type": "markdown",
111
+ "id": "f01d02e7",
112
+ "metadata": {
113
+ "id": "f01d02e7"
114
+ },
115
+ "source": [
116
+ "## **2.** ✅️ Load & inspect inputs"
117
+ ]
118
+ },
119
+ {
120
+ "cell_type": "code",
121
+ "execution_count": 3,
122
+ "id": "29e8f6ce",
123
+ "metadata": {
124
+ "colab": {
125
+ "base_uri": "https://localhost:8080/"
126
+ },
127
+ "id": "29e8f6ce",
128
+ "outputId": "9af6e4c9-59d9-4475-a106-beada0b3338c"
129
+ },
130
+ "outputs": [
131
+ {
132
+ "output_type": "stream",
133
+ "name": "stdout",
134
+ "text": [
135
+ "Loaded: 1000 rows (title-level), 18 rows (monthly)\n"
136
+ ]
137
+ }
138
+ ],
139
+ "source": [
140
+ "\n",
141
+ "must_exist <- function(path, label) {\n",
142
+ " if (!file.exists(path)) stop(paste0(\"Missing \", label, \": \", path))\n",
143
+ "}\n",
144
+ "\n",
145
+ "TITLE_PATH <- \"synthetic_title_level_features.csv\"\n",
146
+ "MONTH_PATH <- \"synthetic_monthly_revenue_series.csv\"\n",
147
+ "\n",
148
+ "must_exist(TITLE_PATH, \"TITLE_PATH\")\n",
149
+ "must_exist(MONTH_PATH, \"MONTH_PATH\")\n",
150
+ "\n",
151
+ "df_title <- read_csv(TITLE_PATH, show_col_types = FALSE)\n",
152
+ "df_month <- read_csv(MONTH_PATH, show_col_types = FALSE)\n",
153
+ "\n",
154
+ "cat(\"Loaded:\", nrow(df_title), \"rows (title-level),\", nrow(df_month), \"rows (monthly)\n",
155
+ "\")"
156
+ ]
157
+ },
158
+ {
159
+ "cell_type": "code",
160
+ "execution_count": 4,
161
+ "id": "9fd04262",
162
+ "metadata": {
163
+ "colab": {
164
+ "base_uri": "https://localhost:8080/"
165
+ },
166
+ "id": "9fd04262",
167
+ "outputId": "b7c028ba-157c-4d4f-db6c-9098104adec5"
168
+ },
169
+ "outputs": [
170
+ {
171
+ "output_type": "stream",
172
+ "name": "stdout",
173
+ "text": [
174
+ "\u001b[90m# A tibble: 1 × 6\u001b[39m\n",
175
+ " n na_avg_revenue na_price na_rating na_share_pos na_share_neg\n",
176
+ " \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m\n",
177
+ "\u001b[90m1\u001b[39m \u001b[4m1\u001b[24m000 0 0 \u001b[4m1\u001b[24m000 0 0\n",
178
+ "Monthly rows after parsing: 18 \n"
179
+ ]
180
+ }
181
+ ],
182
+ "source": [
183
+ "\n",
184
+ "# ---------- helpers ----------\n",
185
+ "safe_num <- function(x) {\n",
186
+ " # strips anything that is not digit or dot\n",
187
+ " suppressWarnings(as.numeric(str_replace_all(as.character(x), \"[^0-9.]\", \"\")))\n",
188
+ "}\n",
189
+ "\n",
190
+ "parse_rating <- function(x) {\n",
191
+ " # Accept: 4, \"4\", \"4.0\", \"4/5\", \"4 out of 5\", \"⭐⭐⭐⭐\", etc.\n",
192
+ " x <- as.character(x)\n",
193
+ " x <- str_replace_all(x, \"⭐\", \"\")\n",
194
+ " x <- str_to_lower(x)\n",
195
+ " x <- str_replace_all(x, \"stars?\", \"\")\n",
196
+ " x <- str_replace_all(x, \"out of\", \"/\")\n",
197
+ " x <- str_replace_all(x, \"\\\\s+\", \"\")\n",
198
+ " x <- str_replace_all(x, \"[^0-9./]\", \"\")\n",
199
+ " suppressWarnings(as.numeric(str_extract(x, \"^[0-9.]+\")))\n",
200
+ "}\n",
201
+ "\n",
202
+ "parse_month <- function(x) {\n",
203
+ " x <- as.character(x)\n",
204
+ " # try YYYY-MM-DD, then YYYY-MM\n",
205
+ " out <- suppressWarnings(ymd(x))\n",
206
+ " if (mean(is.na(out)) > 0.5) out <- suppressWarnings(ymd(paste0(x, \"-01\")))\n",
207
+ " na_idx <- which(is.na(out))\n",
208
+ " if (length(na_idx) > 0) out[na_idx] <- suppressWarnings(ymd(paste0(x[na_idx], \"-01\")))\n",
209
+ " out\n",
210
+ "}\n",
211
+ "\n",
212
+ "# ---------- normalize keys ----------\n",
213
+ "df_title <- df_title %>% mutate(title = str_squish(as.character(title)))\n",
214
+ "df_month <- df_month %>% mutate(month = as.character(month))\n",
215
+ "\n",
216
+ "# ---------- parse numeric columns defensively ----------\n",
217
+ "need_cols_title <- c(\"title\",\"avg_revenue\",\"total_revenue\",\"price\",\"rating\",\"share_positive\",\"share_negative\",\"share_neutral\")\n",
218
+ "missing_title <- setdiff(need_cols_title, names(df_title))\n",
219
+ "if (length(missing_title) > 0) stop(paste0(\"df_title missing columns: \", paste(missing_title, collapse=\", \")))\n",
220
+ "\n",
221
+ "df_title <- df_title %>%\n",
222
+ " mutate(\n",
223
+ " avg_revenue = safe_num(avg_revenue),\n",
224
+ " total_revenue = safe_num(total_revenue),\n",
225
+ " price = safe_num(price),\n",
226
+ " rating = parse_rating(rating),\n",
227
+ " share_positive = safe_num(share_positive),\n",
228
+ " share_negative = safe_num(share_negative),\n",
229
+ " share_neutral = safe_num(share_neutral)\n",
230
+ " )\n",
231
+ "\n",
232
+ "# basic sanity stats\n",
233
+ "hyg <- df_title %>%\n",
234
+ " summarise(\n",
235
+ " n = n(),\n",
236
+ " na_avg_revenue = sum(is.na(avg_revenue)),\n",
237
+ " na_price = sum(is.na(price)),\n",
238
+ " na_rating = sum(is.na(rating)),\n",
239
+ " na_share_pos = sum(is.na(share_positive)),\n",
240
+ " na_share_neg = sum(is.na(share_negative))\n",
241
+ " )\n",
242
+ "\n",
243
+ "print(hyg)\n",
244
+ "\n",
245
+ "# monthly parsing\n",
246
+ "need_cols_month <- c(\"month\",\"total_revenue\")\n",
247
+ "missing_month <- setdiff(need_cols_month, names(df_month))\n",
248
+ "if (length(missing_month) > 0) stop(paste0(\"df_month missing columns: \", paste(missing_month, collapse=\", \")))\n",
249
+ "\n",
250
+ "df_month2 <- df_month %>%\n",
251
+ " mutate(\n",
252
+ " month = parse_month(month),\n",
253
+ " total_revenue = safe_num(total_revenue)\n",
254
+ " ) %>%\n",
255
+ " filter(!is.na(month)) %>%\n",
256
+ " arrange(month)\n",
257
+ "\n",
258
+ "cat(\"Monthly rows after parsing:\", nrow(df_month2), \"\\n\")"
259
+ ]
260
+ },
261
+ {
262
+ "cell_type": "markdown",
263
+ "id": "b8971bc4",
264
+ "metadata": {
265
+ "id": "b8971bc4"
266
+ },
267
+ "source": [
268
+ "## **3.** 💾 Folder for R outputs for Hugging Face"
269
+ ]
270
+ },
271
+ {
272
+ "cell_type": "code",
273
+ "execution_count": 5,
274
+ "id": "dfaa06b1",
275
+ "metadata": {
276
+ "colab": {
277
+ "base_uri": "https://localhost:8080/"
278
+ },
279
+ "id": "dfaa06b1",
280
+ "outputId": "85005150-1d3f-4c16-ffbe-ca7b91b379ab"
281
+ },
282
+ "outputs": [
283
+ {
284
+ "output_type": "stream",
285
+ "name": "stdout",
286
+ "text": [
287
+ "R outputs will be written to: /content/artifacts/r \n"
288
+ ]
289
+ }
290
+ ],
291
+ "source": [
292
+ "\n",
293
+ "ART_DIR <- \"artifacts\"\n",
294
+ "R_FIG_DIR <- file.path(ART_DIR, \"r\", \"figures\")\n",
295
+ "R_TAB_DIR <- file.path(ART_DIR, \"r\", \"tables\")\n",
296
+ "\n",
297
+ "dir.create(R_FIG_DIR, recursive = TRUE, showWarnings = FALSE)\n",
298
+ "dir.create(R_TAB_DIR, recursive = TRUE, showWarnings = FALSE)\n",
299
+ "\n",
300
+ "cat(\"R outputs will be written to:\", normalizePath(file.path(ART_DIR, \"r\"), winslash = \"/\"), \"\n",
301
+ "\")"
302
+ ]
303
+ },
304
+ {
305
+ "cell_type": "markdown",
306
+ "id": "f880c72d",
307
+ "metadata": {
308
+ "id": "f880c72d"
309
+ },
310
+ "source": [
311
+ "## **4.** 🔮 Forecast book sales benchmarking with `accuracy()`"
312
+ ]
313
+ },
314
+ {
315
+ "cell_type": "markdown",
316
+ "source": [
317
+ "We benchmark **three** models on a holdout window (last *h* months):\n",
318
+ "- ARIMA + Fourier (seasonality upgrade)\n",
319
+ "- ETS\n",
320
+ "- Naive baseline\n",
321
+ "\n",
322
+ "Then we export:\n",
323
+ "- `accuracy_table.csv`\n",
324
+ "- `forecast_compare.png`\n",
325
+ "- `rmse_comparison.png`"
326
+ ],
327
+ "metadata": {
328
+ "id": "R0JZlzKegmzW"
329
+ },
330
+ "id": "R0JZlzKegmzW"
331
+ },
332
+ {
333
+ "cell_type": "code",
334
+ "execution_count": 6,
335
+ "id": "62e87992",
336
+ "metadata": {
337
+ "colab": {
338
+ "base_uri": "https://localhost:8080/",
339
+ "height": 82
340
+ },
341
+ "id": "62e87992",
342
+ "outputId": "533ff3fe-2c85-4312-ab04-36007d0cbe1b"
343
+ },
344
+ "outputs": [
345
+ {
346
+ "output_type": "stream",
347
+ "name": "stdout",
348
+ "text": [
349
+ "✅ Saved: artifacts/r/tables/accuracy_table.csv\n",
350
+ "✅ Saved: artifacts/r/figures/rmse_comparison.png\n"
351
+ ]
352
+ },
353
+ {
354
+ "output_type": "display_data",
355
+ "data": {
356
+ "text/html": [
357
+ "<strong>agg_record_805943657:</strong> 2"
358
+ ],
359
+ "text/markdown": "**agg_record_805943657:** 2",
360
+ "text/latex": "\\textbf{agg\\textbackslash{}\\_record\\textbackslash{}\\_805943657:} 2",
361
+ "text/plain": [
362
+ "agg_record_805943657 \n",
363
+ " 2 "
364
+ ]
365
+ },
366
+ "metadata": {}
367
+ },
368
+ {
369
+ "output_type": "stream",
370
+ "name": "stdout",
371
+ "text": [
372
+ "✅ Saved: artifacts/r/figures/forecast_compare.png\n"
373
+ ]
374
+ }
375
+ ],
376
+ "source": [
377
+ "\n",
378
+ "# Build monthly ts\n",
379
+ "start_year <- year(min(df_month2$month, na.rm = TRUE))\n",
380
+ "start_mon <- month(min(df_month2$month, na.rm = TRUE))\n",
381
+ "\n",
382
+ "y <- ts(df_month2$total_revenue, frequency = 12, start = c(start_year, start_mon))\n",
383
+ "\n",
384
+ "# holdout size: min(6, 20% of series), at least 1\n",
385
+ "h_test <- min(6, max(1, floor(length(y) / 5)))\n",
386
+ "train_ts <- head(y, length(y) - h_test)\n",
387
+ "test_ts <- tail(y, h_test)\n",
388
+ "\n",
389
+ "# Model A: ARIMA + Fourier\n",
390
+ "K <- 2\n",
391
+ "xreg_train <- fourier(train_ts, K = K)\n",
392
+ "fit_arima <- auto.arima(train_ts, xreg = xreg_train)\n",
393
+ "xreg_future <- fourier(train_ts, K = K, h = h_test)\n",
394
+ "fc_arima <- forecast(fit_arima, xreg = xreg_future, h = h_test)\n",
395
+ "\n",
396
+ "# Model B: ETS\n",
397
+ "fit_ets <- ets(train_ts)\n",
398
+ "fc_ets <- forecast(fit_ets, h = h_test)\n",
399
+ "\n",
400
+ "# Model C: Naive baseline\n",
401
+ "fc_naive <- naive(train_ts, h = h_test)\n",
402
+ "\n",
403
+ "# accuracy() tables\n",
404
+ "acc_arima <- as.data.frame(accuracy(fc_arima, test_ts))\n",
405
+ "acc_ets <- as.data.frame(accuracy(fc_ets, test_ts))\n",
406
+ "acc_naive <- as.data.frame(accuracy(fc_naive, test_ts))\n",
407
+ "\n",
408
+ "accuracy_tbl <- bind_rows(\n",
409
+ " acc_arima %>% mutate(model = \"ARIMA+Fourier\"),\n",
410
+ " acc_ets %>% mutate(model = \"ETS\"),\n",
411
+ " acc_naive %>% mutate(model = \"Naive\")\n",
412
+ ") %>% relocate(model)\n",
413
+ "\n",
414
+ "write_csv(accuracy_tbl, file.path(R_TAB_DIR, \"accuracy_table.csv\"))\n",
415
+ "cat(\"✅ Saved: artifacts/r/tables/accuracy_table.csv\\n\")\n",
416
+ "\n",
417
+ "# RMSE bar chart\n",
418
+ "p_rmse <- ggplot(accuracy_tbl, aes(x = reorder(model, RMSE), y = RMSE)) +\n",
419
+ " geom_col() +\n",
420
+ " coord_flip() +\n",
421
+ " labs(title = \"Forecast model comparison (RMSE on holdout)\", x = \"\", y = \"RMSE\") +\n",
422
+ " theme_minimal()\n",
423
+ "\n",
424
+ "ggsave(file.path(R_FIG_DIR, \"rmse_comparison.png\"), p_rmse, width = 8, height = 4, dpi = 160)\n",
425
+ "cat(\"✅ Saved: artifacts/r/figures/rmse_comparison.png\\n\")\n",
426
+ "\n",
427
+ "# Side-by-side forecast plots (simple, no extra deps)\n",
428
+ "png(file.path(R_FIG_DIR, \"forecast_compare.png\"), width = 1200, height = 500)\n",
429
+ "par(mfrow = c(1, 3))\n",
430
+ "plot(fc_arima, main = \"ARIMA + Fourier\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
431
+ "plot(fc_ets, main = \"ETS\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
432
+ "plot(fc_naive, main = \"Naive\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
433
+ "dev.off()\n",
434
+ "cat(\"✅ Saved: artifacts/r/figures/forecast_compare.png\\n\")"
435
+ ]
436
+ },
437
+ {
438
+ "cell_type": "markdown",
439
+ "id": "30bc017b",
440
+ "metadata": {
441
+ "id": "30bc017b"
442
+ },
443
+ "source": [
444
+ "## **5.** 💾 Some R metadata for Hugging Face"
445
+ ]
446
+ },
447
+ {
448
+ "cell_type": "code",
449
+ "execution_count": 7,
450
+ "id": "645cb12b",
451
+ "metadata": {
452
+ "colab": {
453
+ "base_uri": "https://localhost:8080/"
454
+ },
455
+ "id": "645cb12b",
456
+ "outputId": "d454e0a5-b986-45ca-9069-65a1c21ce540"
457
+ },
458
+ "outputs": [
459
+ {
460
+ "output_type": "stream",
461
+ "name": "stdout",
462
+ "text": [
463
+ "✅ Saved: artifacts/r/tables/r_meta.json\n",
464
+ "DONE. R artifacts written to: artifacts/r \n"
465
+ ]
466
+ }
467
+ ],
468
+ "source": [
469
+ "# =========================================================\n",
470
+ "# Metadata export (aligned with current notebook objects)\n",
471
+ "# =========================================================\n",
472
+ "\n",
473
+ "meta <- list(\n",
474
+ "\n",
475
+ " # ---------------------------\n",
476
+ " # Dataset footprint\n",
477
+ " # ---------------------------\n",
478
+ " n_titles = nrow(df_title),\n",
479
+ " n_months = nrow(df_month2),\n",
480
+ "\n",
481
+ " # ---------------------------\n",
482
+ " # Forecasting info\n",
483
+ " # (only if these objects exist in your forecasting section)\n",
484
+ " # ---------------------------\n",
485
+ " forecasting = list(\n",
486
+ " holdout_h = h_test,\n",
487
+ " arima_order = forecast::arimaorder(fit_arima),\n",
488
+ " ets_method = fit_ets$method\n",
489
+ " )\n",
490
+ ")\n",
491
+ "\n",
492
+ "jsonlite::write_json(\n",
493
+ " meta,\n",
494
+ " path = file.path(R_TAB_DIR, \"r_meta.json\"),\n",
495
+ " pretty = TRUE,\n",
496
+ " auto_unbox = TRUE\n",
497
+ ")\n",
498
+ "\n",
499
+ "cat(\"✅ Saved: artifacts/r/tables/r_meta.json\\n\")\n",
500
+ "cat(\"DONE. R artifacts written to:\", file.path(ART_DIR, \"r\"), \"\\n\")\n"
501
+ ]
502
+ }
503
+ ],
504
+ "metadata": {
505
+ "colab": {
506
+ "provenance": []
507
+ },
508
+ "kernelspec": {
509
+ "name": "ir",
510
+ "display_name": "R"
511
+ },
512
+ "language_info": {
513
+ "name": "R"
514
+ }
515
+ },
516
+ "nbformat": 4,
517
+ "nbformat_minor": 5
518
+ }