Hugging Face's logo

Spaces:
ESCP
/
RX12WorkshopTemplate
Running

App Files Community
24

Upload 3 files

#7
by ecaragnano - opened
base: refs/heads/main
←
from: refs/pr/7
Discussion Files changed
+5200
-0
Files changed (3) hide show
  1. datacreation.ipynb +1944 -0
  2. pythonanalysis.ipynb +0 -0
  3. ranalysis.ipynb +463 -0
datacreation.ipynb ADDED
@@ -0,0 +1,1944 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "metadata": {
6
+ "id": "4ba6aba8"
7
+ },
8
+ "source": [
9
+ "# 🤖 **Data Collection, Creation, Storage, and Processing**\n"
10
+ ]
11
+ },
12
+ {
13
+ "cell_type": "markdown",
14
+ "metadata": {
15
+ "id": "jpASMyIQMaAq"
16
+ },
17
+ "source": [
18
+ "## **1.** 📦 Install required packages"
19
+ ]
20
+ },
21
+ {
22
+ "cell_type": "code",
23
+ "execution_count": 1,
24
+ "metadata": {
25
+ "colab": {
26
+ "base_uri": "https://localhost:8080/"
27
+ },
28
+ "id": "f48c8f8c",
29
+ "outputId": "f8c7bed7-ef8c-467c-be69-85d59b94608b"
30
+ },
31
+ "outputs": [
32
+ {
33
+ "output_type": "stream",
34
+ "name": "stdout",
35
+ "text": [
36
+ "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.12/dist-packages (4.13.5)\n",
37
+ "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)\n",
38
+ "Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)\n",
39
+ "Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2)\n",
40
+ "Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)\n",
41
+ "Requirement already satisfied: textblob in /usr/local/lib/python3.12/dist-packages (0.19.0)\n",
42
+ "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (2.8.3)\n",
43
+ "Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (4.15.0)\n",
44
+ "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)\n",
45
+ "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)\n",
46
+ "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.3)\n",
47
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)\n",
48
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)\n",
49
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.61.1)\n",
50
+ "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.4.9)\n",
51
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.0)\n",
52
+ "Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)\n",
53
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)\n",
54
+ "Requirement already satisfied: nltk>=3.9 in /usr/local/lib/python3.12/dist-packages (from textblob) (3.9.1)\n",
55
+ "Requirement already satisfied: click in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (8.3.1)\n",
56
+ "Requirement already satisfied: joblib in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (1.5.3)\n",
57
+ "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (2025.11.3)\n",
58
+ "Requirement already satisfied: tqdm in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (4.67.3)\n",
59
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n"
60
+ ]
61
+ }
62
+ ],
63
+ "source": [
64
+ "!pip install beautifulsoup4 pandas matplotlib seaborn numpy textblob"
65
+ ]
66
+ },
67
+ {
68
+ "cell_type": "markdown",
69
+ "metadata": {
70
+ "id": "lquNYCbfL9IM"
71
+ },
72
+ "source": [
73
+ "## **2.** ⛏ Web-scrape all book titles, prices, and ratings from books.toscrape.com"
74
+ ]
75
+ },
76
+ {
77
+ "cell_type": "markdown",
78
+ "metadata": {
79
+ "id": "0IWuNpxxYDJF"
80
+ },
81
+ "source": [
82
+ "### *a. Initial setup*\n",
83
+ "Define the base url of the website you will scrape as well as how and what you will scrape"
84
+ ]
85
+ },
86
+ {
87
+ "cell_type": "code",
88
+ "execution_count": 2,
89
+ "metadata": {
90
+ "id": "91d52125"
91
+ },
92
+ "outputs": [],
93
+ "source": [
94
+ "import requests\n",
95
+ "from bs4 import BeautifulSoup\n",
96
+ "import pandas as pd\n",
97
+ "import time\n",
98
+ "\n",
99
+ "base_url = \"https://books.toscrape.com/catalogue/page-{}.html\"\n",
100
+ "headers = {\"User-Agent\": \"Mozilla/5.0\"}\n",
101
+ "\n",
102
+ "titles, prices, ratings = [], [], []"
103
+ ]
104
+ },
105
+ {
106
+ "cell_type": "markdown",
107
+ "metadata": {
108
+ "id": "oCdTsin2Yfp3"
109
+ },
110
+ "source": [
111
+ "### *b. Fill titles, prices, and ratings from the web pages*"
112
+ ]
113
+ },
114
+ {
115
+ "cell_type": "code",
116
+ "execution_count": 3,
117
+ "metadata": {
118
+ "id": "xqO5Y3dnYhxt"
119
+ },
120
+ "outputs": [],
121
+ "source": [
122
+ "# Loop through all 50 pages\n",
123
+ "for page in range(1, 51):\n",
124
+ " url = base_url.format(page)\n",
125
+ " response = requests.get(url, headers=headers)\n",
126
+ " soup = BeautifulSoup(response.content, \"html.parser\")\n",
127
+ " books = soup.find_all(\"article\", class_=\"product_pod\")\n",
128
+ "\n",
129
+ " for book in books:\n",
130
+ " titles.append(book.h3.a[\"title\"])\n",
131
+ " prices.append(float(book.find(\"p\", class_=\"price_color\").text[1:]))\n",
132
+ " ratings.append(book.p.get(\"class\")[1])\n",
133
+ "\n",
134
+ " time.sleep(0.5) # polite scraping delay"
135
+ ]
136
+ },
137
+ {
138
+ "cell_type": "markdown",
139
+ "metadata": {
140
+ "id": "T0TOeRC4Yrnn"
141
+ },
142
+ "source": [
143
+ "### *c. ✋🏻🛑⛔️ Create a dataframe df_books that contains the now complete \"title\", \"price\", and \"rating\" objects*"
144
+ ]
145
+ },
146
+ {
147
+ "cell_type": "code",
148
+ "execution_count": 5,
149
+ "metadata": {
150
+ "id": "l5FkkNhUYTHh",
151
+ "colab": {
152
+ "base_uri": "https://localhost:8080/",
153
+ "height": 206
154
+ },
155
+ "outputId": "d63c9e7b-72a0-4a4f-c369-6f02a26e3580"
156
+ },
157
+ "outputs": [
158
+ {
159
+ "output_type": "execute_result",
160
+ "data": {
161
+ "text/plain": [
162
+ " title price rating\n",
163
+ "0 A Light in the Attic 51.77 Three\n",
164
+ "1 Tipping the Velvet 53.74 One\n",
165
+ "2 Soumission 50.10 One\n",
166
+ "3 Sharp Objects 47.82 Four\n",
167
+ "4 Sapiens: A Brief History of Humankind 54.23 Five"
168
+ ],
169
+ "text/html": [
170
+ "\n",
171
+ " <div id=\"df-86e37d5d-7584-4f94-af67-8bf904c92d6d\" class=\"colab-df-container\">\n",
172
+ " <div>\n",
173
+ "<style scoped>\n",
174
+ " .dataframe tbody tr th:only-of-type {\n",
175
+ " vertical-align: middle;\n",
176
+ " }\n",
177
+ "\n",
178
+ " .dataframe tbody tr th {\n",
179
+ " vertical-align: top;\n",
180
+ " }\n",
181
+ "\n",
182
+ " .dataframe thead th {\n",
183
+ " text-align: right;\n",
184
+ " }\n",
185
+ "</style>\n",
186
+ "<table border=\"1\" class=\"dataframe\">\n",
187
+ " <thead>\n",
188
+ " <tr style=\"text-align: right;\">\n",
189
+ " <th></th>\n",
190
+ " <th>title</th>\n",
191
+ " <th>price</th>\n",
192
+ " <th>rating</th>\n",
193
+ " </tr>\n",
194
+ " </thead>\n",
195
+ " <tbody>\n",
196
+ " <tr>\n",
197
+ " <th>0</th>\n",
198
+ " <td>A Light in the Attic</td>\n",
199
+ " <td>51.77</td>\n",
200
+ " <td>Three</td>\n",
201
+ " </tr>\n",
202
+ " <tr>\n",
203
+ " <th>1</th>\n",
204
+ " <td>Tipping the Velvet</td>\n",
205
+ " <td>53.74</td>\n",
206
+ " <td>One</td>\n",
207
+ " </tr>\n",
208
+ " <tr>\n",
209
+ " <th>2</th>\n",
210
+ " <td>Soumission</td>\n",
211
+ " <td>50.10</td>\n",
212
+ " <td>One</td>\n",
213
+ " </tr>\n",
214
+ " <tr>\n",
215
+ " <th>3</th>\n",
216
+ " <td>Sharp Objects</td>\n",
217
+ " <td>47.82</td>\n",
218
+ " <td>Four</td>\n",
219
+ " </tr>\n",
220
+ " <tr>\n",
221
+ " <th>4</th>\n",
222
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
223
+ " <td>54.23</td>\n",
224
+ " <td>Five</td>\n",
225
+ " </tr>\n",
226
+ " </tbody>\n",
227
+ "</table>\n",
228
+ "</div>\n",
229
+ " <div class=\"colab-df-buttons\">\n",
230
+ "\n",
231
+ " <div class=\"colab-df-container\">\n",
232
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-86e37d5d-7584-4f94-af67-8bf904c92d6d')\"\n",
233
+ " title=\"Convert this dataframe to an interactive table.\"\n",
234
+ " style=\"display:none;\">\n",
235
+ "\n",
236
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
237
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
238
+ " </svg>\n",
239
+ " </button>\n",
240
+ "\n",
241
+ " <style>\n",
242
+ " .colab-df-container {\n",
243
+ " display:flex;\n",
244
+ " gap: 12px;\n",
245
+ " }\n",
246
+ "\n",
247
+ " .colab-df-convert {\n",
248
+ " background-color: #E8F0FE;\n",
249
+ " border: none;\n",
250
+ " border-radius: 50%;\n",
251
+ " cursor: pointer;\n",
252
+ " display: none;\n",
253
+ " fill: #1967D2;\n",
254
+ " height: 32px;\n",
255
+ " padding: 0 0 0 0;\n",
256
+ " width: 32px;\n",
257
+ " }\n",
258
+ "\n",
259
+ " .colab-df-convert:hover {\n",
260
+ " background-color: #E2EBFA;\n",
261
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
262
+ " fill: #174EA6;\n",
263
+ " }\n",
264
+ "\n",
265
+ " .colab-df-buttons div {\n",
266
+ " margin-bottom: 4px;\n",
267
+ " }\n",
268
+ "\n",
269
+ " [theme=dark] .colab-df-convert {\n",
270
+ " background-color: #3B4455;\n",
271
+ " fill: #D2E3FC;\n",
272
+ " }\n",
273
+ "\n",
274
+ " [theme=dark] .colab-df-convert:hover {\n",
275
+ " background-color: #434B5C;\n",
276
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
277
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
278
+ " fill: #FFFFFF;\n",
279
+ " }\n",
280
+ " </style>\n",
281
+ "\n",
282
+ " <script>\n",
283
+ " const buttonEl =\n",
284
+ " document.querySelector('#df-86e37d5d-7584-4f94-af67-8bf904c92d6d button.colab-df-convert');\n",
285
+ " buttonEl.style.display =\n",
286
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
287
+ "\n",
288
+ " async function convertToInteractive(key) {\n",
289
+ " const element = document.querySelector('#df-86e37d5d-7584-4f94-af67-8bf904c92d6d');\n",
290
+ " const dataTable =\n",
291
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
292
+ " [key], {});\n",
293
+ " if (!dataTable) return;\n",
294
+ "\n",
295
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
296
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
297
+ " + ' to learn more about interactive tables.';\n",
298
+ " element.innerHTML = '';\n",
299
+ " dataTable['output_type'] = 'display_data';\n",
300
+ " await google.colab.output.renderOutput(dataTable, element);\n",
301
+ " const docLink = document.createElement('div');\n",
302
+ " docLink.innerHTML = docLinkHtml;\n",
303
+ " element.appendChild(docLink);\n",
304
+ " }\n",
305
+ " </script>\n",
306
+ " </div>\n",
307
+ "\n",
308
+ "\n",
309
+ " </div>\n",
310
+ " </div>\n"
311
+ ],
312
+ "application/vnd.google.colaboratory.intrinsic+json": {
313
+ "type": "dataframe",
314
+ "variable_name": "df_books",
315
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
316
+ }
317
+ },
318
+ "metadata": {},
319
+ "execution_count": 5
320
+ }
321
+ ],
322
+ "source": [
323
+ "df_books = pd.DataFrame({\n",
324
+ " \"title\": titles,\n",
325
+ " \"price\": prices,\n",
326
+ " \"rating\": ratings\n",
327
+ "})\n",
328
+ "\n",
329
+ "df_books.head()\n"
330
+ ]
331
+ },
332
+ {
333
+ "cell_type": "markdown",
334
+ "metadata": {
335
+ "id": "duI5dv3CZYvF"
336
+ },
337
+ "source": [
338
+ "### *d. Save web-scraped dataframe either as a CSV or Excel file*"
339
+ ]
340
+ },
341
+ {
342
+ "cell_type": "code",
343
+ "execution_count": 6,
344
+ "metadata": {
345
+ "id": "lC1U_YHtZifh"
346
+ },
347
+ "outputs": [],
348
+ "source": [
349
+ "# 💾 Save to CSV\n",
350
+ "df_books.to_csv(\"books_data.csv\", index=False)\n",
351
+ "\n",
352
+ "# 💾 Or save to Excel\n",
353
+ "# df_books.to_excel(\"books_data.xlsx\", index=False)"
354
+ ]
355
+ },
356
+ {
357
+ "cell_type": "markdown",
358
+ "metadata": {
359
+ "id": "qMjRKMBQZlJi"
360
+ },
361
+ "source": [
362
+ "### *e. ✋🏻🛑⛔️ View first fiew lines*"
363
+ ]
364
+ },
365
+ {
366
+ "cell_type": "code",
367
+ "execution_count": 7,
368
+ "metadata": {
369
+ "colab": {
370
+ "base_uri": "https://localhost:8080/",
371
+ "height": 206
372
+ },
373
+ "id": "O_wIvTxYZqCK",
374
+ "outputId": "a4d5e4ee-08bf-406c-cf51-f103a4835e63"
375
+ },
376
+ "outputs": [
377
+ {
378
+ "output_type": "execute_result",
379
+ "data": {
380
+ "text/plain": [
381
+ " title price rating\n",
382
+ "0 A Light in the Attic 51.77 Three\n",
383
+ "1 Tipping the Velvet 53.74 One\n",
384
+ "2 Soumission 50.10 One\n",
385
+ "3 Sharp Objects 47.82 Four\n",
386
+ "4 Sapiens: A Brief History of Humankind 54.23 Five"
387
+ ],
388
+ "text/html": [
389
+ "\n",
390
+ " <div id=\"df-26a92760-9c93-4e97-b654-c225aba212cc\" class=\"colab-df-container\">\n",
391
+ " <div>\n",
392
+ "<style scoped>\n",
393
+ " .dataframe tbody tr th:only-of-type {\n",
394
+ " vertical-align: middle;\n",
395
+ " }\n",
396
+ "\n",
397
+ " .dataframe tbody tr th {\n",
398
+ " vertical-align: top;\n",
399
+ " }\n",
400
+ "\n",
401
+ " .dataframe thead th {\n",
402
+ " text-align: right;\n",
403
+ " }\n",
404
+ "</style>\n",
405
+ "<table border=\"1\" class=\"dataframe\">\n",
406
+ " <thead>\n",
407
+ " <tr style=\"text-align: right;\">\n",
408
+ " <th></th>\n",
409
+ " <th>title</th>\n",
410
+ " <th>price</th>\n",
411
+ " <th>rating</th>\n",
412
+ " </tr>\n",
413
+ " </thead>\n",
414
+ " <tbody>\n",
415
+ " <tr>\n",
416
+ " <th>0</th>\n",
417
+ " <td>A Light in the Attic</td>\n",
418
+ " <td>51.77</td>\n",
419
+ " <td>Three</td>\n",
420
+ " </tr>\n",
421
+ " <tr>\n",
422
+ " <th>1</th>\n",
423
+ " <td>Tipping the Velvet</td>\n",
424
+ " <td>53.74</td>\n",
425
+ " <td>One</td>\n",
426
+ " </tr>\n",
427
+ " <tr>\n",
428
+ " <th>2</th>\n",
429
+ " <td>Soumission</td>\n",
430
+ " <td>50.10</td>\n",
431
+ " <td>One</td>\n",
432
+ " </tr>\n",
433
+ " <tr>\n",
434
+ " <th>3</th>\n",
435
+ " <td>Sharp Objects</td>\n",
436
+ " <td>47.82</td>\n",
437
+ " <td>Four</td>\n",
438
+ " </tr>\n",
439
+ " <tr>\n",
440
+ " <th>4</th>\n",
441
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
442
+ " <td>54.23</td>\n",
443
+ " <td>Five</td>\n",
444
+ " </tr>\n",
445
+ " </tbody>\n",
446
+ "</table>\n",
447
+ "</div>\n",
448
+ " <div class=\"colab-df-buttons\">\n",
449
+ "\n",
450
+ " <div class=\"colab-df-container\">\n",
451
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-26a92760-9c93-4e97-b654-c225aba212cc')\"\n",
452
+ " title=\"Convert this dataframe to an interactive table.\"\n",
453
+ " style=\"display:none;\">\n",
454
+ "\n",
455
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
456
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
457
+ " </svg>\n",
458
+ " </button>\n",
459
+ "\n",
460
+ " <style>\n",
461
+ " .colab-df-container {\n",
462
+ " display:flex;\n",
463
+ " gap: 12px;\n",
464
+ " }\n",
465
+ "\n",
466
+ " .colab-df-convert {\n",
467
+ " background-color: #E8F0FE;\n",
468
+ " border: none;\n",
469
+ " border-radius: 50%;\n",
470
+ " cursor: pointer;\n",
471
+ " display: none;\n",
472
+ " fill: #1967D2;\n",
473
+ " height: 32px;\n",
474
+ " padding: 0 0 0 0;\n",
475
+ " width: 32px;\n",
476
+ " }\n",
477
+ "\n",
478
+ " .colab-df-convert:hover {\n",
479
+ " background-color: #E2EBFA;\n",
480
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
481
+ " fill: #174EA6;\n",
482
+ " }\n",
483
+ "\n",
484
+ " .colab-df-buttons div {\n",
485
+ " margin-bottom: 4px;\n",
486
+ " }\n",
487
+ "\n",
488
+ " [theme=dark] .colab-df-convert {\n",
489
+ " background-color: #3B4455;\n",
490
+ " fill: #D2E3FC;\n",
491
+ " }\n",
492
+ "\n",
493
+ " [theme=dark] .colab-df-convert:hover {\n",
494
+ " background-color: #434B5C;\n",
495
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
496
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
497
+ " fill: #FFFFFF;\n",
498
+ " }\n",
499
+ " </style>\n",
500
+ "\n",
501
+ " <script>\n",
502
+ " const buttonEl =\n",
503
+ " document.querySelector('#df-26a92760-9c93-4e97-b654-c225aba212cc button.colab-df-convert');\n",
504
+ " buttonEl.style.display =\n",
505
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
506
+ "\n",
507
+ " async function convertToInteractive(key) {\n",
508
+ " const element = document.querySelector('#df-26a92760-9c93-4e97-b654-c225aba212cc');\n",
509
+ " const dataTable =\n",
510
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
511
+ " [key], {});\n",
512
+ " if (!dataTable) return;\n",
513
+ "\n",
514
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
515
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
516
+ " + ' to learn more about interactive tables.';\n",
517
+ " element.innerHTML = '';\n",
518
+ " dataTable['output_type'] = 'display_data';\n",
519
+ " await google.colab.output.renderOutput(dataTable, element);\n",
520
+ " const docLink = document.createElement('div');\n",
521
+ " docLink.innerHTML = docLinkHtml;\n",
522
+ " element.appendChild(docLink);\n",
523
+ " }\n",
524
+ " </script>\n",
525
+ " </div>\n",
526
+ "\n",
527
+ "\n",
528
+ " </div>\n",
529
+ " </div>\n"
530
+ ],
531
+ "application/vnd.google.colaboratory.intrinsic+json": {
532
+ "type": "dataframe",
533
+ "variable_name": "df_books",
534
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
535
+ }
536
+ },
537
+ "metadata": {},
538
+ "execution_count": 7
539
+ }
540
+ ],
541
+ "source": [
542
+ "df_books.head()"
543
+ ]
544
+ },
545
+ {
546
+ "cell_type": "markdown",
547
+ "metadata": {
548
+ "id": "p-1Pr2szaqLk"
549
+ },
550
+ "source": [
551
+ "## **3.** 🧩 Create a meaningful connection between real & synthetic datasets"
552
+ ]
553
+ },
554
+ {
555
+ "cell_type": "markdown",
556
+ "metadata": {
557
+ "id": "SIaJUGIpaH4V"
558
+ },
559
+ "source": [
560
+ "### *a. Initial setup*"
561
+ ]
562
+ },
563
+ {
564
+ "cell_type": "code",
565
+ "execution_count": 8,
566
+ "metadata": {
567
+ "id": "-gPXGcRPuV_9"
568
+ },
569
+ "outputs": [],
570
+ "source": [
571
+ "import numpy as np\n",
572
+ "import random\n",
573
+ "from datetime import datetime\n",
574
+ "import warnings\n",
575
+ "\n",
576
+ "warnings.filterwarnings(\"ignore\")\n",
577
+ "random.seed(2025)\n",
578
+ "np.random.seed(2025)"
579
+ ]
580
+ },
581
+ {
582
+ "cell_type": "markdown",
583
+ "metadata": {
584
+ "id": "pY4yCoIuaQqp"
585
+ },
586
+ "source": [
587
+ "### *b. Generate popularity scores based on rating (with some randomness) with a generate_popularity_score function*"
588
+ ]
589
+ },
590
+ {
591
+ "cell_type": "code",
592
+ "execution_count": 9,
593
+ "metadata": {
594
+ "id": "mnd5hdAbaNjz"
595
+ },
596
+ "outputs": [],
597
+ "source": [
598
+ "def generate_popularity_score(rating):\n",
599
+ " base = {\"One\": 2, \"Two\": 3, \"Three\": 3, \"Four\": 4, \"Five\": 4}.get(rating, 3)\n",
600
+ " trend_factor = random.choices([-1, 0, 1], weights=[1, 3, 2])[0]\n",
601
+ " return int(np.clip(base + trend_factor, 1, 5))"
602
+ ]
603
+ },
604
+ {
605
+ "cell_type": "markdown",
606
+ "metadata": {
607
+ "id": "n4-TaNTFgPak"
608
+ },
609
+ "source": [
610
+ "### *c. ✋🏻🛑⛔️ Run the function to create a \"popularity_score\" column from \"rating\"*"
611
+ ]
612
+ },
613
+ {
614
+ "cell_type": "code",
615
+ "execution_count": 10,
616
+ "metadata": {
617
+ "id": "V-G3OCUCgR07",
618
+ "colab": {
619
+ "base_uri": "https://localhost:8080/",
620
+ "height": 206
621
+ },
622
+ "outputId": "8d4003d9-2f6d-4a60-bef3-ac962491e5a5"
623
+ },
624
+ "outputs": [
625
+ {
626
+ "output_type": "execute_result",
627
+ "data": {
628
+ "text/plain": [
629
+ " title price rating popularity_score\n",
630
+ "0 A Light in the Attic 51.77 Three 3\n",
631
+ "1 Tipping the Velvet 53.74 One 2\n",
632
+ "2 Soumission 50.10 One 2\n",
633
+ "3 Sharp Objects 47.82 Four 4\n",
634
+ "4 Sapiens: A Brief History of Humankind 54.23 Five 3"
635
+ ],
636
+ "text/html": [
637
+ "\n",
638
+ " <div id=\"df-9f2cbad8-8a7e-42d2-b8f3-9cb0ca8a22bf\" class=\"colab-df-container\">\n",
639
+ " <div>\n",
640
+ "<style scoped>\n",
641
+ " .dataframe tbody tr th:only-of-type {\n",
642
+ " vertical-align: middle;\n",
643
+ " }\n",
644
+ "\n",
645
+ " .dataframe tbody tr th {\n",
646
+ " vertical-align: top;\n",
647
+ " }\n",
648
+ "\n",
649
+ " .dataframe thead th {\n",
650
+ " text-align: right;\n",
651
+ " }\n",
652
+ "</style>\n",
653
+ "<table border=\"1\" class=\"dataframe\">\n",
654
+ " <thead>\n",
655
+ " <tr style=\"text-align: right;\">\n",
656
+ " <th></th>\n",
657
+ " <th>title</th>\n",
658
+ " <th>price</th>\n",
659
+ " <th>rating</th>\n",
660
+ " <th>popularity_score</th>\n",
661
+ " </tr>\n",
662
+ " </thead>\n",
663
+ " <tbody>\n",
664
+ " <tr>\n",
665
+ " <th>0</th>\n",
666
+ " <td>A Light in the Attic</td>\n",
667
+ " <td>51.77</td>\n",
668
+ " <td>Three</td>\n",
669
+ " <td>3</td>\n",
670
+ " </tr>\n",
671
+ " <tr>\n",
672
+ " <th>1</th>\n",
673
+ " <td>Tipping the Velvet</td>\n",
674
+ " <td>53.74</td>\n",
675
+ " <td>One</td>\n",
676
+ " <td>2</td>\n",
677
+ " </tr>\n",
678
+ " <tr>\n",
679
+ " <th>2</th>\n",
680
+ " <td>Soumission</td>\n",
681
+ " <td>50.10</td>\n",
682
+ " <td>One</td>\n",
683
+ " <td>2</td>\n",
684
+ " </tr>\n",
685
+ " <tr>\n",
686
+ " <th>3</th>\n",
687
+ " <td>Sharp Objects</td>\n",
688
+ " <td>47.82</td>\n",
689
+ " <td>Four</td>\n",
690
+ " <td>4</td>\n",
691
+ " </tr>\n",
692
+ " <tr>\n",
693
+ " <th>4</th>\n",
694
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
695
+ " <td>54.23</td>\n",
696
+ " <td>Five</td>\n",
697
+ " <td>3</td>\n",
698
+ " </tr>\n",
699
+ " </tbody>\n",
700
+ "</table>\n",
701
+ "</div>\n",
702
+ " <div class=\"colab-df-buttons\">\n",
703
+ "\n",
704
+ " <div class=\"colab-df-container\">\n",
705
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9f2cbad8-8a7e-42d2-b8f3-9cb0ca8a22bf')\"\n",
706
+ " title=\"Convert this dataframe to an interactive table.\"\n",
707
+ " style=\"display:none;\">\n",
708
+ "\n",
709
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
710
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
711
+ " </svg>\n",
712
+ " </button>\n",
713
+ "\n",
714
+ " <style>\n",
715
+ " .colab-df-container {\n",
716
+ " display:flex;\n",
717
+ " gap: 12px;\n",
718
+ " }\n",
719
+ "\n",
720
+ " .colab-df-convert {\n",
721
+ " background-color: #E8F0FE;\n",
722
+ " border: none;\n",
723
+ " border-radius: 50%;\n",
724
+ " cursor: pointer;\n",
725
+ " display: none;\n",
726
+ " fill: #1967D2;\n",
727
+ " height: 32px;\n",
728
+ " padding: 0 0 0 0;\n",
729
+ " width: 32px;\n",
730
+ " }\n",
731
+ "\n",
732
+ " .colab-df-convert:hover {\n",
733
+ " background-color: #E2EBFA;\n",
734
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
735
+ " fill: #174EA6;\n",
736
+ " }\n",
737
+ "\n",
738
+ " .colab-df-buttons div {\n",
739
+ " margin-bottom: 4px;\n",
740
+ " }\n",
741
+ "\n",
742
+ " [theme=dark] .colab-df-convert {\n",
743
+ " background-color: #3B4455;\n",
744
+ " fill: #D2E3FC;\n",
745
+ " }\n",
746
+ "\n",
747
+ " [theme=dark] .colab-df-convert:hover {\n",
748
+ " background-color: #434B5C;\n",
749
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
750
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
751
+ " fill: #FFFFFF;\n",
752
+ " }\n",
753
+ " </style>\n",
754
+ "\n",
755
+ " <script>\n",
756
+ " const buttonEl =\n",
757
+ " document.querySelector('#df-9f2cbad8-8a7e-42d2-b8f3-9cb0ca8a22bf button.colab-df-convert');\n",
758
+ " buttonEl.style.display =\n",
759
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
760
+ "\n",
761
+ " async function convertToInteractive(key) {\n",
762
+ " const element = document.querySelector('#df-9f2cbad8-8a7e-42d2-b8f3-9cb0ca8a22bf');\n",
763
+ " const dataTable =\n",
764
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
765
+ " [key], {});\n",
766
+ " if (!dataTable) return;\n",
767
+ "\n",
768
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
769
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
770
+ " + ' to learn more about interactive tables.';\n",
771
+ " element.innerHTML = '';\n",
772
+ " dataTable['output_type'] = 'display_data';\n",
773
+ " await google.colab.output.renderOutput(dataTable, element);\n",
774
+ " const docLink = document.createElement('div');\n",
775
+ " docLink.innerHTML = docLinkHtml;\n",
776
+ " element.appendChild(docLink);\n",
777
+ " }\n",
778
+ " </script>\n",
779
+ " </div>\n",
780
+ "\n",
781
+ "\n",
782
+ " </div>\n",
783
+ " </div>\n"
784
+ ],
785
+ "application/vnd.google.colaboratory.intrinsic+json": {
786
+ "type": "dataframe",
787
+ "variable_name": "df_books",
788
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
789
+ }
790
+ },
791
+ "metadata": {},
792
+ "execution_count": 10
793
+ }
794
+ ],
795
+ "source": [
796
+ "df_books[\"popularity_score\"] = df_books[\"rating\"].apply(generate_popularity_score)\n",
797
+ "\n",
798
+ "df_books.head()\n"
799
+ ]
800
+ },
801
+ {
802
+ "cell_type": "markdown",
803
+ "metadata": {
804
+ "id": "HnngRNTgacYt"
805
+ },
806
+ "source": [
807
+ "### *d. Decide on the sentiment_label based on the popularity score with a get_sentiment function*"
808
+ ]
809
+ },
810
+ {
811
+ "cell_type": "code",
812
+ "execution_count": 12,
813
+ "metadata": {
814
+ "id": "kUtWmr8maZLZ"
815
+ },
816
+ "outputs": [],
817
+ "source": [
818
+ "def get_sentiment(popularity_score):\n",
819
+ " if popularity_score <= 2:\n",
820
+ " return \"negative\"\n",
821
+ " elif popularity_score == 3:\n",
822
+ " return \"neutral\"\n",
823
+ " else:\n",
824
+ " return \"positive\""
825
+ ]
826
+ },
827
+ {
828
+ "cell_type": "markdown",
829
+ "metadata": {
830
+ "id": "HF9F9HIzgT7Z"
831
+ },
832
+ "source": [
833
+ "### *e. ✋🏻🛑⛔️ Run the function to create a \"sentiment_label\" column from \"popularity_score\"*"
834
+ ]
835
+ },
836
+ {
837
+ "cell_type": "code",
838
+ "execution_count": 13,
839
+ "metadata": {
840
+ "id": "tafQj8_7gYCG",
841
+ "colab": {
842
+ "base_uri": "https://localhost:8080/",
843
+ "height": 206
844
+ },
845
+ "outputId": "997d7c6a-ad6a-49e6-dfd1-02a64481d2bc"
846
+ },
847
+ "outputs": [
848
+ {
849
+ "output_type": "execute_result",
850
+ "data": {
851
+ "text/plain": [
852
+ " title price rating popularity_score \\\n",
853
+ "0 A Light in the Attic 51.77 Three 3 \n",
854
+ "1 Tipping the Velvet 53.74 One 2 \n",
855
+ "2 Soumission 50.10 One 2 \n",
856
+ "3 Sharp Objects 47.82 Four 4 \n",
857
+ "4 Sapiens: A Brief History of Humankind 54.23 Five 3 \n",
858
+ "\n",
859
+ " sentiment_label \n",
860
+ "0 neutral \n",
861
+ "1 negative \n",
862
+ "2 negative \n",
863
+ "3 positive \n",
864
+ "4 neutral "
865
+ ],
866
+ "text/html": [
867
+ "\n",
868
+ " <div id=\"df-99679edd-c51f-4c3c-8520-69b9e326dda4\" class=\"colab-df-container\">\n",
869
+ " <div>\n",
870
+ "<style scoped>\n",
871
+ " .dataframe tbody tr th:only-of-type {\n",
872
+ " vertical-align: middle;\n",
873
+ " }\n",
874
+ "\n",
875
+ " .dataframe tbody tr th {\n",
876
+ " vertical-align: top;\n",
877
+ " }\n",
878
+ "\n",
879
+ " .dataframe thead th {\n",
880
+ " text-align: right;\n",
881
+ " }\n",
882
+ "</style>\n",
883
+ "<table border=\"1\" class=\"dataframe\">\n",
884
+ " <thead>\n",
885
+ " <tr style=\"text-align: right;\">\n",
886
+ " <th></th>\n",
887
+ " <th>title</th>\n",
888
+ " <th>price</th>\n",
889
+ " <th>rating</th>\n",
890
+ " <th>popularity_score</th>\n",
891
+ " <th>sentiment_label</th>\n",
892
+ " </tr>\n",
893
+ " </thead>\n",
894
+ " <tbody>\n",
895
+ " <tr>\n",
896
+ " <th>0</th>\n",
897
+ " <td>A Light in the Attic</td>\n",
898
+ " <td>51.77</td>\n",
899
+ " <td>Three</td>\n",
900
+ " <td>3</td>\n",
901
+ " <td>neutral</td>\n",
902
+ " </tr>\n",
903
+ " <tr>\n",
904
+ " <th>1</th>\n",
905
+ " <td>Tipping the Velvet</td>\n",
906
+ " <td>53.74</td>\n",
907
+ " <td>One</td>\n",
908
+ " <td>2</td>\n",
909
+ " <td>negative</td>\n",
910
+ " </tr>\n",
911
+ " <tr>\n",
912
+ " <th>2</th>\n",
913
+ " <td>Soumission</td>\n",
914
+ " <td>50.10</td>\n",
915
+ " <td>One</td>\n",
916
+ " <td>2</td>\n",
917
+ " <td>negative</td>\n",
918
+ " </tr>\n",
919
+ " <tr>\n",
920
+ " <th>3</th>\n",
921
+ " <td>Sharp Objects</td>\n",
922
+ " <td>47.82</td>\n",
923
+ " <td>Four</td>\n",
924
+ " <td>4</td>\n",
925
+ " <td>positive</td>\n",
926
+ " </tr>\n",
927
+ " <tr>\n",
928
+ " <th>4</th>\n",
929
+ " <td>Sapiens: A Brief History of Humankind</td>\n",
930
+ " <td>54.23</td>\n",
931
+ " <td>Five</td>\n",
932
+ " <td>3</td>\n",
933
+ " <td>neutral</td>\n",
934
+ " </tr>\n",
935
+ " </tbody>\n",
936
+ "</table>\n",
937
+ "</div>\n",
938
+ " <div class=\"colab-df-buttons\">\n",
939
+ "\n",
940
+ " <div class=\"colab-df-container\">\n",
941
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-99679edd-c51f-4c3c-8520-69b9e326dda4')\"\n",
942
+ " title=\"Convert this dataframe to an interactive table.\"\n",
943
+ " style=\"display:none;\">\n",
944
+ "\n",
945
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
946
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
947
+ " </svg>\n",
948
+ " </button>\n",
949
+ "\n",
950
+ " <style>\n",
951
+ " .colab-df-container {\n",
952
+ " display:flex;\n",
953
+ " gap: 12px;\n",
954
+ " }\n",
955
+ "\n",
956
+ " .colab-df-convert {\n",
957
+ " background-color: #E8F0FE;\n",
958
+ " border: none;\n",
959
+ " border-radius: 50%;\n",
960
+ " cursor: pointer;\n",
961
+ " display: none;\n",
962
+ " fill: #1967D2;\n",
963
+ " height: 32px;\n",
964
+ " padding: 0 0 0 0;\n",
965
+ " width: 32px;\n",
966
+ " }\n",
967
+ "\n",
968
+ " .colab-df-convert:hover {\n",
969
+ " background-color: #E2EBFA;\n",
970
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
971
+ " fill: #174EA6;\n",
972
+ " }\n",
973
+ "\n",
974
+ " .colab-df-buttons div {\n",
975
+ " margin-bottom: 4px;\n",
976
+ " }\n",
977
+ "\n",
978
+ " [theme=dark] .colab-df-convert {\n",
979
+ " background-color: #3B4455;\n",
980
+ " fill: #D2E3FC;\n",
981
+ " }\n",
982
+ "\n",
983
+ " [theme=dark] .colab-df-convert:hover {\n",
984
+ " background-color: #434B5C;\n",
985
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
986
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
987
+ " fill: #FFFFFF;\n",
988
+ " }\n",
989
+ " </style>\n",
990
+ "\n",
991
+ " <script>\n",
992
+ " const buttonEl =\n",
993
+ " document.querySelector('#df-99679edd-c51f-4c3c-8520-69b9e326dda4 button.colab-df-convert');\n",
994
+ " buttonEl.style.display =\n",
995
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
996
+ "\n",
997
+ " async function convertToInteractive(key) {\n",
998
+ " const element = document.querySelector('#df-99679edd-c51f-4c3c-8520-69b9e326dda4');\n",
999
+ " const dataTable =\n",
1000
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1001
+ " [key], {});\n",
1002
+ " if (!dataTable) return;\n",
1003
+ "\n",
1004
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1005
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1006
+ " + ' to learn more about interactive tables.';\n",
1007
+ " element.innerHTML = '';\n",
1008
+ " dataTable['output_type'] = 'display_data';\n",
1009
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1010
+ " const docLink = document.createElement('div');\n",
1011
+ " docLink.innerHTML = docLinkHtml;\n",
1012
+ " element.appendChild(docLink);\n",
1013
+ " }\n",
1014
+ " </script>\n",
1015
+ " </div>\n",
1016
+ "\n",
1017
+ "\n",
1018
+ " </div>\n",
1019
+ " </div>\n"
1020
+ ],
1021
+ "application/vnd.google.colaboratory.intrinsic+json": {
1022
+ "type": "dataframe",
1023
+ "variable_name": "df_books",
1024
+ "summary": "{\n \"name\": \"df_books\",\n \"rows\": 1000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 14.446689669952772,\n \"min\": 10.0,\n \"max\": 59.99,\n \"num_unique_values\": 903,\n \"samples\": [\n 19.73,\n 55.65,\n 46.31\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"One\",\n \"Two\",\n \"Four\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity_score\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 2,\n 5,\n 4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1025
+ }
1026
+ },
1027
+ "metadata": {},
1028
+ "execution_count": 13
1029
+ }
1030
+ ],
1031
+ "source": [
1032
+ "df_books[\"sentiment_label\"] = df_books[\"popularity_score\"].apply(get_sentiment)\n",
1033
+ "\n",
1034
+ "df_books.head()\n"
1035
+ ]
1036
+ },
1037
+ {
1038
+ "cell_type": "markdown",
1039
+ "metadata": {
1040
+ "id": "T8AdKkmASq9a"
1041
+ },
1042
+ "source": [
1043
+ "## **4.** 📈 Generate synthetic book sales data of 18 months"
1044
+ ]
1045
+ },
1046
+ {
1047
+ "cell_type": "markdown",
1048
+ "metadata": {
1049
+ "id": "OhXbdGD5fH0c"
1050
+ },
1051
+ "source": [
1052
+ "### *a. Create a generate_sales_profit function that would generate sales patterns based on sentiment_label (with some randomness)*"
1053
+ ]
1054
+ },
1055
+ {
1056
+ "cell_type": "code",
1057
+ "execution_count": 16,
1058
+ "metadata": {
1059
+ "id": "qkVhYPXGbgEn"
1060
+ },
1061
+ "outputs": [],
1062
+ "source": [
1063
+ "def generate_sales_profile(sentiment):\n",
1064
+ " months = pd.date_range(end=datetime.today(), periods=18, freq=\"M\")\n",
1065
+ "\n",
1066
+ " if sentiment == \"positive\":\n",
1067
+ " base = random.randint(200, 300)\n",
1068
+ " trend = np.linspace(base, base + random.randint(20, 60), len(months))\n",
1069
+ " elif sentiment == \"negative\":\n",
1070
+ " base = random.randint(20, 80)\n",
1071
+ " trend = np.linspace(base, base - random.randint(10, 30), len(months))\n",
1072
+ " else: # neutral\n",
1073
+ " base = random.randint(80, 160)\n",
1074
+ " trend = np.full(len(months), base + random.randint(-10, 10))\n",
1075
+ "\n",
1076
+ " seasonality = 10 * np.sin(np.linspace(0, 3 * np.pi, len(months)))\n",
1077
+ " noise = np.random.normal(0, 5, len(months))\n",
1078
+ " monthly_sales = np.clip(trend + seasonality + noise, a_min=0, a_max=None).astype(int)\n",
1079
+ "\n",
1080
+ " return list(zip(months.strftime(\"%Y-%m\"), monthly_sales))"
1081
+ ]
1082
+ },
1083
+ {
1084
+ "cell_type": "markdown",
1085
+ "metadata": {
1086
+ "id": "L2ak1HlcgoTe"
1087
+ },
1088
+ "source": [
1089
+ "### *b. Run the function as part of building sales_data*"
1090
+ ]
1091
+ },
1092
+ {
1093
+ "cell_type": "code",
1094
+ "execution_count": 17,
1095
+ "metadata": {
1096
+ "id": "SlJ24AUafoDB"
1097
+ },
1098
+ "outputs": [],
1099
+ "source": [
1100
+ "sales_data = []\n",
1101
+ "for _, row in df_books.iterrows():\n",
1102
+ " records = generate_sales_profile(row[\"sentiment_label\"])\n",
1103
+ " for month, units in records:\n",
1104
+ " sales_data.append({\n",
1105
+ " \"title\": row[\"title\"],\n",
1106
+ " \"month\": month,\n",
1107
+ " \"units_sold\": units,\n",
1108
+ " \"sentiment_label\": row[\"sentiment_label\"]\n",
1109
+ " })"
1110
+ ]
1111
+ },
1112
+ {
1113
+ "cell_type": "markdown",
1114
+ "metadata": {
1115
+ "id": "4IXZKcCSgxnq"
1116
+ },
1117
+ "source": [
1118
+ "### *c. ✋🏻🛑⛔️ Create a df_sales DataFrame from sales_data*"
1119
+ ]
1120
+ },
1121
+ {
1122
+ "cell_type": "code",
1123
+ "execution_count": 18,
1124
+ "metadata": {
1125
+ "id": "wcN6gtiZg-ws",
1126
+ "colab": {
1127
+ "base_uri": "https://localhost:8080/",
1128
+ "height": 206
1129
+ },
1130
+ "outputId": "dc9a9c78-97fe-4e8e-b03a-5605c88794c4"
1131
+ },
1132
+ "outputs": [
1133
+ {
1134
+ "output_type": "execute_result",
1135
+ "data": {
1136
+ "text/plain": [
1137
+ " title month units_sold sentiment_label\n",
1138
+ "0 A Light in the Attic 2024-08 100 neutral\n",
1139
+ "1 A Light in the Attic 2024-09 109 neutral\n",
1140
+ "2 A Light in the Attic 2024-10 102 neutral\n",
1141
+ "3 A Light in the Attic 2024-11 107 neutral\n",
1142
+ "4 A Light in the Attic 2024-12 108 neutral"
1143
+ ],
1144
+ "text/html": [
1145
+ "\n",
1146
+ " <div id=\"df-57178409-c5be-4481-bb4c-bc458aebbcad\" class=\"colab-df-container\">\n",
1147
+ " <div>\n",
1148
+ "<style scoped>\n",
1149
+ " .dataframe tbody tr th:only-of-type {\n",
1150
+ " vertical-align: middle;\n",
1151
+ " }\n",
1152
+ "\n",
1153
+ " .dataframe tbody tr th {\n",
1154
+ " vertical-align: top;\n",
1155
+ " }\n",
1156
+ "\n",
1157
+ " .dataframe thead th {\n",
1158
+ " text-align: right;\n",
1159
+ " }\n",
1160
+ "</style>\n",
1161
+ "<table border=\"1\" class=\"dataframe\">\n",
1162
+ " <thead>\n",
1163
+ " <tr style=\"text-align: right;\">\n",
1164
+ " <th></th>\n",
1165
+ " <th>title</th>\n",
1166
+ " <th>month</th>\n",
1167
+ " <th>units_sold</th>\n",
1168
+ " <th>sentiment_label</th>\n",
1169
+ " </tr>\n",
1170
+ " </thead>\n",
1171
+ " <tbody>\n",
1172
+ " <tr>\n",
1173
+ " <th>0</th>\n",
1174
+ " <td>A Light in the Attic</td>\n",
1175
+ " <td>2024-08</td>\n",
1176
+ " <td>100</td>\n",
1177
+ " <td>neutral</td>\n",
1178
+ " </tr>\n",
1179
+ " <tr>\n",
1180
+ " <th>1</th>\n",
1181
+ " <td>A Light in the Attic</td>\n",
1182
+ " <td>2024-09</td>\n",
1183
+ " <td>109</td>\n",
1184
+ " <td>neutral</td>\n",
1185
+ " </tr>\n",
1186
+ " <tr>\n",
1187
+ " <th>2</th>\n",
1188
+ " <td>A Light in the Attic</td>\n",
1189
+ " <td>2024-10</td>\n",
1190
+ " <td>102</td>\n",
1191
+ " <td>neutral</td>\n",
1192
+ " </tr>\n",
1193
+ " <tr>\n",
1194
+ " <th>3</th>\n",
1195
+ " <td>A Light in the Attic</td>\n",
1196
+ " <td>2024-11</td>\n",
1197
+ " <td>107</td>\n",
1198
+ " <td>neutral</td>\n",
1199
+ " </tr>\n",
1200
+ " <tr>\n",
1201
+ " <th>4</th>\n",
1202
+ " <td>A Light in the Attic</td>\n",
1203
+ " <td>2024-12</td>\n",
1204
+ " <td>108</td>\n",
1205
+ " <td>neutral</td>\n",
1206
+ " </tr>\n",
1207
+ " </tbody>\n",
1208
+ "</table>\n",
1209
+ "</div>\n",
1210
+ " <div class=\"colab-df-buttons\">\n",
1211
+ "\n",
1212
+ " <div class=\"colab-df-container\">\n",
1213
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-57178409-c5be-4481-bb4c-bc458aebbcad')\"\n",
1214
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1215
+ " style=\"display:none;\">\n",
1216
+ "\n",
1217
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1218
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1219
+ " </svg>\n",
1220
+ " </button>\n",
1221
+ "\n",
1222
+ " <style>\n",
1223
+ " .colab-df-container {\n",
1224
+ " display:flex;\n",
1225
+ " gap: 12px;\n",
1226
+ " }\n",
1227
+ "\n",
1228
+ " .colab-df-convert {\n",
1229
+ " background-color: #E8F0FE;\n",
1230
+ " border: none;\n",
1231
+ " border-radius: 50%;\n",
1232
+ " cursor: pointer;\n",
1233
+ " display: none;\n",
1234
+ " fill: #1967D2;\n",
1235
+ " height: 32px;\n",
1236
+ " padding: 0 0 0 0;\n",
1237
+ " width: 32px;\n",
1238
+ " }\n",
1239
+ "\n",
1240
+ " .colab-df-convert:hover {\n",
1241
+ " background-color: #E2EBFA;\n",
1242
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1243
+ " fill: #174EA6;\n",
1244
+ " }\n",
1245
+ "\n",
1246
+ " .colab-df-buttons div {\n",
1247
+ " margin-bottom: 4px;\n",
1248
+ " }\n",
1249
+ "\n",
1250
+ " [theme=dark] .colab-df-convert {\n",
1251
+ " background-color: #3B4455;\n",
1252
+ " fill: #D2E3FC;\n",
1253
+ " }\n",
1254
+ "\n",
1255
+ " [theme=dark] .colab-df-convert:hover {\n",
1256
+ " background-color: #434B5C;\n",
1257
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1258
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1259
+ " fill: #FFFFFF;\n",
1260
+ " }\n",
1261
+ " </style>\n",
1262
+ "\n",
1263
+ " <script>\n",
1264
+ " const buttonEl =\n",
1265
+ " document.querySelector('#df-57178409-c5be-4481-bb4c-bc458aebbcad button.colab-df-convert');\n",
1266
+ " buttonEl.style.display =\n",
1267
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1268
+ "\n",
1269
+ " async function convertToInteractive(key) {\n",
1270
+ " const element = document.querySelector('#df-57178409-c5be-4481-bb4c-bc458aebbcad');\n",
1271
+ " const dataTable =\n",
1272
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1273
+ " [key], {});\n",
1274
+ " if (!dataTable) return;\n",
1275
+ "\n",
1276
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1277
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1278
+ " + ' to learn more about interactive tables.';\n",
1279
+ " element.innerHTML = '';\n",
1280
+ " dataTable['output_type'] = 'display_data';\n",
1281
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1282
+ " const docLink = document.createElement('div');\n",
1283
+ " docLink.innerHTML = docLinkHtml;\n",
1284
+ " element.appendChild(docLink);\n",
1285
+ " }\n",
1286
+ " </script>\n",
1287
+ " </div>\n",
1288
+ "\n",
1289
+ "\n",
1290
+ " </div>\n",
1291
+ " </div>\n"
1292
+ ],
1293
+ "application/vnd.google.colaboratory.intrinsic+json": {
1294
+ "type": "dataframe",
1295
+ "variable_name": "df_sales",
1296
+ "summary": "{\n \"name\": \"df_sales\",\n \"rows\": 18000,\n \"fields\": [\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 999,\n \"samples\": [\n \"The Grownup\",\n \"Persepolis: The Story of a Childhood (Persepolis #1-2)\",\n \"Ayumi's Violin\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"month\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 18,\n \"samples\": [\n \"2024-08\",\n \"2024-09\",\n \"2025-04\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"units_sold\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 98,\n \"min\": 0,\n \"max\": 362,\n \"num_unique_values\": 354,\n \"samples\": [\n 214,\n 289,\n 205\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sentiment_label\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"neutral\",\n \"negative\",\n \"positive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1297
+ }
1298
+ },
1299
+ "metadata": {},
1300
+ "execution_count": 18
1301
+ }
1302
+ ],
1303
+ "source": [
1304
+ "df_sales = pd.DataFrame(sales_data)\n",
1305
+ "\n",
1306
+ "df_sales.head()\n"
1307
+ ]
1308
+ },
1309
+ {
1310
+ "cell_type": "markdown",
1311
+ "metadata": {
1312
+ "id": "EhIjz9WohAmZ"
1313
+ },
1314
+ "source": [
1315
+ "### *d. Save df_sales as synthetic_sales_data.csv & view first few lines*"
1316
+ ]
1317
+ },
1318
+ {
1319
+ "cell_type": "code",
1320
+ "execution_count": 19,
1321
+ "metadata": {
1322
+ "colab": {
1323
+ "base_uri": "https://localhost:8080/"
1324
+ },
1325
+ "id": "MzbZvLcAhGaH",
1326
+ "outputId": "e6c7b374-aa6a-4b3a-aa7c-e0613551de3a"
1327
+ },
1328
+ "outputs": [
1329
+ {
1330
+ "output_type": "stream",
1331
+ "name": "stdout",
1332
+ "text": [
1333
+ " title month units_sold sentiment_label\n",
1334
+ "0 A Light in the Attic 2024-08 100 neutral\n",
1335
+ "1 A Light in the Attic 2024-09 109 neutral\n",
1336
+ "2 A Light in the Attic 2024-10 102 neutral\n",
1337
+ "3 A Light in the Attic 2024-11 107 neutral\n",
1338
+ "4 A Light in the Attic 2024-12 108 neutral\n"
1339
+ ]
1340
+ }
1341
+ ],
1342
+ "source": [
1343
+ "df_sales.to_csv(\"synthetic_sales_data.csv\", index=False)\n",
1344
+ "\n",
1345
+ "print(df_sales.head())"
1346
+ ]
1347
+ },
1348
+ {
1349
+ "cell_type": "markdown",
1350
+ "metadata": {
1351
+ "id": "7g9gqBgQMtJn"
1352
+ },
1353
+ "source": [
1354
+ "## **5.** 🎯 Generate synthetic customer reviews"
1355
+ ]
1356
+ },
1357
+ {
1358
+ "cell_type": "markdown",
1359
+ "metadata": {
1360
+ "id": "Gi4y9M9KuDWx"
1361
+ },
1362
+ "source": [
1363
+ "### *a. ✋🏻🛑⛔️ Ask ChatGPT to create a list of 50 distinct generic book review texts for the sentiment labels \"positive\", \"neutral\", and \"negative\" called synthetic_reviews_by_sentiment*"
1364
+ ]
1365
+ },
1366
+ {
1367
+ "cell_type": "code",
1368
+ "execution_count": 22,
1369
+ "metadata": {
1370
+ "id": "b3cd2a50"
1371
+ },
1372
+ "outputs": [],
1373
+ "source": [
1374
+ "synthetic_reviews_by_sentiment = {\n",
1375
+ " \"positive\": [\n",
1376
+ " \"An engaging and beautifully written story.\",\n",
1377
+ " \"Absolutely loved the characters and plot development.\",\n",
1378
+ " \"A compelling read from start to finish.\",\n",
1379
+ " \"The storytelling was immersive and powerful.\",\n",
1380
+ " \"Highly recommended for fans of the genre.\",\n",
1381
+ " \"A masterpiece of modern fiction.\",\n",
1382
+ " \"The author’s writing style is captivating.\",\n",
1383
+ " \"An unforgettable and inspiring journey.\",\n",
1384
+ " \"Brilliant pacing and strong character arcs.\",\n",
1385
+ " \"Exceeded all my expectations.\",\n",
1386
+ " \"A heartwarming and uplifting book.\",\n",
1387
+ " \"Richly detailed and emotionally satisfying.\",\n",
1388
+ " \"One of the best books I’ve read this year.\",\n",
1389
+ " \"A truly enjoyable reading experience.\",\n",
1390
+ " \"The plot twists were clever and exciting.\",\n",
1391
+ " \"Beautiful prose and meaningful themes.\",\n",
1392
+ " \"A well-crafted and satisfying story.\",\n",
1393
+ " \"Fantastic world-building and creativity.\",\n",
1394
+ " \"A delightful and memorable novel.\",\n",
1395
+ " \"Gripping from the very first page.\",\n",
1396
+ " \"An exceptional and thought-provoking book.\",\n",
1397
+ " \"The characters felt real and relatable.\",\n",
1398
+ " \"A thoroughly entertaining read.\",\n",
1399
+ " \"Incredibly well executed storyline.\",\n",
1400
+ " \"A powerful and moving narrative.\",\n",
1401
+ " \"Loved every chapter of this book.\",\n",
1402
+ " \"A refreshing and original concept.\",\n",
1403
+ " \"Expertly written and emotionally rich.\",\n",
1404
+ " \"An inspiring and beautifully told tale.\",\n",
1405
+ " \"Top-notch writing and storytelling.\",\n",
1406
+ " \"A captivating literary experience.\",\n",
1407
+ " \"Deeply engaging and well-paced.\",\n",
1408
+ " \"An outstanding piece of work.\",\n",
1409
+ " \"The author did a fantastic job.\",\n",
1410
+ " \"A brilliant and immersive novel.\",\n",
1411
+ " \"Thoroughly enjoyed the journey.\",\n",
1412
+ " \"An imaginative and compelling story.\",\n",
1413
+ " \"A satisfying and rewarding read.\",\n",
1414
+ " \"Emotionally impactful and meaningful.\",\n",
1415
+ " \"A standout book in its category.\",\n",
1416
+ " \"Skillfully written and engaging.\",\n",
1417
+ " \"A truly remarkable novel.\",\n",
1418
+ " \"A page-turner I couldn’t put down.\",\n",
1419
+ " \"An excellent blend of drama and depth.\",\n",
1420
+ " \"Charming, thoughtful, and well-written.\",\n",
1421
+ " \"A gripping and emotional adventure.\",\n",
1422
+ " \"Creative, engaging, and beautifully crafted.\",\n",
1423
+ " \"A highly enjoyable literary escape.\",\n",
1424
+ " \"The narrative was both powerful and moving.\",\n",
1425
+ " \"A book I would gladly read again.\"\n",
1426
+ " ],\n",
1427
+ " \"neutral\": [\n",
1428
+ " \"An average read with some interesting moments.\",\n",
1429
+ " \"The book was okay but not memorable.\",\n",
1430
+ " \"Decent storyline but lacked depth.\",\n",
1431
+ " \"A fairly standard plot overall.\",\n",
1432
+ " \"Some parts were engaging, others less so.\",\n",
1433
+ " \"An acceptable but predictable read.\",\n",
1434
+ " \"The characters were moderately developed.\",\n",
1435
+ " \"It had potential but felt uneven.\",\n",
1436
+ " \"A simple and straightforward narrative.\",\n",
1437
+ " \"Not bad, but not particularly exciting.\",\n",
1438
+ " \"The pacing was somewhat inconsistent.\",\n",
1439
+ " \"An ordinary reading experience.\",\n",
1440
+ " \"The writing style was fine overall.\",\n",
1441
+ " \"A typical book in its genre.\",\n",
1442
+ " \"Some enjoyable sections throughout.\",\n",
1443
+ " \"It was neither great nor disappointing.\",\n",
1444
+ " \"The story was serviceable but plain.\",\n",
1445
+ " \"A mild and easy read.\",\n",
1446
+ " \"The themes were somewhat underexplored.\",\n",
1447
+ " \"An average execution of a good idea.\",\n",
1448
+ " \"Reasonably entertaining at times.\",\n",
1449
+ " \"The book had its moments.\",\n",
1450
+ " \"A passable and steady story.\",\n",
1451
+ " \"It met expectations but didn’t exceed them.\",\n",
1452
+ " \"Fairly engaging but not outstanding.\",\n",
1453
+ " \"The plot was clear but predictable.\",\n",
1454
+ " \"A competent but unremarkable novel.\",\n",
1455
+ " \"It was fine for a casual read.\",\n",
1456
+ " \"Moderately enjoyable throughout.\",\n",
1457
+ " \"The book was simply okay.\",\n",
1458
+ " \"An adequate story with simple characters.\",\n",
1459
+ " \"Not particularly impactful.\",\n",
1460
+ " \"Some parts felt rushed.\",\n",
1461
+ " \"The ending was satisfactory but plain.\",\n",
1462
+ " \"An average addition to the genre.\",\n",
1463
+ " \"A light and uncomplicated read.\",\n",
1464
+ " \"It held my attention occasionally.\",\n",
1465
+ " \"The writing was straightforward.\",\n",
1466
+ " \"Nothing particularly new or exciting.\",\n",
1467
+ " \"A decent but forgettable story.\",\n",
1468
+ " \"The narrative was somewhat flat.\",\n",
1469
+ " \"An okay book for passing time.\",\n",
1470
+ " \"Balanced but lacking strong highlights.\",\n",
1471
+ " \"The book was mildly engaging.\",\n",
1472
+ " \"Somewhat predictable but readable.\",\n",
1473
+ " \"An acceptable overall experience.\",\n",
1474
+ " \"It delivered what it promised.\",\n",
1475
+ " \"The characters were somewhat relatable.\",\n",
1476
+ " \"A standard reading experience.\",\n",
1477
+ " \"Neither impressive nor disappointing.\"\n",
1478
+ " ],\n",
1479
+ " \"negative\": [\n",
1480
+ " \"I struggled to stay interested in the story.\",\n",
1481
+ " \"The plot felt weak and unconvincing.\",\n",
1482
+ " \"Not an enjoyable reading experience.\",\n",
1483
+ " \"The characters lacked depth and realism.\",\n",
1484
+ " \"I found the pacing painfully slow.\",\n",
1485
+ " \"The writing style was difficult to follow.\",\n",
1486
+ " \"A disappointing and forgettable book.\",\n",
1487
+ " \"The story failed to engage me.\",\n",
1488
+ " \"Poorly developed characters throughout.\",\n",
1489
+ " \"The ending felt rushed and unsatisfying.\",\n",
1490
+ " \"The plot was confusing and inconsistent.\",\n",
1491
+ " \"I wouldn’t recommend this book.\",\n",
1492
+ " \"The narrative lacked excitement.\",\n",
1493
+ " \"It didn’t live up to its premise.\",\n",
1494
+ " \"The dialogue felt forced and unnatural.\",\n",
1495
+ " \"Overall, quite underwhelming.\",\n",
1496
+ " \"The story lacked originality.\",\n",
1497
+ " \"Difficult to connect with the characters.\",\n",
1498
+ " \"The book felt unnecessarily long.\",\n",
1499
+ " \"A frustrating reading experience.\",\n",
1500
+ " \"The themes were poorly executed.\",\n",
1501
+ " \"The writing felt flat and uninspired.\",\n",
1502
+ " \"I had trouble finishing it.\",\n",
1503
+ " \"The storyline was predictable and dull.\",\n",
1504
+ " \"The book lacked emotional impact.\",\n",
1505
+ " \"Not worth the time invested.\",\n",
1506
+ " \"The structure felt disorganized.\",\n",
1507
+ " \"The characters were forgettable.\",\n",
1508
+ " \"The narrative lacked clarity.\",\n",
1509
+ " \"A bland and uninspired novel.\",\n",
1510
+ " \"The book didn’t hold my attention.\",\n",
1511
+ " \"An unsatisfying and weak conclusion.\",\n",
1512
+ " \"The story felt repetitive.\",\n",
1513
+ " \"Very little character development.\",\n",
1514
+ " \"The plot holes were distracting.\",\n",
1515
+ " \"A disappointing effort overall.\",\n",
1516
+ " \"The pacing made it hard to enjoy.\",\n",
1517
+ " \"The book lacked focus and direction.\",\n",
1518
+ " \"Unremarkable and tedious to read.\",\n",
1519
+ " \"The writing felt overly simplistic.\",\n",
1520
+ " \"The story lacked depth and tension.\",\n",
1521
+ " \"I expected much more from this book.\",\n",
1522
+ " \"The narrative was hard to follow.\",\n",
1523
+ " \"It failed to create any excitement.\",\n",
1524
+ " \"The book felt poorly planned.\",\n",
1525
+ " \"An overall dull experience.\",\n",
1526
+ " \"The plot twists were unconvincing.\",\n",
1527
+ " \"The characters felt one-dimensional.\",\n",
1528
+ " \"A weak and forgettable story.\",\n",
1529
+ " \"Not a book I would revisit.\"\n",
1530
+ " ]\n",
1531
+ "}\n"
1532
+ ]
1533
+ },
1534
+ {
1535
+ "cell_type": "markdown",
1536
+ "metadata": {
1537
+ "id": "fQhfVaDmuULT"
1538
+ },
1539
+ "source": [
1540
+ "### *b. Generate 10 reviews per book using random sampling from the corresponding 50*"
1541
+ ]
1542
+ },
1543
+ {
1544
+ "cell_type": "code",
1545
+ "execution_count": 23,
1546
+ "metadata": {
1547
+ "id": "l2SRc3PjuTGM"
1548
+ },
1549
+ "outputs": [],
1550
+ "source": [
1551
+ "review_rows = []\n",
1552
+ "for _, row in df_books.iterrows():\n",
1553
+ " title = row['title']\n",
1554
+ " sentiment_label = row['sentiment_label']\n",
1555
+ " review_pool = synthetic_reviews_by_sentiment[sentiment_label]\n",
1556
+ " sampled_reviews = random.sample(review_pool, 10)\n",
1557
+ " for review_text in sampled_reviews:\n",
1558
+ " review_rows.append({\n",
1559
+ " \"title\": title,\n",
1560
+ " \"sentiment_label\": sentiment_label,\n",
1561
+ " \"review_text\": review_text,\n",
1562
+ " \"rating\": row['rating'],\n",
1563
+ " \"popularity_score\": row['popularity_score']\n",
1564
+ " })"
1565
+ ]
1566
+ },
1567
+ {
1568
+ "cell_type": "markdown",
1569
+ "metadata": {
1570
+ "id": "bmJMXF-Bukdm"
1571
+ },
1572
+ "source": [
1573
+ "### *c. Create the final dataframe df_reviews & save it as synthetic_book_reviews.csv*"
1574
+ ]
1575
+ },
1576
+ {
1577
+ "cell_type": "code",
1578
+ "execution_count": 24,
1579
+ "metadata": {
1580
+ "id": "ZUKUqZsuumsp"
1581
+ },
1582
+ "outputs": [],
1583
+ "source": [
1584
+ "df_reviews = pd.DataFrame(review_rows)\n",
1585
+ "df_reviews.to_csv(\"synthetic_book_reviews.csv\", index=False)"
1586
+ ]
1587
+ },
1588
+ {
1589
+ "cell_type": "markdown",
1590
+ "source": [
1591
+ "### *c. inputs for R*"
1592
+ ],
1593
+ "metadata": {
1594
+ "id": "_602pYUS3gY5"
1595
+ }
1596
+ },
1597
+ {
1598
+ "cell_type": "code",
1599
+ "execution_count": 25,
1600
+ "metadata": {
1601
+ "colab": {
1602
+ "base_uri": "https://localhost:8080/"
1603
+ },
1604
+ "id": "3946e521",
1605
+ "outputId": "cffe49c2-c845-49e4-e161-5848389f4e86"
1606
+ },
1607
+ "outputs": [
1608
+ {
1609
+ "output_type": "stream",
1610
+ "name": "stdout",
1611
+ "text": [
1612
+ "✅ Wrote synthetic_title_level_features.csv\n",
1613
+ "✅ Wrote synthetic_monthly_revenue_series.csv\n"
1614
+ ]
1615
+ }
1616
+ ],
1617
+ "source": [
1618
+ "import numpy as np\n",
1619
+ "\n",
1620
+ "def _safe_num(s):\n",
1621
+ " return pd.to_numeric(\n",
1622
+ " pd.Series(s).astype(str).str.replace(r\"[^0-9.]\", \"\", regex=True),\n",
1623
+ " errors=\"coerce\"\n",
1624
+ " )\n",
1625
+ "\n",
1626
+ "# --- Clean book metadata (price/rating) ---\n",
1627
+ "df_books_r = df_books.copy()\n",
1628
+ "if \"price\" in df_books_r.columns:\n",
1629
+ " df_books_r[\"price\"] = _safe_num(df_books_r[\"price\"])\n",
1630
+ "if \"rating\" in df_books_r.columns:\n",
1631
+ " df_books_r[\"rating\"] = _safe_num(df_books_r[\"rating\"])\n",
1632
+ "\n",
1633
+ "df_books_r[\"title\"] = df_books_r[\"title\"].astype(str).str.strip()\n",
1634
+ "\n",
1635
+ "# --- Clean sales ---\n",
1636
+ "df_sales_r = df_sales.copy()\n",
1637
+ "df_sales_r[\"title\"] = df_sales_r[\"title\"].astype(str).str.strip()\n",
1638
+ "df_sales_r[\"month\"] = pd.to_datetime(df_sales_r[\"month\"], errors=\"coerce\")\n",
1639
+ "df_sales_r[\"units_sold\"] = _safe_num(df_sales_r[\"units_sold\"])\n",
1640
+ "\n",
1641
+ "# --- Clean reviews ---\n",
1642
+ "df_reviews_r = df_reviews.copy()\n",
1643
+ "df_reviews_r[\"title\"] = df_reviews_r[\"title\"].astype(str).str.strip()\n",
1644
+ "df_reviews_r[\"sentiment_label\"] = df_reviews_r[\"sentiment_label\"].astype(str).str.lower().str.strip()\n",
1645
+ "if \"rating\" in df_reviews_r.columns:\n",
1646
+ " df_reviews_r[\"rating\"] = _safe_num(df_reviews_r[\"rating\"])\n",
1647
+ "if \"popularity_score\" in df_reviews_r.columns:\n",
1648
+ " df_reviews_r[\"popularity_score\"] = _safe_num(df_reviews_r[\"popularity_score\"])\n",
1649
+ "\n",
1650
+ "# --- Sentiment shares per title (from reviews) ---\n",
1651
+ "sent_counts = (\n",
1652
+ " df_reviews_r.groupby([\"title\", \"sentiment_label\"])\n",
1653
+ " .size()\n",
1654
+ " .unstack(fill_value=0)\n",
1655
+ ")\n",
1656
+ "for lab in [\"positive\", \"neutral\", \"negative\"]:\n",
1657
+ " if lab not in sent_counts.columns:\n",
1658
+ " sent_counts[lab] = 0\n",
1659
+ "\n",
1660
+ "sent_counts[\"total_reviews\"] = sent_counts[[\"positive\", \"neutral\", \"negative\"]].sum(axis=1)\n",
1661
+ "den = sent_counts[\"total_reviews\"].replace(0, np.nan)\n",
1662
+ "sent_counts[\"share_positive\"] = sent_counts[\"positive\"] / den\n",
1663
+ "sent_counts[\"share_neutral\"] = sent_counts[\"neutral\"] / den\n",
1664
+ "sent_counts[\"share_negative\"] = sent_counts[\"negative\"] / den\n",
1665
+ "sent_counts = sent_counts.reset_index()\n",
1666
+ "\n",
1667
+ "# --- Sales aggregation per title ---\n",
1668
+ "sales_by_title = (\n",
1669
+ " df_sales_r.dropna(subset=[\"title\"])\n",
1670
+ " .groupby(\"title\", as_index=False)\n",
1671
+ " .agg(\n",
1672
+ " months_observed=(\"month\", \"nunique\"),\n",
1673
+ " avg_units_sold=(\"units_sold\", \"mean\"),\n",
1674
+ " total_units_sold=(\"units_sold\", \"sum\"),\n",
1675
+ " )\n",
1676
+ ")\n",
1677
+ "\n",
1678
+ "# --- Title-level features (join sales + books + sentiment) ---\n",
1679
+ "df_title = (\n",
1680
+ " sales_by_title\n",
1681
+ " .merge(df_books_r[[\"title\", \"price\", \"rating\"]], on=\"title\", how=\"left\")\n",
1682
+ " .merge(sent_counts[[\"title\", \"share_positive\", \"share_neutral\", \"share_negative\", \"total_reviews\"]],\n",
1683
+ " on=\"title\", how=\"left\")\n",
1684
+ ")\n",
1685
+ "\n",
1686
+ "df_title[\"avg_revenue\"] = df_title[\"avg_units_sold\"] * df_title[\"price\"]\n",
1687
+ "df_title[\"total_revenue\"] = df_title[\"total_units_sold\"] * df_title[\"price\"]\n",
1688
+ "\n",
1689
+ "df_title.to_csv(\"synthetic_title_level_features.csv\", index=False)\n",
1690
+ "print(\"✅ Wrote synthetic_title_level_features.csv\")\n",
1691
+ "\n",
1692
+ "# --- Monthly revenue series (proxy: units_sold * price) ---\n",
1693
+ "monthly_rev = (\n",
1694
+ " df_sales_r.merge(df_books_r[[\"title\", \"price\"]], on=\"title\", how=\"left\")\n",
1695
+ ")\n",
1696
+ "monthly_rev[\"revenue\"] = monthly_rev[\"units_sold\"] * monthly_rev[\"price\"]\n",
1697
+ "\n",
1698
+ "df_monthly = (\n",
1699
+ " monthly_rev.dropna(subset=[\"month\"])\n",
1700
+ " .groupby(\"month\", as_index=False)[\"revenue\"]\n",
1701
+ " .sum()\n",
1702
+ " .rename(columns={\"revenue\": \"total_revenue\"})\n",
1703
+ " .sort_values(\"month\")\n",
1704
+ ")\n",
1705
+ "# if revenue is all NA (e.g., missing price), fallback to units_sold as a teaching proxy\n",
1706
+ "if df_monthly[\"total_revenue\"].notna().sum() == 0:\n",
1707
+ " df_monthly = (\n",
1708
+ " df_sales_r.dropna(subset=[\"month\"])\n",
1709
+ " .groupby(\"month\", as_index=False)[\"units_sold\"]\n",
1710
+ " .sum()\n",
1711
+ " .rename(columns={\"units_sold\": \"total_revenue\"})\n",
1712
+ " .sort_values(\"month\")\n",
1713
+ " )\n",
1714
+ "\n",
1715
+ "df_monthly[\"month\"] = pd.to_datetime(df_monthly[\"month\"], errors=\"coerce\").dt.strftime(\"%Y-%m-%d\")\n",
1716
+ "df_monthly.to_csv(\"synthetic_monthly_revenue_series.csv\", index=False)\n",
1717
+ "print(\"✅ Wrote synthetic_monthly_revenue_series.csv\")\n"
1718
+ ]
1719
+ },
1720
+ {
1721
+ "cell_type": "markdown",
1722
+ "metadata": {
1723
+ "id": "RYvGyVfXuo54"
1724
+ },
1725
+ "source": [
1726
+ "### *d. ✋🏻🛑⛔️ View the first few lines*"
1727
+ ]
1728
+ },
1729
+ {
1730
+ "cell_type": "code",
1731
+ "execution_count": 26,
1732
+ "metadata": {
1733
+ "colab": {
1734
+ "base_uri": "https://localhost:8080/",
1735
+ "height": 206
1736
+ },
1737
+ "id": "xfE8NMqOurKo",
1738
+ "outputId": "7f4270a2-1a39-40d6-c41b-543cd4263aab"
1739
+ },
1740
+ "outputs": [
1741
+ {
1742
+ "output_type": "execute_result",
1743
+ "data": {
1744
+ "text/plain": [
1745
+ " month total_revenue\n",
1746
+ "0 2024-08-01 5631956.77\n",
1747
+ "1 2024-09-01 5856653.68\n",
1748
+ "2 2024-10-01 6006876.26\n",
1749
+ "3 2024-11-01 6061519.85\n",
1750
+ "4 2024-12-01 6014276.79"
1751
+ ],
1752
+ "text/html": [
1753
+ "\n",
1754
+ " <div id=\"df-8ff3a272-440b-4f56-aab5-3cda1addfc34\" class=\"colab-df-container\">\n",
1755
+ " <div>\n",
1756
+ "<style scoped>\n",
1757
+ " .dataframe tbody tr th:only-of-type {\n",
1758
+ " vertical-align: middle;\n",
1759
+ " }\n",
1760
+ "\n",
1761
+ " .dataframe tbody tr th {\n",
1762
+ " vertical-align: top;\n",
1763
+ " }\n",
1764
+ "\n",
1765
+ " .dataframe thead th {\n",
1766
+ " text-align: right;\n",
1767
+ " }\n",
1768
+ "</style>\n",
1769
+ "<table border=\"1\" class=\"dataframe\">\n",
1770
+ " <thead>\n",
1771
+ " <tr style=\"text-align: right;\">\n",
1772
+ " <th></th>\n",
1773
+ " <th>month</th>\n",
1774
+ " <th>total_revenue</th>\n",
1775
+ " </tr>\n",
1776
+ " </thead>\n",
1777
+ " <tbody>\n",
1778
+ " <tr>\n",
1779
+ " <th>0</th>\n",
1780
+ " <td>2024-08-01</td>\n",
1781
+ " <td>5631956.77</td>\n",
1782
+ " </tr>\n",
1783
+ " <tr>\n",
1784
+ " <th>1</th>\n",
1785
+ " <td>2024-09-01</td>\n",
1786
+ " <td>5856653.68</td>\n",
1787
+ " </tr>\n",
1788
+ " <tr>\n",
1789
+ " <th>2</th>\n",
1790
+ " <td>2024-10-01</td>\n",
1791
+ " <td>6006876.26</td>\n",
1792
+ " </tr>\n",
1793
+ " <tr>\n",
1794
+ " <th>3</th>\n",
1795
+ " <td>2024-11-01</td>\n",
1796
+ " <td>6061519.85</td>\n",
1797
+ " </tr>\n",
1798
+ " <tr>\n",
1799
+ " <th>4</th>\n",
1800
+ " <td>2024-12-01</td>\n",
1801
+ " <td>6014276.79</td>\n",
1802
+ " </tr>\n",
1803
+ " </tbody>\n",
1804
+ "</table>\n",
1805
+ "</div>\n",
1806
+ " <div class=\"colab-df-buttons\">\n",
1807
+ "\n",
1808
+ " <div class=\"colab-df-container\">\n",
1809
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8ff3a272-440b-4f56-aab5-3cda1addfc34')\"\n",
1810
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1811
+ " style=\"display:none;\">\n",
1812
+ "\n",
1813
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1814
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1815
+ " </svg>\n",
1816
+ " </button>\n",
1817
+ "\n",
1818
+ " <style>\n",
1819
+ " .colab-df-container {\n",
1820
+ " display:flex;\n",
1821
+ " gap: 12px;\n",
1822
+ " }\n",
1823
+ "\n",
1824
+ " .colab-df-convert {\n",
1825
+ " background-color: #E8F0FE;\n",
1826
+ " border: none;\n",
1827
+ " border-radius: 50%;\n",
1828
+ " cursor: pointer;\n",
1829
+ " display: none;\n",
1830
+ " fill: #1967D2;\n",
1831
+ " height: 32px;\n",
1832
+ " padding: 0 0 0 0;\n",
1833
+ " width: 32px;\n",
1834
+ " }\n",
1835
+ "\n",
1836
+ " .colab-df-convert:hover {\n",
1837
+ " background-color: #E2EBFA;\n",
1838
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1839
+ " fill: #174EA6;\n",
1840
+ " }\n",
1841
+ "\n",
1842
+ " .colab-df-buttons div {\n",
1843
+ " margin-bottom: 4px;\n",
1844
+ " }\n",
1845
+ "\n",
1846
+ " [theme=dark] .colab-df-convert {\n",
1847
+ " background-color: #3B4455;\n",
1848
+ " fill: #D2E3FC;\n",
1849
+ " }\n",
1850
+ "\n",
1851
+ " [theme=dark] .colab-df-convert:hover {\n",
1852
+ " background-color: #434B5C;\n",
1853
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1854
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1855
+ " fill: #FFFFFF;\n",
1856
+ " }\n",
1857
+ " </style>\n",
1858
+ "\n",
1859
+ " <script>\n",
1860
+ " const buttonEl =\n",
1861
+ " document.querySelector('#df-8ff3a272-440b-4f56-aab5-3cda1addfc34 button.colab-df-convert');\n",
1862
+ " buttonEl.style.display =\n",
1863
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1864
+ "\n",
1865
+ " async function convertToInteractive(key) {\n",
1866
+ " const element = document.querySelector('#df-8ff3a272-440b-4f56-aab5-3cda1addfc34');\n",
1867
+ " const dataTable =\n",
1868
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1869
+ " [key], {});\n",
1870
+ " if (!dataTable) return;\n",
1871
+ "\n",
1872
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1873
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1874
+ " + ' to learn more about interactive tables.';\n",
1875
+ " element.innerHTML = '';\n",
1876
+ " dataTable['output_type'] = 'display_data';\n",
1877
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1878
+ " const docLink = document.createElement('div');\n",
1879
+ " docLink.innerHTML = docLinkHtml;\n",
1880
+ " element.appendChild(docLink);\n",
1881
+ " }\n",
1882
+ " </script>\n",
1883
+ " </div>\n",
1884
+ "\n",
1885
+ "\n",
1886
+ " </div>\n",
1887
+ " </div>\n"
1888
+ ],
1889
+ "application/vnd.google.colaboratory.intrinsic+json": {
1890
+ "type": "dataframe",
1891
+ "variable_name": "df_monthly",
1892
+ "summary": "{\n \"name\": \"df_monthly\",\n \"rows\": 18,\n \"fields\": [\n {\n \"column\": \"month\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 18,\n \"samples\": [\n \"2024-08-01\",\n \"2024-09-01\",\n \"2025-04-01\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"total_revenue\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 280014.47416254965,\n \"min\": 5523223.83,\n \"max\": 6376265.35,\n \"num_unique_values\": 18,\n \"samples\": [\n 5631956.77,\n 5856653.68,\n 5523223.83\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1893
+ }
1894
+ },
1895
+ "metadata": {},
1896
+ "execution_count": 26
1897
+ }
1898
+ ],
1899
+ "source": [
1900
+ "df_title.head()\n",
1901
+ "\n",
1902
+ "df_monthly.head()\n"
1903
+ ]
1904
+ }
1905
+ ],
1906
+ "metadata": {
1907
+ "colab": {
1908
+ "collapsed_sections": [
1909
+ "jpASMyIQMaAq",
1910
+ "lquNYCbfL9IM",
1911
+ "0IWuNpxxYDJF",
1912
+ "oCdTsin2Yfp3",
1913
+ "T0TOeRC4Yrnn",
1914
+ "duI5dv3CZYvF",
1915
+ "qMjRKMBQZlJi",
1916
+ "p-1Pr2szaqLk",
1917
+ "SIaJUGIpaH4V",
1918
+ "pY4yCoIuaQqp",
1919
+ "n4-TaNTFgPak",
1920
+ "HnngRNTgacYt",
1921
+ "HF9F9HIzgT7Z",
1922
+ "T8AdKkmASq9a",
1923
+ "OhXbdGD5fH0c",
1924
+ "L2ak1HlcgoTe",
1925
+ "4IXZKcCSgxnq",
1926
+ "EhIjz9WohAmZ",
1927
+ "Gi4y9M9KuDWx",
1928
+ "fQhfVaDmuULT",
1929
+ "bmJMXF-Bukdm",
1930
+ "RYvGyVfXuo54"
1931
+ ],
1932
+ "provenance": []
1933
+ },
1934
+ "kernelspec": {
1935
+ "display_name": "Python 3",
1936
+ "name": "python3"
1937
+ },
1938
+ "language_info": {
1939
+ "name": "python"
1940
+ }
1941
+ },
1942
+ "nbformat": 4,
1943
+ "nbformat_minor": 0
1944
+ }
pythonanalysis.ipynb ADDED
The diff for this file is too large to render. See raw diff
 
ranalysis.ipynb ADDED
@@ -0,0 +1,463 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "markdown",
5
+ "id": "75fd9cc6",
6
+ "metadata": {
7
+ "id": "75fd9cc6"
8
+ },
9
+ "source": [
10
+ "# **🤖 Benchmarking & Modeling**"
11
+ ]
12
+ },
13
+ {
14
+ "cell_type": "markdown",
15
+ "id": "fb807724",
16
+ "metadata": {
17
+ "id": "fb807724"
18
+ },
19
+ "source": [
20
+ "## **1.** 📦 Setup"
21
+ ]
22
+ },
23
+ {
24
+ "cell_type": "code",
25
+ "execution_count": null,
26
+ "id": "d40cd131",
27
+ "metadata": {
28
+ "id": "d40cd131"
29
+ },
30
+ "outputs": [],
31
+ "source": [
32
+ "\n",
33
+ "# Uncomment the next line once:\n",
34
+ "install.packages(c(\"readr\",\"dplyr\",\"stringr\",\"tidyr\",\"lubridate\",\"ggplot2\",\"forecast\",\"broom\",\"jsonlite\"), repos=\"https://cloud.r-project.org\")\n",
35
+ "\n",
36
+ "suppressPackageStartupMessages({\n",
37
+ " library(readr)\n",
38
+ " library(dplyr)\n",
39
+ " library(stringr)\n",
40
+ " library(tidyr)\n",
41
+ " library(lubridate)\n",
42
+ " library(ggplot2)\n",
43
+ " library(forecast)\n",
44
+ " library(broom)\n",
45
+ " library(jsonlite)\n",
46
+ "})"
47
+ ]
48
+ },
49
+ {
50
+ "cell_type": "markdown",
51
+ "id": "f01d02e7",
52
+ "metadata": {
53
+ "id": "f01d02e7"
54
+ },
55
+ "source": [
56
+ "## **2.** ✅️ Load & inspect inputs"
57
+ ]
58
+ },
59
+ {
60
+ "cell_type": "code",
61
+ "execution_count": null,
62
+ "id": "29e8f6ce",
63
+ "metadata": {
64
+ "colab": {
65
+ "base_uri": "https://localhost:8080/"
66
+ },
67
+ "id": "29e8f6ce",
68
+ "outputId": "5a1bda1c-c58d-43d0-c85e-db5041c8bc49"
69
+ },
70
+ "outputs": [
71
+ {
72
+ "output_type": "stream",
73
+ "name": "stdout",
74
+ "text": [
75
+ "Loaded: 1000 rows (title-level), 18 rows (monthly)\n"
76
+ ]
77
+ }
78
+ ],
79
+ "source": [
80
+ "\n",
81
+ "must_exist <- function(path, label) {\n",
82
+ " if (!file.exists(path)) stop(paste0(\"Missing \", label, \": \", path))\n",
83
+ "}\n",
84
+ "\n",
85
+ "TITLE_PATH <- \"synthetic_title_level_features.csv\"\n",
86
+ "MONTH_PATH <- \"synthetic_monthly_revenue_series.csv\"\n",
87
+ "\n",
88
+ "must_exist(TITLE_PATH, \"TITLE_PATH\")\n",
89
+ "must_exist(MONTH_PATH, \"MONTH_PATH\")\n",
90
+ "\n",
91
+ "df_title <- read_csv(TITLE_PATH, show_col_types = FALSE)\n",
92
+ "df_month <- read_csv(MONTH_PATH, show_col_types = FALSE)\n",
93
+ "\n",
94
+ "cat(\"Loaded:\", nrow(df_title), \"rows (title-level),\", nrow(df_month), \"rows (monthly)\n",
95
+ "\")"
96
+ ]
97
+ },
98
+ {
99
+ "cell_type": "code",
100
+ "execution_count": null,
101
+ "id": "9fd04262",
102
+ "metadata": {
103
+ "colab": {
104
+ "base_uri": "https://localhost:8080/"
105
+ },
106
+ "id": "9fd04262",
107
+ "outputId": "5f031538-96be-4758-904d-9201ec3c3ea7"
108
+ },
109
+ "outputs": [
110
+ {
111
+ "output_type": "stream",
112
+ "name": "stdout",
113
+ "text": [
114
+ "\u001b[90m# A tibble: 1 × 6\u001b[39m\n",
115
+ " n na_avg_revenue na_price na_rating na_share_pos na_share_neg\n",
116
+ " \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m \u001b[3m\u001b[90m<int>\u001b[39m\u001b[23m\n",
117
+ "\u001b[90m1\u001b[39m \u001b[4m1\u001b[24m000 0 0 \u001b[4m1\u001b[24m000 0 0\n",
118
+ "Monthly rows after parsing: 18 \n"
119
+ ]
120
+ }
121
+ ],
122
+ "source": [
123
+ "\n",
124
+ "# ---------- helpers ----------\n",
125
+ "safe_num <- function(x) {\n",
126
+ " # strips anything that is not digit or dot\n",
127
+ " suppressWarnings(as.numeric(str_replace_all(as.character(x), \"[^0-9.]\", \"\")))\n",
128
+ "}\n",
129
+ "\n",
130
+ "parse_rating <- function(x) {\n",
131
+ " # Accept: 4, \"4\", \"4.0\", \"4/5\", \"4 out of 5\", \"⭐⭐⭐⭐\", etc.\n",
132
+ " x <- as.character(x)\n",
133
+ " x <- str_replace_all(x, \"⭐\", \"\")\n",
134
+ " x <- str_to_lower(x)\n",
135
+ " x <- str_replace_all(x, \"stars?\", \"\")\n",
136
+ " x <- str_replace_all(x, \"out of\", \"/\")\n",
137
+ " x <- str_replace_all(x, \"\\\\s+\", \"\")\n",
138
+ " x <- str_replace_all(x, \"[^0-9./]\", \"\")\n",
139
+ " suppressWarnings(as.numeric(str_extract(x, \"^[0-9.]+\")))\n",
140
+ "}\n",
141
+ "\n",
142
+ "parse_month <- function(x) {\n",
143
+ " x <- as.character(x)\n",
144
+ " # try YYYY-MM-DD, then YYYY-MM\n",
145
+ " out <- suppressWarnings(ymd(x))\n",
146
+ " if (mean(is.na(out)) > 0.5) out <- suppressWarnings(ymd(paste0(x, \"-01\")))\n",
147
+ " na_idx <- which(is.na(out))\n",
148
+ " if (length(na_idx) > 0) out[na_idx] <- suppressWarnings(ymd(paste0(x[na_idx], \"-01\")))\n",
149
+ " out\n",
150
+ "}\n",
151
+ "\n",
152
+ "# ---------- normalize keys ----------\n",
153
+ "df_title <- df_title %>% mutate(title = str_squish(as.character(title)))\n",
154
+ "df_month <- df_month %>% mutate(month = as.character(month))\n",
155
+ "\n",
156
+ "# ---------- parse numeric columns defensively ----------\n",
157
+ "need_cols_title <- c(\"title\",\"avg_revenue\",\"total_revenue\",\"price\",\"rating\",\"share_positive\",\"share_negative\",\"share_neutral\")\n",
158
+ "missing_title <- setdiff(need_cols_title, names(df_title))\n",
159
+ "if (length(missing_title) > 0) stop(paste0(\"df_title missing columns: \", paste(missing_title, collapse=\", \")))\n",
160
+ "\n",
161
+ "df_title <- df_title %>%\n",
162
+ " mutate(\n",
163
+ " avg_revenue = safe_num(avg_revenue),\n",
164
+ " total_revenue = safe_num(total_revenue),\n",
165
+ " price = safe_num(price),\n",
166
+ " rating = parse_rating(rating),\n",
167
+ " share_positive = safe_num(share_positive),\n",
168
+ " share_negative = safe_num(share_negative),\n",
169
+ " share_neutral = safe_num(share_neutral)\n",
170
+ " )\n",
171
+ "\n",
172
+ "# basic sanity stats\n",
173
+ "hyg <- df_title %>%\n",
174
+ " summarise(\n",
175
+ " n = n(),\n",
176
+ " na_avg_revenue = sum(is.na(avg_revenue)),\n",
177
+ " na_price = sum(is.na(price)),\n",
178
+ " na_rating = sum(is.na(rating)),\n",
179
+ " na_share_pos = sum(is.na(share_positive)),\n",
180
+ " na_share_neg = sum(is.na(share_negative))\n",
181
+ " )\n",
182
+ "\n",
183
+ "print(hyg)\n",
184
+ "\n",
185
+ "# monthly parsing\n",
186
+ "need_cols_month <- c(\"month\",\"total_revenue\")\n",
187
+ "missing_month <- setdiff(need_cols_month, names(df_month))\n",
188
+ "if (length(missing_month) > 0) stop(paste0(\"df_month missing columns: \", paste(missing_month, collapse=\", \")))\n",
189
+ "\n",
190
+ "df_month2 <- df_month %>%\n",
191
+ " mutate(\n",
192
+ " month = parse_month(month),\n",
193
+ " total_revenue = safe_num(total_revenue)\n",
194
+ " ) %>%\n",
195
+ " filter(!is.na(month)) %>%\n",
196
+ " arrange(month)\n",
197
+ "\n",
198
+ "cat(\"Monthly rows after parsing:\", nrow(df_month2), \"\\n\")"
199
+ ]
200
+ },
201
+ {
202
+ "cell_type": "markdown",
203
+ "id": "b8971bc4",
204
+ "metadata": {
205
+ "id": "b8971bc4"
206
+ },
207
+ "source": [
208
+ "## **3.** 💾 Folder for R outputs for Hugging Face"
209
+ ]
210
+ },
211
+ {
212
+ "cell_type": "code",
213
+ "execution_count": null,
214
+ "id": "dfaa06b1",
215
+ "metadata": {
216
+ "colab": {
217
+ "base_uri": "https://localhost:8080/"
218
+ },
219
+ "id": "dfaa06b1",
220
+ "outputId": "73f6437a-39f4-4968-f88a-99f10a3fd8ae"
221
+ },
222
+ "outputs": [
223
+ {
224
+ "output_type": "stream",
225
+ "name": "stdout",
226
+ "text": [
227
+ "R outputs will be written to: /content/artifacts/r \n"
228
+ ]
229
+ }
230
+ ],
231
+ "source": [
232
+ "\n",
233
+ "ART_DIR <- \"artifacts\"\n",
234
+ "R_FIG_DIR <- file.path(ART_DIR, \"r\", \"figures\")\n",
235
+ "R_TAB_DIR <- file.path(ART_DIR, \"r\", \"tables\")\n",
236
+ "\n",
237
+ "dir.create(R_FIG_DIR, recursive = TRUE, showWarnings = FALSE)\n",
238
+ "dir.create(R_TAB_DIR, recursive = TRUE, showWarnings = FALSE)\n",
239
+ "\n",
240
+ "cat(\"R outputs will be written to:\", normalizePath(file.path(ART_DIR, \"r\"), winslash = \"/\"), \"\n",
241
+ "\")"
242
+ ]
243
+ },
244
+ {
245
+ "cell_type": "markdown",
246
+ "id": "f880c72d",
247
+ "metadata": {
248
+ "id": "f880c72d"
249
+ },
250
+ "source": [
251
+ "## **4.** 🔮 Forecast book sales benchmarking with `accuracy()`"
252
+ ]
253
+ },
254
+ {
255
+ "cell_type": "markdown",
256
+ "source": [
257
+ "We benchmark **three** models on a holdout window (last *h* months):\n",
258
+ "- ARIMA + Fourier (seasonality upgrade)\n",
259
+ "- ETS\n",
260
+ "- Naive baseline\n",
261
+ "\n",
262
+ "Then we export:\n",
263
+ "- `accuracy_table.csv`\n",
264
+ "- `forecast_compare.png`\n",
265
+ "- `rmse_comparison.png`"
266
+ ],
267
+ "metadata": {
268
+ "id": "R0JZlzKegmzW"
269
+ },
270
+ "id": "R0JZlzKegmzW"
271
+ },
272
+ {
273
+ "cell_type": "code",
274
+ "execution_count": null,
275
+ "id": "62e87992",
276
+ "metadata": {
277
+ "colab": {
278
+ "base_uri": "https://localhost:8080/"
279
+ },
280
+ "id": "62e87992",
281
+ "outputId": "73b36487-a25d-4bb9-cf80-8d5a654a2f0d"
282
+ },
283
+ "outputs": [
284
+ {
285
+ "output_type": "stream",
286
+ "name": "stdout",
287
+ "text": [
288
+ "✅ Saved: artifacts/r/tables/accuracy_table.csv\n",
289
+ "✅ Saved: artifacts/r/figures/rmse_comparison.png\n"
290
+ ]
291
+ },
292
+ {
293
+ "output_type": "display_data",
294
+ "data": {
295
+ "text/html": [
296
+ "<strong>agg_record_872216040:</strong> 2"
297
+ ],
298
+ "text/markdown": "**agg_record_872216040:** 2",
299
+ "text/latex": "\\textbf{agg\\textbackslash{}\\_record\\textbackslash{}\\_872216040:} 2",
300
+ "text/plain": [
301
+ "agg_record_872216040 \n",
302
+ " 2 "
303
+ ]
304
+ },
305
+ "metadata": {}
306
+ },
307
+ {
308
+ "output_type": "stream",
309
+ "name": "stdout",
310
+ "text": [
311
+ "✅ Saved: artifacts/r/figures/forecast_compare.png\n"
312
+ ]
313
+ }
314
+ ],
315
+ "source": [
316
+ "\n",
317
+ "# Build monthly ts\n",
318
+ "start_year <- year(min(df_month2$month, na.rm = TRUE))\n",
319
+ "start_mon <- month(min(df_month2$month, na.rm = TRUE))\n",
320
+ "\n",
321
+ "y <- ts(df_month2$total_revenue, frequency = 12, start = c(start_year, start_mon))\n",
322
+ "\n",
323
+ "# holdout size: min(6, 20% of series), at least 1\n",
324
+ "h_test <- min(6, max(1, floor(length(y) / 5)))\n",
325
+ "train_ts <- head(y, length(y) - h_test)\n",
326
+ "test_ts <- tail(y, h_test)\n",
327
+ "\n",
328
+ "# Model A: ARIMA + Fourier\n",
329
+ "K <- 2\n",
330
+ "xreg_train <- fourier(train_ts, K = K)\n",
331
+ "fit_arima <- auto.arima(train_ts, xreg = xreg_train)\n",
332
+ "xreg_future <- fourier(train_ts, K = K, h = h_test)\n",
333
+ "fc_arima <- forecast(fit_arima, xreg = xreg_future, h = h_test)\n",
334
+ "\n",
335
+ "# Model B: ETS\n",
336
+ "fit_ets <- ets(train_ts)\n",
337
+ "fc_ets <- forecast(fit_ets, h = h_test)\n",
338
+ "\n",
339
+ "# Model C: Naive baseline\n",
340
+ "fc_naive <- naive(train_ts, h = h_test)\n",
341
+ "\n",
342
+ "# accuracy() tables\n",
343
+ "acc_arima <- as.data.frame(accuracy(fc_arima, test_ts))\n",
344
+ "acc_ets <- as.data.frame(accuracy(fc_ets, test_ts))\n",
345
+ "acc_naive <- as.data.frame(accuracy(fc_naive, test_ts))\n",
346
+ "\n",
347
+ "accuracy_tbl <- bind_rows(\n",
348
+ " acc_arima %>% mutate(model = \"ARIMA+Fourier\"),\n",
349
+ " acc_ets %>% mutate(model = \"ETS\"),\n",
350
+ " acc_naive %>% mutate(model = \"Naive\")\n",
351
+ ") %>% relocate(model)\n",
352
+ "\n",
353
+ "write_csv(accuracy_tbl, file.path(R_TAB_DIR, \"accuracy_table.csv\"))\n",
354
+ "cat(\"✅ Saved: artifacts/r/tables/accuracy_table.csv\\n\")\n",
355
+ "\n",
356
+ "# RMSE bar chart\n",
357
+ "p_rmse <- ggplot(accuracy_tbl, aes(x = reorder(model, RMSE), y = RMSE)) +\n",
358
+ " geom_col() +\n",
359
+ " coord_flip() +\n",
360
+ " labs(title = \"Forecast model comparison (RMSE on holdout)\", x = \"\", y = \"RMSE\") +\n",
361
+ " theme_minimal()\n",
362
+ "\n",
363
+ "ggsave(file.path(R_FIG_DIR, \"rmse_comparison.png\"), p_rmse, width = 8, height = 4, dpi = 160)\n",
364
+ "cat(\"✅ Saved: artifacts/r/figures/rmse_comparison.png\\n\")\n",
365
+ "\n",
366
+ "# Side-by-side forecast plots (simple, no extra deps)\n",
367
+ "png(file.path(R_FIG_DIR, \"forecast_compare.png\"), width = 1200, height = 500)\n",
368
+ "par(mfrow = c(1, 3))\n",
369
+ "plot(fc_arima, main = \"ARIMA + Fourier\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
370
+ "plot(fc_ets, main = \"ETS\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
371
+ "plot(fc_naive, main = \"Naive\", xlab = \"Time\", ylab = \"Total revenue\"); lines(test_ts, col = \"black\")\n",
372
+ "dev.off()\n",
373
+ "cat(\"✅ Saved: artifacts/r/figures/forecast_compare.png\\n\")"
374
+ ]
375
+ },
376
+ {
377
+ "cell_type": "markdown",
378
+ "id": "30bc017b",
379
+ "metadata": {
380
+ "id": "30bc017b"
381
+ },
382
+ "source": [
383
+ "## **5.** 💾 Some R metadata for Hugging Face"
384
+ ]
385
+ },
386
+ {
387
+ "cell_type": "code",
388
+ "execution_count": null,
389
+ "id": "645cb12b",
390
+ "metadata": {
391
+ "colab": {
392
+ "base_uri": "https://localhost:8080/"
393
+ },
394
+ "id": "645cb12b",
395
+ "outputId": "c00c26da-7d27-4c78-a296-aa33807495d4"
396
+ },
397
+ "outputs": [
398
+ {
399
+ "output_type": "stream",
400
+ "name": "stdout",
401
+ "text": [
402
+ "✅ Saved: artifacts/r/tables/r_meta.json\n",
403
+ "DONE. R artifacts written to: artifacts/r \n"
404
+ ]
405
+ }
406
+ ],
407
+ "source": [
408
+ "# =========================================================\n",
409
+ "# Metadata export (aligned with current notebook objects)\n",
410
+ "# =========================================================\n",
411
+ "\n",
412
+ "meta <- list(\n",
413
+ "\n",
414
+ " # ---------------------------\n",
415
+ " # Dataset footprint\n",
416
+ " # ---------------------------\n",
417
+ " n_titles = nrow(df_title),\n",
418
+ " n_months = nrow(df_month2),\n",
419
+ "\n",
420
+ " # ---------------------------\n",
421
+ " # Forecasting info\n",
422
+ " # (only if these objects exist in your forecasting section)\n",
423
+ " # ---------------------------\n",
424
+ " forecasting = list(\n",
425
+ " holdout_h = h_test,\n",
426
+ " arima_order = forecast::arimaorder(fit_arima),\n",
427
+ " ets_method = fit_ets$method\n",
428
+ " )\n",
429
+ ")\n",
430
+ "\n",
431
+ "jsonlite::write_json(\n",
432
+ " meta,\n",
433
+ " path = file.path(R_TAB_DIR, \"r_meta.json\"),\n",
434
+ " pretty = TRUE,\n",
435
+ " auto_unbox = TRUE\n",
436
+ ")\n",
437
+ "\n",
438
+ "cat(\"✅ Saved: artifacts/r/tables/r_meta.json\\n\")\n",
439
+ "cat(\"DONE. R artifacts written to:\", file.path(ART_DIR, \"r\"), \"\\n\")\n"
440
+ ]
441
+ }
442
+ ],
443
+ "metadata": {
444
+ "colab": {
445
+ "provenance": [],
446
+ "collapsed_sections": [
447
+ "f01d02e7",
448
+ "b8971bc4",
449
+ "f880c72d",
450
+ "30bc017b"
451
+ ]
452
+ },
453
+ "kernelspec": {
454
+ "name": "ir",
455
+ "display_name": "R"
456
+ },
457
+ "language_info": {
458
+ "name": "R"
459
+ }
460
+ },
461
+ "nbformat": 4,
462
+ "nbformat_minor": 5
463
+ }