Hugging Face's logo

recommender-system
/
learning-models

Model card Files
xet
Community
Selleri Development commited on
Commit
e6fd1ec
·
1 Parent(s): 8527ae6

Add Dataset Code

Browse files
Files changed (1) hide show
  1. 5. Create Datasets.ipynb +1447 -0
5. Create Datasets.ipynb ADDED
@@ -0,0 +1,1447 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": []
7
+ },
8
+ "kernelspec": {
9
+ "name": "python3",
10
+ "display_name": "Python 3"
11
+ },
12
+ "language_info": {
13
+ "name": "python"
14
+ }
15
+ },
16
+ "cells": [
17
+ {
18
+ "cell_type": "code",
19
+ "source": [
20
+ "# Mengunduh dataset MovieLens 100k\n",
21
+ "!wget -q https://files.grouplens.org/datasets/movielens/ml-100k.zip\n",
22
+ "!unzip -q ml-100k.zip\n",
23
+ "\n",
24
+ "# Mengunduh dataset MovieLens 1M\n",
25
+ "!wget -q https://files.grouplens.org/datasets/movielens/ml-1m.zip\n",
26
+ "!unzip -q ml-1m.zip\n",
27
+ "\n",
28
+ "# Mengunduh dataset MovieLens Metadata\n",
29
+ "!unzip -q movies_metadata.zip"
30
+ ],
31
+ "metadata": {
32
+ "colab": {
33
+ "base_uri": "https://localhost:8080/"
34
+ },
35
+ "id": "kqom8x_fb61t",
36
+ "outputId": "cccfb8ce-aada-4a9c-e03d-f3e05258dab9"
37
+ },
38
+ "execution_count": 32,
39
+ "outputs": [
40
+ {
41
+ "output_type": "stream",
42
+ "name": "stdout",
43
+ "text": [
44
+ "replace ml-100k/allbut.pl? [y]es, [n]o, [A]ll, [N]one, [r]ename: A\n",
45
+ "replace ml-1m/movies.dat? [y]es, [n]o, [A]ll, [N]one, [r]ename: A\n"
46
+ ]
47
+ }
48
+ ]
49
+ },
50
+ {
51
+ "cell_type": "markdown",
52
+ "source": [
53
+ "## Load Dataset Movielens\n",
54
+ "Dataset ini harus terdiri dari tiga file master yaitu\n",
55
+ "1. Users yang berisikan user_id, gender, age, occupation, zip_code\n",
56
+ "2. Movies yang berisikan movie_id, title, genres, is_adult, original_language, original_title, overview, popularity, release_date, revenue, runtime, vote_average, dan vote_count.\n",
57
+ "3. Ratings yang berisikan user_id, movie_id, rating, dan timestamp"
58
+ ],
59
+ "metadata": {
60
+ "id": "GWFqG_HXbvQI"
61
+ }
62
+ },
63
+ {
64
+ "cell_type": "code",
65
+ "source": [
66
+ "import pandas as pd\n",
67
+ "import numpy as np\n",
68
+ "from sklearn.model_selection import train_test_split"
69
+ ],
70
+ "metadata": {
71
+ "id": "qI07ntK6dAmy"
72
+ },
73
+ "execution_count": 177,
74
+ "outputs": []
75
+ },
76
+ {
77
+ "cell_type": "code",
78
+ "source": [
79
+ "# Memuat data\n",
80
+ "ratings = pd.read_csv('ml-100k/u.data', sep='\\t', names=['user_id', 'movie_id', 'rating', 'timestamp'])\n",
81
+ "users = pd.read_csv('ml-100k/u.user', sep='|', names=['user_id', 'gender', 'age', 'occupation', 'zip_code'])\n",
82
+ "movies = pd.read_csv('ml-100k/u.item', sep='|', encoding='ISO-8859-1', header=None, names=['movie_id', 'title', 'release_date', 'imdb_url'], usecols=[0,1,2,4])"
83
+ ],
84
+ "metadata": {
85
+ "id": "p_Al0TLpcuYN"
86
+ },
87
+ "execution_count": 178,
88
+ "outputs": []
89
+ },
90
+ {
91
+ "cell_type": "code",
92
+ "source": [
93
+ "# Memuat data\n",
94
+ "ml1_movies = pd.read_csv('ml-1m/movies.dat', sep='::', encoding='ISO-8859-1', header=None, names=['movie_id', 'title', 'genres'], usecols=[0,1,2])\n",
95
+ "ml1_movies.head(1)"
96
+ ],
97
+ "metadata": {
98
+ "colab": {
99
+ "base_uri": "https://localhost:8080/",
100
+ "height": 0
101
+ },
102
+ "id": "-Umv6xZFjK_H",
103
+ "outputId": "3e789a07-5b10-4026-d26d-fce92496dba3"
104
+ },
105
+ "execution_count": 179,
106
+ "outputs": [
107
+ {
108
+ "output_type": "stream",
109
+ "name": "stderr",
110
+ "text": [
111
+ "<ipython-input-179-e71d00712615>:2: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.\n",
112
+ " ml1_movies = pd.read_csv('ml-1m/movies.dat', sep='::', encoding='ISO-8859-1', header=None, names=['movie_id', 'title', 'genres'], usecols=[0,1,2])\n"
113
+ ]
114
+ },
115
+ {
116
+ "output_type": "execute_result",
117
+ "data": {
118
+ "text/plain": [
119
+ " movie_id title genres\n",
120
+ "0 1 Toy Story (1995) Animation|Children's|Comedy"
121
+ ],
122
+ "text/html": [
123
+ "\n",
124
+ " <div id=\"df-0121428d-a55a-4066-834d-e387ad094c88\" class=\"colab-df-container\">\n",
125
+ " <div>\n",
126
+ "<style scoped>\n",
127
+ " .dataframe tbody tr th:only-of-type {\n",
128
+ " vertical-align: middle;\n",
129
+ " }\n",
130
+ "\n",
131
+ " .dataframe tbody tr th {\n",
132
+ " vertical-align: top;\n",
133
+ " }\n",
134
+ "\n",
135
+ " .dataframe thead th {\n",
136
+ " text-align: right;\n",
137
+ " }\n",
138
+ "</style>\n",
139
+ "<table border=\"1\" class=\"dataframe\">\n",
140
+ " <thead>\n",
141
+ " <tr style=\"text-align: right;\">\n",
142
+ " <th></th>\n",
143
+ " <th>movie_id</th>\n",
144
+ " <th>title</th>\n",
145
+ " <th>genres</th>\n",
146
+ " </tr>\n",
147
+ " </thead>\n",
148
+ " <tbody>\n",
149
+ " <tr>\n",
150
+ " <th>0</th>\n",
151
+ " <td>1</td>\n",
152
+ " <td>Toy Story (1995)</td>\n",
153
+ " <td>Animation|Children's|Comedy</td>\n",
154
+ " </tr>\n",
155
+ " </tbody>\n",
156
+ "</table>\n",
157
+ "</div>\n",
158
+ " <div class=\"colab-df-buttons\">\n",
159
+ "\n",
160
+ " <div class=\"colab-df-container\">\n",
161
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0121428d-a55a-4066-834d-e387ad094c88')\"\n",
162
+ " title=\"Convert this dataframe to an interactive table.\"\n",
163
+ " style=\"display:none;\">\n",
164
+ "\n",
165
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
166
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
167
+ " </svg>\n",
168
+ " </button>\n",
169
+ "\n",
170
+ " <style>\n",
171
+ " .colab-df-container {\n",
172
+ " display:flex;\n",
173
+ " gap: 12px;\n",
174
+ " }\n",
175
+ "\n",
176
+ " .colab-df-convert {\n",
177
+ " background-color: #E8F0FE;\n",
178
+ " border: none;\n",
179
+ " border-radius: 50%;\n",
180
+ " cursor: pointer;\n",
181
+ " display: none;\n",
182
+ " fill: #1967D2;\n",
183
+ " height: 32px;\n",
184
+ " padding: 0 0 0 0;\n",
185
+ " width: 32px;\n",
186
+ " }\n",
187
+ "\n",
188
+ " .colab-df-convert:hover {\n",
189
+ " background-color: #E2EBFA;\n",
190
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
191
+ " fill: #174EA6;\n",
192
+ " }\n",
193
+ "\n",
194
+ " .colab-df-buttons div {\n",
195
+ " margin-bottom: 4px;\n",
196
+ " }\n",
197
+ "\n",
198
+ " [theme=dark] .colab-df-convert {\n",
199
+ " background-color: #3B4455;\n",
200
+ " fill: #D2E3FC;\n",
201
+ " }\n",
202
+ "\n",
203
+ " [theme=dark] .colab-df-convert:hover {\n",
204
+ " background-color: #434B5C;\n",
205
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
206
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
207
+ " fill: #FFFFFF;\n",
208
+ " }\n",
209
+ " </style>\n",
210
+ "\n",
211
+ " <script>\n",
212
+ " const buttonEl =\n",
213
+ " document.querySelector('#df-0121428d-a55a-4066-834d-e387ad094c88 button.colab-df-convert');\n",
214
+ " buttonEl.style.display =\n",
215
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
216
+ "\n",
217
+ " async function convertToInteractive(key) {\n",
218
+ " const element = document.querySelector('#df-0121428d-a55a-4066-834d-e387ad094c88');\n",
219
+ " const dataTable =\n",
220
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
221
+ " [key], {});\n",
222
+ " if (!dataTable) return;\n",
223
+ "\n",
224
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
225
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
226
+ " + ' to learn more about interactive tables.';\n",
227
+ " element.innerHTML = '';\n",
228
+ " dataTable['output_type'] = 'display_data';\n",
229
+ " await google.colab.output.renderOutput(dataTable, element);\n",
230
+ " const docLink = document.createElement('div');\n",
231
+ " docLink.innerHTML = docLinkHtml;\n",
232
+ " element.appendChild(docLink);\n",
233
+ " }\n",
234
+ " </script>\n",
235
+ " </div>\n",
236
+ "\n",
237
+ "\n",
238
+ " </div>\n",
239
+ " </div>\n"
240
+ ],
241
+ "application/vnd.google.colaboratory.intrinsic+json": {
242
+ "type": "dataframe",
243
+ "variable_name": "ml1_movies",
244
+ "summary": "{\n \"name\": \"ml1_movies\",\n \"rows\": 3883,\n \"fields\": [\n {\n \"column\": \"movie_id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1146,\n \"min\": 1,\n \"max\": 3952,\n \"num_unique_values\": 3883,\n \"samples\": [\n 1365,\n 2706,\n 3667\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 3883,\n \"samples\": [\n \"Ridicule (1996)\",\n \"American Pie (1999)\",\n \"Rent-A-Cop (1988)\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 301,\n \"samples\": [\n \"Action|Adventure|Comedy|Horror\",\n \"Romance|Western\",\n \"Action|Adventure|Children's|Comedy\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
245
+ }
246
+ },
247
+ "metadata": {},
248
+ "execution_count": 179
249
+ }
250
+ ]
251
+ },
252
+ {
253
+ "cell_type": "code",
254
+ "source": [
255
+ "# Menggabungkan kolom 'genres' dari ml1_movies ke movies berdasarkan 'movie_id'\n",
256
+ "movies = movies.merge(ml1_movies[['movie_id', 'genres']], on='movie_id', how='left')\n",
257
+ "# Extract year from title\n",
258
+ "movies[[\"title\", \"year\"]] = movies[\"title\"].str.extract('(.*)\\((\\d+)\\)')\n",
259
+ "# Remove trailing whitespace from title\n",
260
+ "movies[\"title\"] = movies[\"title\"].str.strip()\n",
261
+ "ml1_movies = ml1_movies.iloc[0:0]"
262
+ ],
263
+ "metadata": {
264
+ "id": "SIMv7RJdlV84"
265
+ },
266
+ "execution_count": 180,
267
+ "outputs": []
268
+ },
269
+ {
270
+ "cell_type": "code",
271
+ "source": [
272
+ "ml_meta_movies = pd.read_csv('movies_metadata.csv', low_memory=False)\n",
273
+ "ml_meta_movies.head(1)"
274
+ ],
275
+ "metadata": {
276
+ "colab": {
277
+ "base_uri": "https://localhost:8080/",
278
+ "height": 0
279
+ },
280
+ "id": "zf-ApwFIldKw",
281
+ "outputId": "24de81f0-6af8-4b9a-c264-91d55ecc47ef"
282
+ },
283
+ "execution_count": 181,
284
+ "outputs": [
285
+ {
286
+ "output_type": "execute_result",
287
+ "data": {
288
+ "text/plain": [
289
+ " adult belongs_to_collection budget \\\n",
290
+ "0 False {'id': 10194, 'name': 'Toy Story Collection', ... 30000000 \n",
291
+ "\n",
292
+ " genres \\\n",
293
+ "0 [{'id': 16, 'name': 'Animation'}, {'id': 35, '... \n",
294
+ "\n",
295
+ " homepage id imdb_id original_language \\\n",
296
+ "0 http://toystory.disney.com/toy-story 862 tt0114709 en \n",
297
+ "\n",
298
+ " original_title overview ... \\\n",
299
+ "0 Toy Story Led by Woody, Andy's toys live happily in his ... ... \n",
300
+ "\n",
301
+ " release_date revenue runtime spoken_languages \\\n",
302
+ "0 1995-10-30 373554033.0 81.0 [{'iso_639_1': 'en', 'name': 'English'}] \n",
303
+ "\n",
304
+ " status tagline title video vote_average vote_count \n",
305
+ "0 Released NaN Toy Story False 7.7 5415.0 \n",
306
+ "\n",
307
+ "[1 rows x 24 columns]"
308
+ ],
309
+ "text/html": [
310
+ "\n",
311
+ " <div id=\"df-d8591515-3aef-458e-9707-9fb81eb55634\" class=\"colab-df-container\">\n",
312
+ " <div>\n",
313
+ "<style scoped>\n",
314
+ " .dataframe tbody tr th:only-of-type {\n",
315
+ " vertical-align: middle;\n",
316
+ " }\n",
317
+ "\n",
318
+ " .dataframe tbody tr th {\n",
319
+ " vertical-align: top;\n",
320
+ " }\n",
321
+ "\n",
322
+ " .dataframe thead th {\n",
323
+ " text-align: right;\n",
324
+ " }\n",
325
+ "</style>\n",
326
+ "<table border=\"1\" class=\"dataframe\">\n",
327
+ " <thead>\n",
328
+ " <tr style=\"text-align: right;\">\n",
329
+ " <th></th>\n",
330
+ " <th>adult</th>\n",
331
+ " <th>belongs_to_collection</th>\n",
332
+ " <th>budget</th>\n",
333
+ " <th>genres</th>\n",
334
+ " <th>homepage</th>\n",
335
+ " <th>id</th>\n",
336
+ " <th>imdb_id</th>\n",
337
+ " <th>original_language</th>\n",
338
+ " <th>original_title</th>\n",
339
+ " <th>overview</th>\n",
340
+ " <th>...</th>\n",
341
+ " <th>release_date</th>\n",
342
+ " <th>revenue</th>\n",
343
+ " <th>runtime</th>\n",
344
+ " <th>spoken_languages</th>\n",
345
+ " <th>status</th>\n",
346
+ " <th>tagline</th>\n",
347
+ " <th>title</th>\n",
348
+ " <th>video</th>\n",
349
+ " <th>vote_average</th>\n",
350
+ " <th>vote_count</th>\n",
351
+ " </tr>\n",
352
+ " </thead>\n",
353
+ " <tbody>\n",
354
+ " <tr>\n",
355
+ " <th>0</th>\n",
356
+ " <td>False</td>\n",
357
+ " <td>{'id': 10194, 'name': 'Toy Story Collection', ...</td>\n",
358
+ " <td>30000000</td>\n",
359
+ " <td>[{'id': 16, 'name': 'Animation'}, {'id': 35, '...</td>\n",
360
+ " <td>http://toystory.disney.com/toy-story</td>\n",
361
+ " <td>862</td>\n",
362
+ " <td>tt0114709</td>\n",
363
+ " <td>en</td>\n",
364
+ " <td>Toy Story</td>\n",
365
+ " <td>Led by Woody, Andy's toys live happily in his ...</td>\n",
366
+ " <td>...</td>\n",
367
+ " <td>1995-10-30</td>\n",
368
+ " <td>373554033.0</td>\n",
369
+ " <td>81.0</td>\n",
370
+ " <td>[{'iso_639_1': 'en', 'name': 'English'}]</td>\n",
371
+ " <td>Released</td>\n",
372
+ " <td>NaN</td>\n",
373
+ " <td>Toy Story</td>\n",
374
+ " <td>False</td>\n",
375
+ " <td>7.7</td>\n",
376
+ " <td>5415.0</td>\n",
377
+ " </tr>\n",
378
+ " </tbody>\n",
379
+ "</table>\n",
380
+ "<p>1 rows × 24 columns</p>\n",
381
+ "</div>\n",
382
+ " <div class=\"colab-df-buttons\">\n",
383
+ "\n",
384
+ " <div class=\"colab-df-container\">\n",
385
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d8591515-3aef-458e-9707-9fb81eb55634')\"\n",
386
+ " title=\"Convert this dataframe to an interactive table.\"\n",
387
+ " style=\"display:none;\">\n",
388
+ "\n",
389
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
390
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
391
+ " </svg>\n",
392
+ " </button>\n",
393
+ "\n",
394
+ " <style>\n",
395
+ " .colab-df-container {\n",
396
+ " display:flex;\n",
397
+ " gap: 12px;\n",
398
+ " }\n",
399
+ "\n",
400
+ " .colab-df-convert {\n",
401
+ " background-color: #E8F0FE;\n",
402
+ " border: none;\n",
403
+ " border-radius: 50%;\n",
404
+ " cursor: pointer;\n",
405
+ " display: none;\n",
406
+ " fill: #1967D2;\n",
407
+ " height: 32px;\n",
408
+ " padding: 0 0 0 0;\n",
409
+ " width: 32px;\n",
410
+ " }\n",
411
+ "\n",
412
+ " .colab-df-convert:hover {\n",
413
+ " background-color: #E2EBFA;\n",
414
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
415
+ " fill: #174EA6;\n",
416
+ " }\n",
417
+ "\n",
418
+ " .colab-df-buttons div {\n",
419
+ " margin-bottom: 4px;\n",
420
+ " }\n",
421
+ "\n",
422
+ " [theme=dark] .colab-df-convert {\n",
423
+ " background-color: #3B4455;\n",
424
+ " fill: #D2E3FC;\n",
425
+ " }\n",
426
+ "\n",
427
+ " [theme=dark] .colab-df-convert:hover {\n",
428
+ " background-color: #434B5C;\n",
429
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
430
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
431
+ " fill: #FFFFFF;\n",
432
+ " }\n",
433
+ " </style>\n",
434
+ "\n",
435
+ " <script>\n",
436
+ " const buttonEl =\n",
437
+ " document.querySelector('#df-d8591515-3aef-458e-9707-9fb81eb55634 button.colab-df-convert');\n",
438
+ " buttonEl.style.display =\n",
439
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
440
+ "\n",
441
+ " async function convertToInteractive(key) {\n",
442
+ " const element = document.querySelector('#df-d8591515-3aef-458e-9707-9fb81eb55634');\n",
443
+ " const dataTable =\n",
444
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
445
+ " [key], {});\n",
446
+ " if (!dataTable) return;\n",
447
+ "\n",
448
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
449
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
450
+ " + ' to learn more about interactive tables.';\n",
451
+ " element.innerHTML = '';\n",
452
+ " dataTable['output_type'] = 'display_data';\n",
453
+ " await google.colab.output.renderOutput(dataTable, element);\n",
454
+ " const docLink = document.createElement('div');\n",
455
+ " docLink.innerHTML = docLinkHtml;\n",
456
+ " element.appendChild(docLink);\n",
457
+ " }\n",
458
+ " </script>\n",
459
+ " </div>\n",
460
+ "\n",
461
+ "\n",
462
+ " </div>\n",
463
+ " </div>\n"
464
+ ],
465
+ "application/vnd.google.colaboratory.intrinsic+json": {
466
+ "type": "dataframe",
467
+ "variable_name": "ml_meta_movies"
468
+ }
469
+ },
470
+ "metadata": {},
471
+ "execution_count": 181
472
+ }
473
+ ]
474
+ },
475
+ {
476
+ "cell_type": "code",
477
+ "source": [
478
+ "print(movies[\"movie_id\"].nunique())\n",
479
+ "print(users[\"user_id\"].nunique())"
480
+ ],
481
+ "metadata": {
482
+ "colab": {
483
+ "base_uri": "https://localhost:8080/"
484
+ },
485
+ "id": "fNy0OWLnqHkC",
486
+ "outputId": "fafeda16-2ece-4738-ae68-3189b7d30cca"
487
+ },
488
+ "execution_count": 182,
489
+ "outputs": [
490
+ {
491
+ "output_type": "stream",
492
+ "name": "stdout",
493
+ "text": [
494
+ "1682\n",
495
+ "943\n"
496
+ ]
497
+ }
498
+ ]
499
+ },
500
+ {
501
+ "cell_type": "code",
502
+ "source": [
503
+ "# prompt: i wanna check that all title on dataframe movies is exists on dataframe ml_meta_movies with column title or original_title and how many title is not exists, with text is lowercase, and remove the row on movies if is not exists.\n",
504
+ "\n",
505
+ "# Convert titles to lowercase for comparison\n",
506
+ "movies['title_lower'] = movies['title'].str.lower()\n",
507
+ "ml_meta_movies['title_lower'] = ml_meta_movies['title'].str.lower()\n",
508
+ "ml_meta_movies['original_title_lower'] = ml_meta_movies['original_title'].str.lower()\n",
509
+ "\n",
510
+ "# Check which titles in 'movies' exist in 'ml_meta_movies'\n",
511
+ "movies_exist = movies['title_lower'].isin(ml_meta_movies['title_lower']) | movies['title_lower'].isin(ml_meta_movies['original_title_lower'])\n",
512
+ "\n",
513
+ "# Count how many titles don't exist\n",
514
+ "not_exist_count = (~movies_exist).sum()\n",
515
+ "print(\"Number of titles not existing in ml_meta_movies:\", not_exist_count)\n",
516
+ "\n",
517
+ "# Remove rows from 'movies' where titles don't exist\n",
518
+ "movies = movies[movies_exist]\n",
519
+ "\n",
520
+ "# Drop the temporary lowercase title columns\n",
521
+ "movies = movies.drop(['title_lower'], axis=1)\n",
522
+ "ml_meta_movies = ml_meta_movies.drop(['title_lower', 'original_title_lower'], axis=1)\n",
523
+ "movies.reset_index(drop=True, inplace=True)"
524
+ ],
525
+ "metadata": {
526
+ "colab": {
527
+ "base_uri": "https://localhost:8080/"
528
+ },
529
+ "id": "Wg9oVcqi9m7p",
530
+ "outputId": "b664355e-2d3f-4357-a02b-1f6d0e39b355"
531
+ },
532
+ "execution_count": 183,
533
+ "outputs": [
534
+ {
535
+ "output_type": "stream",
536
+ "name": "stdout",
537
+ "text": [
538
+ "Number of titles not existing in ml_meta_movies: 518\n"
539
+ ]
540
+ }
541
+ ]
542
+ },
543
+ {
544
+ "cell_type": "code",
545
+ "source": [
546
+ "# prompt: remove all rows on ratings dataframe if the column movie_id is not exists on movies dataframe\n",
547
+ "\n",
548
+ "# Filter ratings DataFrame based on movie existence\n",
549
+ "ratings = ratings[ratings['movie_id'].isin(movies['movie_id'])]\n",
550
+ "ratings.reset_index(drop=True, inplace=True)"
551
+ ],
552
+ "metadata": {
553
+ "id": "2FZmRDYP-MQM"
554
+ },
555
+ "execution_count": 184,
556
+ "outputs": []
557
+ },
558
+ {
559
+ "cell_type": "code",
560
+ "source": [
561
+ "# prompt: Can you reset movie_id column on movies datafram start to 1 and syncronize to movie_id on ratings dataframe\n",
562
+ "\n",
563
+ "# Create a mapping of old movie_id to new movie_id\n",
564
+ "movie_id_map = {old_id: new_id for new_id, old_id in enumerate(movies['movie_id'].unique(), start=1)}\n",
565
+ "\n",
566
+ "# Apply the mapping to the movies DataFrame\n",
567
+ "movies['movie_id'] = movies['movie_id'].map(movie_id_map)\n",
568
+ "\n",
569
+ "# Apply the mapping to the ratings DataFrame\n",
570
+ "ratings['movie_id'] = ratings['movie_id'].map(movie_id_map)"
571
+ ],
572
+ "metadata": {
573
+ "colab": {
574
+ "base_uri": "https://localhost:8080/"
575
+ },
576
+ "id": "WPOSzdEoB7qP",
577
+ "outputId": "02a53bdb-92f0-456d-8aae-d4b30777d04e"
578
+ },
579
+ "execution_count": 185,
580
+ "outputs": [
581
+ {
582
+ "output_type": "stream",
583
+ "name": "stderr",
584
+ "text": [
585
+ "<ipython-input-185-05ef1c64b40f>:10: SettingWithCopyWarning: \n",
586
+ "A value is trying to be set on a copy of a slice from a DataFrame.\n",
587
+ "Try using .loc[row_indexer,col_indexer] = value instead\n",
588
+ "\n",
589
+ "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
590
+ " ratings['movie_id'] = ratings['movie_id'].map(movie_id_map)\n"
591
+ ]
592
+ }
593
+ ]
594
+ },
595
+ {
596
+ "cell_type": "code",
597
+ "source": [
598
+ "# prompt: Can you reset user_id column on ratings datafram start to 1 and syncronize to user_id on users dataframe and remove all rows on users when the user_id is not exist on unique user_id on ratings\n",
599
+ "\n",
600
+ "# Get unique user_ids from ratings\n",
601
+ "unique_rating_users = ratings['user_id'].unique()\n",
602
+ "\n",
603
+ "# Filter users DataFrame to keep only users present in ratings\n",
604
+ "users = users[users['user_id'].isin(unique_rating_users)]\n",
605
+ "users.reset_index(drop=True, inplace=True)\n",
606
+ "\n",
607
+ "# Create a mapping of old user_id to new user_id\n",
608
+ "user_id_map = {old_id: new_id for new_id, old_id in enumerate(users['user_id'].unique(), start=1)}\n",
609
+ "\n",
610
+ "# Apply the mapping to the users DataFrame\n",
611
+ "users['user_id'] = users['user_id'].map(user_id_map)\n",
612
+ "\n",
613
+ "# Apply the mapping to the ratings DataFrame\n",
614
+ "ratings['user_id'] = ratings['user_id'].map(user_id_map)\n"
615
+ ],
616
+ "metadata": {
617
+ "colab": {
618
+ "base_uri": "https://localhost:8080/"
619
+ },
620
+ "id": "YulqHdu7Dmf6",
621
+ "outputId": "9ff98c2c-0b7d-419c-8698-bfb51c48ca09"
622
+ },
623
+ "execution_count": 186,
624
+ "outputs": [
625
+ {
626
+ "output_type": "stream",
627
+ "name": "stderr",
628
+ "text": [
629
+ "<ipython-input-186-63a67bcdbf80>:17: SettingWithCopyWarning: \n",
630
+ "A value is trying to be set on a copy of a slice from a DataFrame.\n",
631
+ "Try using .loc[row_indexer,col_indexer] = value instead\n",
632
+ "\n",
633
+ "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
634
+ " ratings['user_id'] = ratings['user_id'].map(user_id_map)\n"
635
+ ]
636
+ }
637
+ ]
638
+ },
639
+ {
640
+ "cell_type": "code",
641
+ "source": [
642
+ "# prompt: Now i want to copy columns adult, original_language, original_title, overview, popularity, release_date, revenue, runtime, vote_average, dan vote_count from ml_meta_movies to movies dataframe based on title or original_title\n",
643
+ "\n",
644
+ "# Create temporary lowercase title columns for efficient comparison\n",
645
+ "movies['title_lower'] = movies['title'].str.lower()\n",
646
+ "ml_meta_movies['title_lower'] = ml_meta_movies['title'].str.lower()\n",
647
+ "ml_meta_movies['original_title_lower'] = ml_meta_movies['original_title'].str.lower()\n",
648
+ "\n",
649
+ "# Initialize new columns in 'movies' DataFrame\n",
650
+ "movies['adult'] = None\n",
651
+ "movies['original_language'] = None\n",
652
+ "movies['original_title'] = None\n",
653
+ "movies['overview'] = None\n",
654
+ "movies['popularity'] = None\n",
655
+ "movies['release_date'] = None\n",
656
+ "movies['revenue'] = None\n",
657
+ "movies['runtime'] = None\n",
658
+ "movies['vote_average'] = None\n",
659
+ "movies['vote_count'] = None\n",
660
+ "\n",
661
+ "# Iterate over 'movies' and copy data from 'ml_meta_movies'\n",
662
+ "for index, row in movies.iterrows():\n",
663
+ " title_lower = row['title_lower']\n",
664
+ " match = ml_meta_movies[(ml_meta_movies['title_lower'] == title_lower) | (ml_meta_movies['original_title_lower'] == title_lower)]\n",
665
+ " if not match.empty:\n",
666
+ " movies.loc[index, 'adult'] = match['adult'].iloc[0]\n",
667
+ " movies.loc[index, 'original_language'] = match['original_language'].iloc[0]\n",
668
+ " movies.loc[index, 'original_title'] = match['original_title'].iloc[0]\n",
669
+ " movies.loc[index, 'overview'] = match['overview'].iloc[0]\n",
670
+ " movies.loc[index, 'popularity'] = match['popularity'].iloc[0]\n",
671
+ " movies.loc[index, 'release_date'] = match['release_date'].iloc[0]\n",
672
+ " movies.loc[index, 'revenue'] = match['revenue'].iloc[0]\n",
673
+ " movies.loc[index, 'runtime'] = match['runtime'].iloc[0]\n",
674
+ " movies.loc[index, 'vote_average'] = match['vote_average'].iloc[0]\n",
675
+ " movies.loc[index, 'vote_count'] = match['vote_count'].iloc[0]\n",
676
+ "\n",
677
+ "# Drop the temporary lowercase title columns\n",
678
+ "movies = movies.drop(['title_lower'], axis=1)\n",
679
+ "ml_meta_movies = ml_meta_movies.drop(['title_lower', 'original_title_lower'], axis=1)\n"
680
+ ],
681
+ "metadata": {
682
+ "id": "joj4h0U2JNRL"
683
+ },
684
+ "execution_count": 187,
685
+ "outputs": []
686
+ },
687
+ {
688
+ "cell_type": "markdown",
689
+ "source": [
690
+ "## Show Tables"
691
+ ],
692
+ "metadata": {
693
+ "id": "jA-vHTcjiaYj"
694
+ }
695
+ },
696
+ {
697
+ "cell_type": "code",
698
+ "source": [
699
+ "# Ratings\n",
700
+ "ratings.head(1)"
701
+ ],
702
+ "metadata": {
703
+ "colab": {
704
+ "base_uri": "https://localhost:8080/",
705
+ "height": 0
706
+ },
707
+ "id": "-gEkmf5mevq2",
708
+ "outputId": "b0495a83-b6f9-41da-d493-4f04dd3efb3e"
709
+ },
710
+ "execution_count": 188,
711
+ "outputs": [
712
+ {
713
+ "output_type": "execute_result",
714
+ "data": {
715
+ "text/plain": [
716
+ " user_id movie_id rating timestamp\n",
717
+ "0 196 169 3 881250949"
718
+ ],
719
+ "text/html": [
720
+ "\n",
721
+ " <div id=\"df-046071fc-1ebc-4261-8a1c-d4bca4119035\" class=\"colab-df-container\">\n",
722
+ " <div>\n",
723
+ "<style scoped>\n",
724
+ " .dataframe tbody tr th:only-of-type {\n",
725
+ " vertical-align: middle;\n",
726
+ " }\n",
727
+ "\n",
728
+ " .dataframe tbody tr th {\n",
729
+ " vertical-align: top;\n",
730
+ " }\n",
731
+ "\n",
732
+ " .dataframe thead th {\n",
733
+ " text-align: right;\n",
734
+ " }\n",
735
+ "</style>\n",
736
+ "<table border=\"1\" class=\"dataframe\">\n",
737
+ " <thead>\n",
738
+ " <tr style=\"text-align: right;\">\n",
739
+ " <th></th>\n",
740
+ " <th>user_id</th>\n",
741
+ " <th>movie_id</th>\n",
742
+ " <th>rating</th>\n",
743
+ " <th>timestamp</th>\n",
744
+ " </tr>\n",
745
+ " </thead>\n",
746
+ " <tbody>\n",
747
+ " <tr>\n",
748
+ " <th>0</th>\n",
749
+ " <td>196</td>\n",
750
+ " <td>169</td>\n",
751
+ " <td>3</td>\n",
752
+ " <td>881250949</td>\n",
753
+ " </tr>\n",
754
+ " </tbody>\n",
755
+ "</table>\n",
756
+ "</div>\n",
757
+ " <div class=\"colab-df-buttons\">\n",
758
+ "\n",
759
+ " <div class=\"colab-df-container\">\n",
760
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-046071fc-1ebc-4261-8a1c-d4bca4119035')\"\n",
761
+ " title=\"Convert this dataframe to an interactive table.\"\n",
762
+ " style=\"display:none;\">\n",
763
+ "\n",
764
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
765
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
766
+ " </svg>\n",
767
+ " </button>\n",
768
+ "\n",
769
+ " <style>\n",
770
+ " .colab-df-container {\n",
771
+ " display:flex;\n",
772
+ " gap: 12px;\n",
773
+ " }\n",
774
+ "\n",
775
+ " .colab-df-convert {\n",
776
+ " background-color: #E8F0FE;\n",
777
+ " border: none;\n",
778
+ " border-radius: 50%;\n",
779
+ " cursor: pointer;\n",
780
+ " display: none;\n",
781
+ " fill: #1967D2;\n",
782
+ " height: 32px;\n",
783
+ " padding: 0 0 0 0;\n",
784
+ " width: 32px;\n",
785
+ " }\n",
786
+ "\n",
787
+ " .colab-df-convert:hover {\n",
788
+ " background-color: #E2EBFA;\n",
789
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
790
+ " fill: #174EA6;\n",
791
+ " }\n",
792
+ "\n",
793
+ " .colab-df-buttons div {\n",
794
+ " margin-bottom: 4px;\n",
795
+ " }\n",
796
+ "\n",
797
+ " [theme=dark] .colab-df-convert {\n",
798
+ " background-color: #3B4455;\n",
799
+ " fill: #D2E3FC;\n",
800
+ " }\n",
801
+ "\n",
802
+ " [theme=dark] .colab-df-convert:hover {\n",
803
+ " background-color: #434B5C;\n",
804
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
805
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
806
+ " fill: #FFFFFF;\n",
807
+ " }\n",
808
+ " </style>\n",
809
+ "\n",
810
+ " <script>\n",
811
+ " const buttonEl =\n",
812
+ " document.querySelector('#df-046071fc-1ebc-4261-8a1c-d4bca4119035 button.colab-df-convert');\n",
813
+ " buttonEl.style.display =\n",
814
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
815
+ "\n",
816
+ " async function convertToInteractive(key) {\n",
817
+ " const element = document.querySelector('#df-046071fc-1ebc-4261-8a1c-d4bca4119035');\n",
818
+ " const dataTable =\n",
819
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
820
+ " [key], {});\n",
821
+ " if (!dataTable) return;\n",
822
+ "\n",
823
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
824
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
825
+ " + ' to learn more about interactive tables.';\n",
826
+ " element.innerHTML = '';\n",
827
+ " dataTable['output_type'] = 'display_data';\n",
828
+ " await google.colab.output.renderOutput(dataTable, element);\n",
829
+ " const docLink = document.createElement('div');\n",
830
+ " docLink.innerHTML = docLinkHtml;\n",
831
+ " element.appendChild(docLink);\n",
832
+ " }\n",
833
+ " </script>\n",
834
+ " </div>\n",
835
+ "\n",
836
+ "\n",
837
+ " </div>\n",
838
+ " </div>\n"
839
+ ],
840
+ "application/vnd.google.colaboratory.intrinsic+json": {
841
+ "type": "dataframe",
842
+ "variable_name": "ratings",
843
+ "summary": "{\n \"name\": \"ratings\",\n \"rows\": 72799,\n \"fields\": [\n {\n \"column\": \"user_id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 266,\n \"min\": 1,\n \"max\": 943,\n \"num_unique_values\": 943,\n \"samples\": [\n 1,\n 204,\n 812\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"movie_id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 236,\n \"min\": 1,\n \"max\": 1164,\n \"num_unique_values\": 1164,\n \"samples\": [\n 652,\n 683,\n 485\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rating\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 1,\n 5,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"timestamp\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 5332640,\n \"min\": 874724710,\n \"max\": 893286638,\n \"num_unique_values\": 41739,\n \"samples\": [\n 892836523,\n 891224840,\n 882910457\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
844
+ }
845
+ },
846
+ "metadata": {},
847
+ "execution_count": 188
848
+ }
849
+ ]
850
+ },
851
+ {
852
+ "cell_type": "code",
853
+ "source": [
854
+ "# Users\n",
855
+ "users.head(1)"
856
+ ],
857
+ "metadata": {
858
+ "colab": {
859
+ "base_uri": "https://localhost:8080/",
860
+ "height": 0
861
+ },
862
+ "id": "02jusHhgipqH",
863
+ "outputId": "f8b6812e-1106-4f08-967f-f524b2b7e45b"
864
+ },
865
+ "execution_count": 189,
866
+ "outputs": [
867
+ {
868
+ "output_type": "execute_result",
869
+ "data": {
870
+ "text/plain": [
871
+ " user_id gender age occupation zip_code\n",
872
+ "0 1 24 M technician 85711"
873
+ ],
874
+ "text/html": [
875
+ "\n",
876
+ " <div id=\"df-1b7be07c-888a-4129-ab46-b905f73eb705\" class=\"colab-df-container\">\n",
877
+ " <div>\n",
878
+ "<style scoped>\n",
879
+ " .dataframe tbody tr th:only-of-type {\n",
880
+ " vertical-align: middle;\n",
881
+ " }\n",
882
+ "\n",
883
+ " .dataframe tbody tr th {\n",
884
+ " vertical-align: top;\n",
885
+ " }\n",
886
+ "\n",
887
+ " .dataframe thead th {\n",
888
+ " text-align: right;\n",
889
+ " }\n",
890
+ "</style>\n",
891
+ "<table border=\"1\" class=\"dataframe\">\n",
892
+ " <thead>\n",
893
+ " <tr style=\"text-align: right;\">\n",
894
+ " <th></th>\n",
895
+ " <th>user_id</th>\n",
896
+ " <th>gender</th>\n",
897
+ " <th>age</th>\n",
898
+ " <th>occupation</th>\n",
899
+ " <th>zip_code</th>\n",
900
+ " </tr>\n",
901
+ " </thead>\n",
902
+ " <tbody>\n",
903
+ " <tr>\n",
904
+ " <th>0</th>\n",
905
+ " <td>1</td>\n",
906
+ " <td>24</td>\n",
907
+ " <td>M</td>\n",
908
+ " <td>technician</td>\n",
909
+ " <td>85711</td>\n",
910
+ " </tr>\n",
911
+ " </tbody>\n",
912
+ "</table>\n",
913
+ "</div>\n",
914
+ " <div class=\"colab-df-buttons\">\n",
915
+ "\n",
916
+ " <div class=\"colab-df-container\">\n",
917
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1b7be07c-888a-4129-ab46-b905f73eb705')\"\n",
918
+ " title=\"Convert this dataframe to an interactive table.\"\n",
919
+ " style=\"display:none;\">\n",
920
+ "\n",
921
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
922
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
923
+ " </svg>\n",
924
+ " </button>\n",
925
+ "\n",
926
+ " <style>\n",
927
+ " .colab-df-container {\n",
928
+ " display:flex;\n",
929
+ " gap: 12px;\n",
930
+ " }\n",
931
+ "\n",
932
+ " .colab-df-convert {\n",
933
+ " background-color: #E8F0FE;\n",
934
+ " border: none;\n",
935
+ " border-radius: 50%;\n",
936
+ " cursor: pointer;\n",
937
+ " display: none;\n",
938
+ " fill: #1967D2;\n",
939
+ " height: 32px;\n",
940
+ " padding: 0 0 0 0;\n",
941
+ " width: 32px;\n",
942
+ " }\n",
943
+ "\n",
944
+ " .colab-df-convert:hover {\n",
945
+ " background-color: #E2EBFA;\n",
946
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
947
+ " fill: #174EA6;\n",
948
+ " }\n",
949
+ "\n",
950
+ " .colab-df-buttons div {\n",
951
+ " margin-bottom: 4px;\n",
952
+ " }\n",
953
+ "\n",
954
+ " [theme=dark] .colab-df-convert {\n",
955
+ " background-color: #3B4455;\n",
956
+ " fill: #D2E3FC;\n",
957
+ " }\n",
958
+ "\n",
959
+ " [theme=dark] .colab-df-convert:hover {\n",
960
+ " background-color: #434B5C;\n",
961
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
962
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
963
+ " fill: #FFFFFF;\n",
964
+ " }\n",
965
+ " </style>\n",
966
+ "\n",
967
+ " <script>\n",
968
+ " const buttonEl =\n",
969
+ " document.querySelector('#df-1b7be07c-888a-4129-ab46-b905f73eb705 button.colab-df-convert');\n",
970
+ " buttonEl.style.display =\n",
971
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
972
+ "\n",
973
+ " async function convertToInteractive(key) {\n",
974
+ " const element = document.querySelector('#df-1b7be07c-888a-4129-ab46-b905f73eb705');\n",
975
+ " const dataTable =\n",
976
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
977
+ " [key], {});\n",
978
+ " if (!dataTable) return;\n",
979
+ "\n",
980
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
981
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
982
+ " + ' to learn more about interactive tables.';\n",
983
+ " element.innerHTML = '';\n",
984
+ " dataTable['output_type'] = 'display_data';\n",
985
+ " await google.colab.output.renderOutput(dataTable, element);\n",
986
+ " const docLink = document.createElement('div');\n",
987
+ " docLink.innerHTML = docLinkHtml;\n",
988
+ " element.appendChild(docLink);\n",
989
+ " }\n",
990
+ " </script>\n",
991
+ " </div>\n",
992
+ "\n",
993
+ "\n",
994
+ " </div>\n",
995
+ " </div>\n"
996
+ ],
997
+ "application/vnd.google.colaboratory.intrinsic+json": {
998
+ "type": "dataframe",
999
+ "variable_name": "users",
1000
+ "summary": "{\n \"name\": \"users\",\n \"rows\": 943,\n \"fields\": [\n {\n \"column\": \"user_id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 272,\n \"min\": 1,\n \"max\": 943,\n \"num_unique_values\": 943,\n \"samples\": [\n 97,\n 266,\n 811\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"gender\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 12,\n \"min\": 7,\n \"max\": 73,\n \"num_unique_values\": 61,\n \"samples\": [\n 24,\n 57,\n 52\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"age\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"F\",\n \"M\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"occupation\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 21,\n \"samples\": [\n \"technician\",\n \"healthcare\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"zip_code\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 795,\n \"samples\": [\n \"90016\",\n \"15232\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1001
+ }
1002
+ },
1003
+ "metadata": {},
1004
+ "execution_count": 189
1005
+ }
1006
+ ]
1007
+ },
1008
+ {
1009
+ "cell_type": "code",
1010
+ "source": [
1011
+ "# Movies\n",
1012
+ "movies.head(1)"
1013
+ ],
1014
+ "metadata": {
1015
+ "colab": {
1016
+ "base_uri": "https://localhost:8080/",
1017
+ "height": 0
1018
+ },
1019
+ "id": "5y3nX2tQivh_",
1020
+ "outputId": "ec5a7f2c-eb79-4fc9-a58e-5473a6074d2a"
1021
+ },
1022
+ "execution_count": 190,
1023
+ "outputs": [
1024
+ {
1025
+ "output_type": "execute_result",
1026
+ "data": {
1027
+ "text/plain": [
1028
+ " movie_id title release_date \\\n",
1029
+ "0 1 Toy Story 1995-10-30 \n",
1030
+ "\n",
1031
+ " imdb_url \\\n",
1032
+ "0 http://us.imdb.com/M/title-exact?Toy%20Story%2... \n",
1033
+ "\n",
1034
+ " genres year adult original_language original_title \\\n",
1035
+ "0 Animation|Children's|Comedy 1995 False en Toy Story \n",
1036
+ "\n",
1037
+ " overview popularity revenue \\\n",
1038
+ "0 Led by Woody, Andy's toys live happily in his ... 21.946943 373554033.0 \n",
1039
+ "\n",
1040
+ " runtime vote_average vote_count \n",
1041
+ "0 81.0 7.7 5415.0 "
1042
+ ],
1043
+ "text/html": [
1044
+ "\n",
1045
+ " <div id=\"df-1c36e9a6-1049-44ca-879c-ccaf267483f7\" class=\"colab-df-container\">\n",
1046
+ " <div>\n",
1047
+ "<style scoped>\n",
1048
+ " .dataframe tbody tr th:only-of-type {\n",
1049
+ " vertical-align: middle;\n",
1050
+ " }\n",
1051
+ "\n",
1052
+ " .dataframe tbody tr th {\n",
1053
+ " vertical-align: top;\n",
1054
+ " }\n",
1055
+ "\n",
1056
+ " .dataframe thead th {\n",
1057
+ " text-align: right;\n",
1058
+ " }\n",
1059
+ "</style>\n",
1060
+ "<table border=\"1\" class=\"dataframe\">\n",
1061
+ " <thead>\n",
1062
+ " <tr style=\"text-align: right;\">\n",
1063
+ " <th></th>\n",
1064
+ " <th>movie_id</th>\n",
1065
+ " <th>title</th>\n",
1066
+ " <th>release_date</th>\n",
1067
+ " <th>imdb_url</th>\n",
1068
+ " <th>genres</th>\n",
1069
+ " <th>year</th>\n",
1070
+ " <th>adult</th>\n",
1071
+ " <th>original_language</th>\n",
1072
+ " <th>original_title</th>\n",
1073
+ " <th>overview</th>\n",
1074
+ " <th>popularity</th>\n",
1075
+ " <th>revenue</th>\n",
1076
+ " <th>runtime</th>\n",
1077
+ " <th>vote_average</th>\n",
1078
+ " <th>vote_count</th>\n",
1079
+ " </tr>\n",
1080
+ " </thead>\n",
1081
+ " <tbody>\n",
1082
+ " <tr>\n",
1083
+ " <th>0</th>\n",
1084
+ " <td>1</td>\n",
1085
+ " <td>Toy Story</td>\n",
1086
+ " <td>1995-10-30</td>\n",
1087
+ " <td>http://us.imdb.com/M/title-exact?Toy%20Story%2...</td>\n",
1088
+ " <td>Animation|Children's|Comedy</td>\n",
1089
+ " <td>1995</td>\n",
1090
+ " <td>False</td>\n",
1091
+ " <td>en</td>\n",
1092
+ " <td>Toy Story</td>\n",
1093
+ " <td>Led by Woody, Andy's toys live happily in his ...</td>\n",
1094
+ " <td>21.946943</td>\n",
1095
+ " <td>373554033.0</td>\n",
1096
+ " <td>81.0</td>\n",
1097
+ " <td>7.7</td>\n",
1098
+ " <td>5415.0</td>\n",
1099
+ " </tr>\n",
1100
+ " </tbody>\n",
1101
+ "</table>\n",
1102
+ "</div>\n",
1103
+ " <div class=\"colab-df-buttons\">\n",
1104
+ "\n",
1105
+ " <div class=\"colab-df-container\">\n",
1106
+ " <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1c36e9a6-1049-44ca-879c-ccaf267483f7')\"\n",
1107
+ " title=\"Convert this dataframe to an interactive table.\"\n",
1108
+ " style=\"display:none;\">\n",
1109
+ "\n",
1110
+ " <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
1111
+ " <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
1112
+ " </svg>\n",
1113
+ " </button>\n",
1114
+ "\n",
1115
+ " <style>\n",
1116
+ " .colab-df-container {\n",
1117
+ " display:flex;\n",
1118
+ " gap: 12px;\n",
1119
+ " }\n",
1120
+ "\n",
1121
+ " .colab-df-convert {\n",
1122
+ " background-color: #E8F0FE;\n",
1123
+ " border: none;\n",
1124
+ " border-radius: 50%;\n",
1125
+ " cursor: pointer;\n",
1126
+ " display: none;\n",
1127
+ " fill: #1967D2;\n",
1128
+ " height: 32px;\n",
1129
+ " padding: 0 0 0 0;\n",
1130
+ " width: 32px;\n",
1131
+ " }\n",
1132
+ "\n",
1133
+ " .colab-df-convert:hover {\n",
1134
+ " background-color: #E2EBFA;\n",
1135
+ " box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
1136
+ " fill: #174EA6;\n",
1137
+ " }\n",
1138
+ "\n",
1139
+ " .colab-df-buttons div {\n",
1140
+ " margin-bottom: 4px;\n",
1141
+ " }\n",
1142
+ "\n",
1143
+ " [theme=dark] .colab-df-convert {\n",
1144
+ " background-color: #3B4455;\n",
1145
+ " fill: #D2E3FC;\n",
1146
+ " }\n",
1147
+ "\n",
1148
+ " [theme=dark] .colab-df-convert:hover {\n",
1149
+ " background-color: #434B5C;\n",
1150
+ " box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
1151
+ " filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
1152
+ " fill: #FFFFFF;\n",
1153
+ " }\n",
1154
+ " </style>\n",
1155
+ "\n",
1156
+ " <script>\n",
1157
+ " const buttonEl =\n",
1158
+ " document.querySelector('#df-1c36e9a6-1049-44ca-879c-ccaf267483f7 button.colab-df-convert');\n",
1159
+ " buttonEl.style.display =\n",
1160
+ " google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
1161
+ "\n",
1162
+ " async function convertToInteractive(key) {\n",
1163
+ " const element = document.querySelector('#df-1c36e9a6-1049-44ca-879c-ccaf267483f7');\n",
1164
+ " const dataTable =\n",
1165
+ " await google.colab.kernel.invokeFunction('convertToInteractive',\n",
1166
+ " [key], {});\n",
1167
+ " if (!dataTable) return;\n",
1168
+ "\n",
1169
+ " const docLinkHtml = 'Like what you see? Visit the ' +\n",
1170
+ " '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
1171
+ " + ' to learn more about interactive tables.';\n",
1172
+ " element.innerHTML = '';\n",
1173
+ " dataTable['output_type'] = 'display_data';\n",
1174
+ " await google.colab.output.renderOutput(dataTable, element);\n",
1175
+ " const docLink = document.createElement('div');\n",
1176
+ " docLink.innerHTML = docLinkHtml;\n",
1177
+ " element.appendChild(docLink);\n",
1178
+ " }\n",
1179
+ " </script>\n",
1180
+ " </div>\n",
1181
+ "\n",
1182
+ "\n",
1183
+ " </div>\n",
1184
+ " </div>\n"
1185
+ ],
1186
+ "application/vnd.google.colaboratory.intrinsic+json": {
1187
+ "type": "dataframe",
1188
+ "variable_name": "movies",
1189
+ "summary": "{\n \"name\": \"movies\",\n \"rows\": 1164,\n \"fields\": [\n {\n \"column\": \"movie_id\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 336,\n \"min\": 1,\n \"max\": 1164,\n \"num_unique_values\": 1164,\n \"samples\": [\n 765,\n 102,\n 774\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1145,\n \"samples\": [\n \"Titanic\",\n \"Hard Eight\",\n \"Immortal Beloved\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"release_date\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 756,\n \"samples\": [\n \"1977-04-06\",\n \"1973-12-17\",\n \"1994-05-13\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb_url\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1149,\n \"samples\": [\n \"http://us.imdb.com/M/title-exact?Shall%20we%20DANSU%3F%20%281996%29\",\n \"http://us.imdb.com/M/title-exact?Koyaanisqatsi%20(1983)\",\n \"http://us.imdb.com/M/title-exact?Conan+the+Barbarian+(1981)\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 176,\n \"samples\": [\n \"Documentary\",\n \"Comedy|Drama|Romance\",\n \"Action|Romance\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"year\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 65,\n \"samples\": [\n \"1943\",\n \"1952\",\n \"1995\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"adult\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"False\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"original_language\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 20,\n \"samples\": [\n \"en\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"original_title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1145,\n \"samples\": [\n \"Titanic\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"overview\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1139,\n \"samples\": [\n \"Dorothy Parker remembers the heyday of the Algonquin Round Table, a circle of friends whose barbed wit, like hers, was fueled by alcohol and flirted with despair.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"popularity\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1144,\n \"samples\": [\n \"26.88907\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"revenue\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": 0.0,\n \"max\": 1845034188.0,\n \"num_unique_values\": 585,\n \"samples\": [\n 19075720.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": 0.0,\n \"max\": 242.0,\n \"num_unique_values\": 113,\n \"samples\": [\n 141.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"vote_average\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": 0.0,\n \"max\": 10.0,\n \"num_unique_values\": 57,\n \"samples\": [\n 7.7\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"vote_count\",\n \"properties\": {\n \"dtype\": \"date\",\n \"min\": 0.0,\n \"max\": 8670.0,\n \"num_unique_values\": 484,\n \"samples\": [\n 92.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
1190
+ }
1191
+ },
1192
+ "metadata": {},
1193
+ "execution_count": 190
1194
+ }
1195
+ ]
1196
+ },
1197
+ {
1198
+ "cell_type": "markdown",
1199
+ "source": [
1200
+ "## Sample Recsys to check dataset is valid for embedding or not"
1201
+ ],
1202
+ "metadata": {
1203
+ "id": "hU4wX4dn-zXO"
1204
+ }
1205
+ },
1206
+ {
1207
+ "cell_type": "code",
1208
+ "source": [
1209
+ "# Memuat data\n",
1210
+ "data = ratings.copy()\n",
1211
+ "data = data[['user_id', 'movie_id', 'rating']]\n",
1212
+ "\n",
1213
+ "# Normalisasi ID pengguna dan item (karena ID asli mungkin tidak dimulai dari 0)\n",
1214
+ "data['user_id'] = data['user_id'] - 1\n",
1215
+ "data['movie_id'] = data['movie_id'] - 1\n",
1216
+ "\n",
1217
+ "# Melihat statistik dataset\n",
1218
+ "num_users = data['user_id'].nunique()\n",
1219
+ "num_items = data['movie_id'].nunique()\n",
1220
+ "print(f\"Number of users: {num_users}, Number of items: {num_items}\")\n",
1221
+ "\n",
1222
+ "# Split dataset menjadi train dan test\n",
1223
+ "train_data, test_data = train_test_split(data, test_size=0.2, random_state=42)"
1224
+ ],
1225
+ "metadata": {
1226
+ "colab": {
1227
+ "base_uri": "https://localhost:8080/"
1228
+ },
1229
+ "id": "kI90v3Mu-9Bd",
1230
+ "outputId": "0fdd7e17-60d0-4727-9421-6c081f12d621"
1231
+ },
1232
+ "execution_count": 191,
1233
+ "outputs": [
1234
+ {
1235
+ "output_type": "stream",
1236
+ "name": "stdout",
1237
+ "text": [
1238
+ "Number of users: 943, Number of items: 1164\n"
1239
+ ]
1240
+ }
1241
+ ]
1242
+ },
1243
+ {
1244
+ "cell_type": "code",
1245
+ "source": [
1246
+ "import torch\n",
1247
+ "import torch.nn as nn\n",
1248
+ "import torch.optim as optim\n",
1249
+ "from torch.utils.data import DataLoader, Dataset\n",
1250
+ "\n",
1251
+ "class MovieLensDataset(Dataset):\n",
1252
+ " def __init__(self, data):\n",
1253
+ " self.user_ids = torch.tensor(data['user_id'].values, dtype=torch.long)\n",
1254
+ " self.item_ids = torch.tensor(data['movie_id'].values, dtype=torch.long)\n",
1255
+ " self.ratings = torch.tensor(data['rating'].values, dtype=torch.float32)\n",
1256
+ "\n",
1257
+ " def __len__(self):\n",
1258
+ " return len(self.ratings)\n",
1259
+ "\n",
1260
+ " def __getitem__(self, idx):\n",
1261
+ " return self.user_ids[idx], self.item_ids[idx], self.ratings[idx]\n",
1262
+ "\n",
1263
+ "class MFModel(nn.Module):\n",
1264
+ " def __init__(self, num_users, num_items, embedding_size):\n",
1265
+ " super(MFModel, self).__init__()\n",
1266
+ " self.user_embedding = nn.Embedding(num_users, embedding_size)\n",
1267
+ " self.item_embedding = nn.Embedding(num_items, embedding_size)\n",
1268
+ "\n",
1269
+ " def forward(self, user_id, item_id):\n",
1270
+ " user_vec = self.user_embedding(user_id)\n",
1271
+ " item_vec = self.item_embedding(item_id)\n",
1272
+ " dot_product = torch.sum(user_vec * item_vec, dim=1)\n",
1273
+ " return dot_product\n",
1274
+ "\n",
1275
+ " def regularization_loss(self):\n",
1276
+ " return self.reg_factor * (torch.norm(self.user_embedding.weight) + torch.norm(self.item_embedding.weight))"
1277
+ ],
1278
+ "metadata": {
1279
+ "id": "s025DVSf_nhh"
1280
+ },
1281
+ "execution_count": 192,
1282
+ "outputs": []
1283
+ },
1284
+ {
1285
+ "cell_type": "code",
1286
+ "source": [
1287
+ "# DataLoader untuk training\n",
1288
+ "train_dataset = MovieLensDataset(train_data)\n",
1289
+ "train_loader = DataLoader(train_dataset, batch_size=256, shuffle=True)\n",
1290
+ "\n",
1291
+ "# Hyperparameters\n",
1292
+ "embedding_size = 30\n",
1293
+ "reg_factor = 0.01\n",
1294
+ "model = MFModel(num_users, num_items, embedding_size)\n",
1295
+ "criterion = nn.MSELoss()\n",
1296
+ "optimizer = optim.Adam(model.parameters(), lr=0.01)\n",
1297
+ "\n",
1298
+ "# Training loop\n",
1299
+ "for epoch in range(10):\n",
1300
+ " model.train()\n",
1301
+ " total_loss = 0\n",
1302
+ " for data_user_id, data_item_id, data_rating in train_loader:\n",
1303
+ " optimizer.zero_grad()\n",
1304
+ " predictions = model(data_user_id, data_item_id)\n",
1305
+ " loss = criterion(predictions, data_rating)\n",
1306
+ " loss.backward()\n",
1307
+ " optimizer.step()\n",
1308
+ " total_loss += loss.item()\n",
1309
+ " print(f\"Epoch {epoch+1}, Loss: {total_loss/len(train_loader):.4f}\")"
1310
+ ],
1311
+ "metadata": {
1312
+ "colab": {
1313
+ "base_uri": "https://localhost:8080/"
1314
+ },
1315
+ "id": "QmZazumN_ylY",
1316
+ "outputId": "60b6fb1c-0206-4b91-f41c-cb7f7cf0bae4"
1317
+ },
1318
+ "execution_count": 193,
1319
+ "outputs": [
1320
+ {
1321
+ "output_type": "stream",
1322
+ "name": "stdout",
1323
+ "text": [
1324
+ "Epoch 1, Loss: 34.7072\n",
1325
+ "Epoch 2, Loss: 18.3940\n",
1326
+ "Epoch 3, Loss: 9.6635\n",
1327
+ "Epoch 4, Loss: 4.3265\n",
1328
+ "Epoch 5, Loss: 2.3596\n",
1329
+ "Epoch 6, Loss: 1.5818\n",
1330
+ "Epoch 7, Loss: 1.1944\n",
1331
+ "Epoch 8, Loss: 0.9714\n",
1332
+ "Epoch 9, Loss: 0.8330\n",
1333
+ "Epoch 10, Loss: 0.7384\n"
1334
+ ]
1335
+ }
1336
+ ]
1337
+ },
1338
+ {
1339
+ "cell_type": "code",
1340
+ "source": [
1341
+ "from sklearn.metrics import mean_squared_error\n",
1342
+ "import numpy as np\n",
1343
+ "\n",
1344
+ "model.eval()\n",
1345
+ "test_dataset = MovieLensDataset(test_data)\n",
1346
+ "test_loader = DataLoader(test_dataset, batch_size=256, shuffle=False)\n",
1347
+ "\n",
1348
+ "predictions, targets = [], []\n",
1349
+ "with torch.no_grad():\n",
1350
+ " for data_user_id, data_item_id, data_rating in test_loader:\n",
1351
+ " output = model(data_user_id, data_item_id)\n",
1352
+ " predictions.extend(output.numpy())\n",
1353
+ " targets.extend(data_rating.numpy())\n",
1354
+ "\n",
1355
+ "rmse = np.sqrt(mean_squared_error(targets, predictions))\n",
1356
+ "print(f\"Test RMSE: {rmse:.4f}\")"
1357
+ ],
1358
+ "metadata": {
1359
+ "colab": {
1360
+ "base_uri": "https://localhost:8080/"
1361
+ },
1362
+ "id": "w9mD2UhHI0Kx",
1363
+ "outputId": "76b52339-ec74-420e-f0b8-add08e66842d"
1364
+ },
1365
+ "execution_count": 194,
1366
+ "outputs": [
1367
+ {
1368
+ "output_type": "stream",
1369
+ "name": "stdout",
1370
+ "text": [
1371
+ "Test RMSE: 1.8248\n"
1372
+ ]
1373
+ }
1374
+ ]
1375
+ },
1376
+ {
1377
+ "cell_type": "code",
1378
+ "source": [
1379
+ "def get_top_n_recommendations_pytorch(model, user_id, N=10):\n",
1380
+ " # Dapatkan semua item yang tersedia\n",
1381
+ " all_items = np.array(range(num_items))\n",
1382
+ "\n",
1383
+ " # Cek item yang sudah dirating oleh user\n",
1384
+ " rated_items = train_data[train_data['user_id'] == user_id]['movie_id'].values\n",
1385
+ "\n",
1386
+ " # Ambil item yang belum dirating oleh user\n",
1387
+ " items_to_predict = np.setdiff1d(all_items, rated_items)\n",
1388
+ "\n",
1389
+ " # Prediksi rating untuk item-item tersebut\n",
1390
+ " model.eval()\n",
1391
+ " with torch.no_grad():\n",
1392
+ " user_ids = torch.tensor([user_id] * len(items_to_predict))\n",
1393
+ " item_ids = torch.tensor(items_to_predict)\n",
1394
+ " predicted_ratings = model(user_ids, item_ids).numpy()\n",
1395
+ "\n",
1396
+ " # Urutkan item berdasarkan rating tertinggi\n",
1397
+ " top_n_items = items_to_predict[np.argsort(predicted_ratings)[-N:][::-1]]\n",
1398
+ "\n",
1399
+ " return top_n_items\n",
1400
+ "\n",
1401
+ "# Contoh penggunaan\n",
1402
+ "user_id = 0\n",
1403
+ "top_n_recommendations = get_top_n_recommendations_pytorch(model, user_id, N=10)\n",
1404
+ "print(f\"Top 10 recommended items for user {user_id}: {top_n_recommendations}\")"
1405
+ ],
1406
+ "metadata": {
1407
+ "colab": {
1408
+ "base_uri": "https://localhost:8080/"
1409
+ },
1410
+ "id": "04UHnGBtI8CB",
1411
+ "outputId": "93f34309-174b-41bd-af2c-f8a353efe5ad"
1412
+ },
1413
+ "execution_count": 195,
1414
+ "outputs": [
1415
+ {
1416
+ "output_type": "stream",
1417
+ "name": "stdout",
1418
+ "text": [
1419
+ "Top 10 recommended items for user 0: [1151 916 718 1134 832 941 327 347 631 434]\n"
1420
+ ]
1421
+ }
1422
+ ]
1423
+ },
1424
+ {
1425
+ "cell_type": "markdown",
1426
+ "source": [
1427
+ "## Export the datasets\n"
1428
+ ],
1429
+ "metadata": {
1430
+ "id": "PEfUd5FtKQS8"
1431
+ }
1432
+ },
1433
+ {
1434
+ "cell_type": "code",
1435
+ "source": [
1436
+ "ratings.to_csv(f\"ratings.csv\", index=False)\n",
1437
+ "movies.to_csv(f\"movies.csv\", index=False)\n",
1438
+ "users.to_csv(f\"users.csv\", index=False)"
1439
+ ],
1440
+ "metadata": {
1441
+ "id": "tVZFC_urL4Yd"
1442
+ },
1443
+ "execution_count": 196,
1444
+ "outputs": []
1445
+ }
1446
+ ]
1447
+ }