{"cells": [{"cell_type": "markdown", "metadata": {"id": "4ba6aba8"}, "source": ["# \ud83e\udd16 **Data Collection, Creation, Storage, and Processing**\n"]}, {"cell_type": "markdown", "metadata": {"id": "jpASMyIQMaAq"}, "source": ["## **1.** \ud83d\udce6 Install required packages"]}, {"cell_type": "code", "execution_count": 1, "metadata": {"colab": {"base_uri": "https://localhost:8080/"}, "id": "f48c8f8c", "outputId": "13d0dd5e-82c6-489f-b1f0-e970186a4eb7"}, "outputs": [{"output_type": "stream", "name": "stdout", "text": ["Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.12/dist-packages (4.13.5)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (2.2.2)\n", "Requirement already satisfied: matplotlib in /usr/local/lib/python3.12/dist-packages (3.10.0)\n", "Requirement already satisfied: seaborn in /usr/local/lib/python3.12/dist-packages (0.13.2)\n", "Requirement already satisfied: numpy in /usr/local/lib/python3.12/dist-packages (2.0.2)\n", "Requirement already satisfied: textblob in /usr/local/lib/python3.12/dist-packages (0.19.0)\n", "Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (2.8.3)\n", "Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.12/dist-packages (from beautifulsoup4) (4.15.0)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas) (2.9.0.post0)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.2)\n", "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas) (2025.3)\n", "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.3.3)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (0.12.1)\n", "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (4.61.1)\n", "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (1.4.9)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (26.0)\n", "Requirement already satisfied: pillow>=8 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (11.3.0)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.12/dist-packages (from matplotlib) (3.3.2)\n", "Requirement already satisfied: nltk>=3.9 in /usr/local/lib/python3.12/dist-packages (from textblob) (3.9.1)\n", "Requirement already satisfied: click in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (8.3.1)\n", "Requirement already satisfied: joblib in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (1.5.3)\n", "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (2025.11.3)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.12/dist-packages (from nltk>=3.9->textblob) (4.67.3)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)\n"]}], "source": ["!pip install beautifulsoup4 pandas matplotlib seaborn numpy textblob"]}, {"cell_type": "markdown", "metadata": {"id": "lquNYCbfL9IM"}, "source": ["## **2.** \u26cf Web-scrape all book titles, prices, and ratings from books.toscrape.com"]}, {"cell_type": "markdown", "metadata": {"id": "0IWuNpxxYDJF"}, "source": ["### *a. Initial setup*\n", "Define the base url of the website you will scrape as well as how and what you will scrape"]}, {"cell_type": "code", "execution_count": 2, "metadata": {"id": "91d52125"}, "outputs": [], "source": ["import requests\n", "from bs4 import BeautifulSoup\n", "import pandas as pd\n", "import time\n", "\n", "base_url = \"https://books.toscrape.com/catalogue/page-{}.html\"\n", "headers = {\"User-Agent\": \"Mozilla/5.0\"}\n", "\n", "titles, prices, ratings = [], [], []"]}, {"cell_type": "markdown", "metadata": {"id": "oCdTsin2Yfp3"}, "source": ["### *b. Fill titles, prices, and ratings from the web pages*"]}, {"cell_type": "code", "execution_count": 3, "metadata": {"id": "xqO5Y3dnYhxt"}, "outputs": [], "source": ["# Loop through all 50 pages\n", "for page in range(1, 51):\n", " url = base_url.format(page)\n", " response = requests.get(url, headers=headers)\n", " soup = BeautifulSoup(response.content, \"html.parser\")\n", " books = soup.find_all(\"article\", class_=\"product_pod\")\n", "\n", " for book in books:\n", " titles.append(book.h3.a[\"title\"])\n", " prices.append(float(book.find(\"p\", class_=\"price_color\").text[1:]))\n", " ratings.append(book.p.get(\"class\")[1])\n", "\n", " time.sleep(0.5) # polite scraping delay"]}, {"cell_type": "markdown", "metadata": {"id": "T0TOeRC4Yrnn"}, "source": ["### *c. \u270b\ud83c\udffb\ud83d\uded1\u26d4\ufe0f Create a dataframe df_books that contains the now complete \"title\", \"price\", and \"rating\" objects*"]}, {"cell_type": "code", "execution_count": 4, "metadata": {"id": "l5FkkNhUYTHh"}, "outputs": [], "source": []}, {"cell_type": "markdown", "metadata": {"id": "duI5dv3CZYvF"}, "source": ["### *d. Save web-scraped dataframe either as a CSV or Excel file*"]}, {"cell_type": "code", "execution_count": 5, "metadata": {"id": "lC1U_YHtZifh"}, "outputs": [], "source": ["# \ud83d\udcbe Save to CSV\n", "df_books.to_csv(\"books_data.csv\", index=False)\n", "\n", "# \ud83d\udcbe Or save to Excel\n", "# df_books.to_excel(\"books_data.xlsx\", index=False)"]}, {"cell_type": "markdown", "metadata": {"id": "qMjRKMBQZlJi"}, "source": ["### *e. \u270b\ud83c\udffb\ud83d\uded1\u26d4\ufe0f View first fiew lines*"]}, {"cell_type": "code", "execution_count": 6, "metadata": {"colab": {"base_uri": "https://localhost:8080/", "height": 206}, "id": "O_wIvTxYZqCK", "outputId": "349b36b0-c008-4fd5-d4a4-dba38ae18337"}, "outputs": [{"output_type": "execute_result", "data": {"text/plain": [" title price rating\n", "0 A Light in the Attic 51.77 Three\n", "1 Tipping the Velvet 53.74 One\n", "2 Soumission 50.10 One\n", "3 Sharp Objects 47.82 Four\n", "4 Sapiens: A Brief History of Humankind 54.23 Five"], "text/html": ["\n", "
| \n", " | title | \n", "price | \n", "rating | \n", "
|---|---|---|---|
| 0 | \n", "A Light in the Attic | \n", "51.77 | \n", "Three | \n", "
| 1 | \n", "Tipping the Velvet | \n", "53.74 | \n", "One | \n", "
| 2 | \n", "Soumission | \n", "50.10 | \n", "One | \n", "
| 3 | \n", "Sharp Objects | \n", "47.82 | \n", "Four | \n", "
| 4 | \n", "Sapiens: A Brief History of Humankind | \n", "54.23 | \n", "Five | \n", "