BERTopic/notebooks/BERTopic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AXHLDxJdRzBi"
   },
   "source": [
    "# **BERTopic - Tutorial**\n",
    "We start with installing bertopic from pypi before preparing the data. \n",
    "\n",
    "**NOTE**: Make sure to select a GPU runtime. Otherwise, the model can take quite some time to create the document embeddings!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y3VGFZ1USMTu"
   },
   "source": [
    "# **Prepare data**\n",
    "For this example, we use the popular 20 Newsgroups dataset which contains roughly 18000 newsgroups posts on 20 topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JJij3WP6SEQD",
    "outputId": "6b4d3f7b-9f7f-426f-dea8-ab1e5083eb94"
   },
   "outputs": [],
   "source": [
    "from bertopic import BERTopic\n",
    "from sklearn.datasets import fetch_20newsgroups\n",
    "\n",
    "docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SBcNmZJzSTY8"
   },
   "source": [
    "# **Create Topics**\n",
    "We select the \"english\" as the main language for our documents. If you want a multilingual model that supports 50+ languages, please select \"multilingual\" instead. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TfhfzqkoSJ1I",
    "outputId": "d51e1f2c-d5db-44b6-d172-58881d54d8e6"
   },
   "outputs": [],
   "source": [
    "model = BERTopic(language=\"english\")\n",
    "topics, probs = model.fit_transform(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0ua80usww-rj"
   },
   "source": [
    "We can then extract most frequent topics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 202
    },
    "id": "nNptKBzHSbyS",
    "outputId": "25855ec5-d642-4864-cc64-404df61fabc6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Topic</th>\n",
       "      <th>Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1</td>\n",
       "      <td>6224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>586</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>480</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Topic  Count\n",
       "0     -1   6224\n",
       "1      0   1833\n",
       "2      1    586\n",
       "3      2    526\n",
       "4      3    480"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_topic_freq().head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-BtOgifV7Q-H"
   },
   "source": [
    "-1 refers to all outliers and should typically be ignored. Next, let's take a look at the most frequent topic that was generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UCw_q8I7Sb03",
    "outputId": "0076f8e0-3355-409f-f3f2-6e3478c6c1ba"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('game', 0.010436520264926533),\n",
       " ('team', 0.009084781216657336),\n",
       " ('games', 0.007234962878545154),\n",
       " ('he', 0.007131864903744603),\n",
       " ('players', 0.006351685439415695),\n",
       " ('season', 0.006282132628187317),\n",
       " ('hockey', 0.006157176094578879),\n",
       " ('play', 0.005817036121709545),\n",
       " ('25', 0.005687199271496157),\n",
       " ('year', 0.0056711166453823885)]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_topic(0)[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u_XSeuhs7bpK"
   },
   "source": [
    "Note that the model is stocastich which mmeans that the topics might differ across runs. \n",
    "\n",
    "For a full list of support languages, see the values below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "aY3ZkwLE7nvy",
    "outputId": "c913a643-5946-4b7d-9ce7-72c1ea7c7fd1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['afrikaans', 'albanian', 'amharic', 'arabic', 'armenian', 'assamese', 'azerbaijani', 'basque', 'belarusian', 'bengali', 'bengali romanize', 'bosnian', 'breton', 'bulgarian', 'burmese', 'burmese zawgyi font', 'catalan', 'chinese (simplified)', 'chinese (traditional)', 'croatian', 'czech', 'danish', 'dutch', 'english', 'esperanto', 'estonian', 'filipino', 'finnish', 'french', 'galician', 'georgian', 'german', 'greek', 'gujarati', 'hausa', 'hebrew', 'hindi', 'hindi romanize', 'hungarian', 'icelandic', 'indonesian', 'irish', 'italian', 'japanese', 'javanese', 'kannada', 'kazakh', 'khmer', 'korean', 'kurdish (kurmanji)', 'kyrgyz', 'lao', 'latin', 'latvian', 'lithuanian', 'macedonian', 'malagasy', 'malay', 'malayalam', 'marathi', 'mongolian', 'nepali', 'norwegian', 'oriya', 'oromo', 'pashto', 'persian', 'polish', 'portuguese', 'punjabi', 'romanian', 'russian', 'sanskrit', 'scottish gaelic', 'serbian', 'sindhi', 'sinhala', 'slovak', 'slovenian', 'somali', 'spanish', 'sundanese', 'swahili', 'swedish', 'tamil', 'tamil romanize', 'telugu', 'telugu romanize', 'thai', 'turkish', 'ukrainian', 'urdu', 'urdu romanize', 'uyghur', 'uzbek', 'vietnamese', 'welsh', 'western frisian', 'xhosa', 'yiddish']\n"
     ]
    }
   ],
   "source": [
    "from bertopic.backend import languages\n",
    "print(languages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Attributes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a number of attributes that you can access after having trained your BERTopic model:\n",
    "\n",
    "\n",
    "| Attribute | Description |\n",
    "|------------------------|---------------------------------------------------------------------------------------------|\n",
    "| topics_               | The topics that are generated for each document after training or updating the topic model. |\n",
    "| probabilities_ | The probabilities that are generated for each document if HDBSCAN is used. |\n",
    "| topic_sizes_           | The size of each topic                                                                      |\n",
    "| topic_mapper_          | A class for tracking topics and their mappings anytime they are merged/reduced.             |\n",
    "| topic_representations_ | The top *n* terms per topic and their respective c-TF-IDF values.                             |\n",
    "| c_tf_idf_              | The topic-term matrix as calculated through c-TF-IDF.                                       |\n",
    "| topic_labels_          | The default labels for each topic.                                                          |\n",
    "| custom_labels_         | Custom labels for each topic as generated through `.set_topic_labels`.                                                               |\n",
    "| topic_embeddings_      | The embeddings for each topic if `embedding_model` was used.                                                              |\n",
    "| representative_docs_   | The representative documents for each topic if HDBSCAN is used.                                                |\n",
    "\n",
    "For example, to access the predicted topics for the first 10 documents, we simply run the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, -1, -1, -1, 30, -1, -1, 0, 0, -1]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.topics_[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2eyImbal7lb8"
   },
   "source": [
    "# **Embedding model**\n",
    "You can select any model from `sentence-transformers` and use it instead of the preselected models by simply passing the model through  \n",
    "BERTopic with `embedding_model`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1dR2ckNK782p"
   },
   "outputs": [],
   "source": [
    "# st_model = BERTopic(embedding_model=\"xlm-r-bert-base-nli-stsb-mean-tokens\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SoMc1W-x7-b5"
   },
   "source": [
    "Click [here](https://www.sbert.net/docs/pretrained_models.html) for a list of supported sentence transformers models.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M8c8LenB8Zyl"
   },
   "source": [
    "# **Visualize Topics**\n",
    "After having trained our `BERTopic` model, we can iteratively go through perhaps a hundred topic to get a good \n",
    "understanding of the topics that were extract. However, that takes quite some time and lacks a global representation. \n",
    "Instead, we can visualize the topics that were generated in a way very similar to \n",
    "[LDAvis](https://github.com/cpsievert/LDAvis):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 667
    },
    "id": "AQKBcla28bY0",
    "outputId": "582cefe4-a82c-4915-f2e0-3ad307d3c723"
   },
   "outputs": [],
   "source": [
    "model.visualize_topics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9antKpdC91A-"
   },
   "source": [
    "# **Topic Reduction**\n",
    "Finally, we can also reduce the number of topics after having trained a BERTopic model. The advantage of doing so, \n",
    "is that you can decide the number of topics after knowing how many are actually created. It is difficult to \n",
    "predict before training your model how many topics that are in your documents and how many will be extracted. \n",
    "Instead, we can decide afterwards how many topics seems realistic:\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "5m4Nd7Us-Peg"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bertopic._bertopic.BERTopic at 0x2c1b7ba8fd0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.reduce_topics(docs, nr_topics=60)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c4m3UMsw-Zxk"
   },
   "source": [
    "# **Topic Representation**\n",
    "When you have trained a model and viewed the topics and the words that represent them,\n",
    "you might not be satisfied with the representation. Perhaps you forgot to remove\n",
    "stop_words or you want to try out a different n_gram_range. We can use the function `update_topics` to update \n",
    "the topic representation with new parameters for `c-TF-IDF`: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "KWm7B-FJ-iYW"
   },
   "outputs": [],
   "source": [
    "model.update_topics(docs, n_gram_range=(1, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LXYJ745O-03Z"
   },
   "source": [
    "# **Search Topics**\n",
    "After having trained our model, we can use `find_topics` to search for topics that are similar \n",
    "to an input search_term. Here, we are going to be searching for topics that closely relate the \n",
    "search term \"gpu\". Then, we extract the most similar topic and check the results: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lAdiVYej-2i-",
    "outputId": "29d69ab2-1e0d-4561-8cee-e43654e4479e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[18, 21, 24, 28, 36]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar_topics, similarity = model.find_topics(\"gpu\", top_n=5); similar_topics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "T-DaaqSA-2nH",
    "outputId": "99027647-f939-4f9d-9707-7e2f7dbb7b9a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('monitor', 0.02604954056000538),\n",
       " ('the', 0.011592478015427971),\n",
       " ('monitors', 0.011386430026081376),\n",
       " ('vga', 0.0104858000492717),\n",
       " ('video', 0.008950171344439876),\n",
       " ('with', 0.008770262226959034),\n",
       " ('to', 0.008627947931402424),\n",
       " ('it', 0.008211951060402312),\n",
       " ('and', 0.00815957039585335),\n",
       " ('horizontal', 0.007798501042289411)]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.get_topic(18)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wekNoQNuUVoU"
   },
   "source": [
    "# **Model serialization**\n",
    "The model and its internal settings can easily be saved. Note that the documents and embeddings will not be saved. However, UMAP and HDBSCAN will be saved. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nWUF1uxiSb_a"
   },
   "outputs": [],
   "source": [
    "# Save model\n",
    "model.save(\"my_model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "y_eHBI1jSb6i"
   },
   "outputs": [],
   "source": [
    "# Load model\n",
    "my_model = BERTopic.load(\"my_model\")"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "BERTopic.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "bertopic",
   "language": "python",
   "name": "bertopic"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}