|
|
import streamlit as st |
|
|
|
|
|
|
|
|
sidebar = st.sidebar |
|
|
|
|
|
|
|
|
sidebar.header('π NLP Exploration') |
|
|
|
|
|
|
|
|
sidebar_option = sidebar.radio('Choose a section to explore:', ['π§ What is NLP?', 'π NLP Lifecycle', 'βοΈ NLP Techniques']) |
|
|
|
|
|
|
|
|
if 'selected_page' not in st.session_state: |
|
|
st.session_state.selected_page = sidebar_option |
|
|
|
|
|
|
|
|
if sidebar_option != st.session_state.selected_page: |
|
|
st.session_state.selected_page = sidebar_option |
|
|
|
|
|
|
|
|
def set_title(title, color="black"): |
|
|
st.markdown(f"<h1 style='text-align: center; color: {color}; font-size: 36px;'>{title}</h1>", unsafe_allow_html=True) |
|
|
|
|
|
|
|
|
def add_paragraph(content): |
|
|
st.markdown(f"<p style='font-size: 16px; line-height: 1.6;'>{content}</p>", unsafe_allow_html=True) |
|
|
|
|
|
|
|
|
def add_bullet_list(items): |
|
|
for item in items: |
|
|
st.markdown(f"- {item}", unsafe_allow_html=True) |
|
|
|
|
|
|
|
|
if st.session_state.selected_page == 'π§ What is NLP?': |
|
|
set_title('Introduction to Natural Language Processing (NLP)', color="darkblue") |
|
|
|
|
|
st.markdown("<h2 style='text-align: center; color: darkblue;'>π What is NLP?</h2>", unsafe_allow_html=True) |
|
|
|
|
|
add_paragraph(""" |
|
|
Natural Language Processing (NLP) is a branch of Artificial Intelligence (AI) that focuses on enabling computers to understand, interpret, and generate human language in a way that is meaningful. NLP allows machines to process and analyze large amounts of unstructured natural language data, including text and speech. |
|
|
""") |
|
|
|
|
|
st.markdown("<h3 style='text-align: center; color: darkgreen;'>Key Concepts of NLP</h3>", unsafe_allow_html=True) |
|
|
|
|
|
add_bullet_list([ |
|
|
"π **Syntax**: The arrangement of words in a sentence to ensure proper structure.", |
|
|
"π¬ **Semantics**: The study of meaning in words and sentences.", |
|
|
"π **Pragmatics**: How context influences the interpretation of language.", |
|
|
"π **Discourse**: The way that meaning is influenced by the flow of conversation or written text." |
|
|
]) |
|
|
|
|
|
st.markdown("<h3 style='text-align: center; color: darkgreen;'>Applications of NLP</h3>", unsafe_allow_html=True) |
|
|
|
|
|
add_bullet_list([ |
|
|
"π **Machine Translation**: Translating text between languages, such as Google Translate.", |
|
|
"π£οΈ **Speech Recognition**: Converting spoken language into text, as seen in Siri or Google Assistant.", |
|
|
"π **Sentiment Analysis**: Analyzing text to determine its sentiment, such as positive or negative reviews.", |
|
|
"βοΈ **Text Summarization**: Creating concise summaries from longer texts, such as news articles." |
|
|
]) |
|
|
|
|
|
add_paragraph(""" |
|
|
NLP is used in a wide variety of industries like healthcare, finance, and customer service to automate tasks, analyze data, and improve customer experiences. By enabling computers to process human language, NLP opens up new possibilities for innovation and efficiency. |
|
|
""") |
|
|
|
|
|
|
|
|
elif st.session_state.selected_page == "π NLP Lifecycle": |
|
|
lifecycle_option = sidebar.radio("Select NLP Lifecycle Step:", [ |
|
|
"π Overview of the NLP Life Cycle", |
|
|
"π― Problem Definition", |
|
|
"π Data Collection", |
|
|
"π Simple EDA", |
|
|
"π§Ή Data Preprocessing", |
|
|
"π Feature Engineering", |
|
|
"ποΈββοΈ Model Training", |
|
|
"π
Evaluation", |
|
|
"π Deployment" |
|
|
]) |
|
|
|
|
|
if lifecycle_option == "π Overview of the NLP Life Cycle": |
|
|
set_title('NLP Life Cycle Overview', color="darkblue") |
|
|
|
|
|
add_paragraph(""" |
|
|
The NLP lifecycle is a comprehensive process that involves several key stages, from defining the problem to deploying a model and maintaining it. Each step plays a crucial role in transforming raw text data into actionable insights or automated systems. The stages can be broken down as follows: |
|
|
""") |
|
|
|
|
|
add_bullet_list([ |
|
|
"1. **Problem Definition**: Identifying the business problem that can be solved using NLP.", |
|
|
"2. **Data Collection**: Gathering relevant textual data that will be used to train models.", |
|
|
"3. **Exploratory Data Analysis (EDA)**: Understanding the data's structure, identifying patterns, and spotting potential issues.", |
|
|
"4. **Data Preprocessing**: Cleaning and formatting data to prepare it for analysis, such as removing noise or handling missing values.", |
|
|
"5. **Feature Engineering**: Creating new features or selecting important attributes to improve model performance.", |
|
|
"6. **Model Training**: Selecting the appropriate algorithms and training models using the prepared data.", |
|
|
"7. **Evaluation**: Assessing the model's accuracy and performance using various metrics like precision, recall, and F1-score.", |
|
|
"8. **Deployment**: Implementing the model into a production environment, ready for real-world use.", |
|
|
"9. **Monitoring and Maintenance**: Continuously checking model performance and making updates to keep it effective." |
|
|
]) |
|
|
|
|
|
add_paragraph(""" |
|
|
The NLP lifecycle is iterative; as new data becomes available or requirements change, steps like model retraining and maintenance are necessary. This ensures that NLP systems remain relevant and effective over time. |
|
|
""") |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
elif st.session_state.selected_page == 'βοΈ NLP Techniques': |
|
|
set_title('Techniques in Natural Language Processing', color="darkblue") |
|
|
|
|
|
st.markdown("<h2 style='text-align: center; color: darkblue;'>π§ Key NLP Techniques</h2>", unsafe_allow_html=True) |
|
|
|
|
|
add_paragraph(""" |
|
|
NLP involves several techniques and algorithms to process and analyze text data effectively. Some of the key techniques include: |
|
|
""") |
|
|
|
|
|
add_bullet_list([ |
|
|
"π§ **Tokenization**: Breaking text into smaller units like words or sentences to make it easier to process.", |
|
|
"π€ **Stemming and Lemmatization**: Reducing words to their base or root form to standardize text (e.g., 'running' to 'run').", |
|
|
"π― **Named Entity Recognition (NER)**: Identifying entities like names, dates, or locations within text.", |
|
|
"π‘ **Part-of-Speech Tagging**: Assigning a part of speech (e.g., noun, verb) to each word in a sentence.", |
|
|
"π **Word Embeddings**: Mapping words into numerical vectors (e.g., Word2Vec, GloVe) to capture semantic meaning.", |
|
|
"π **Topic Modeling**: Grouping a set of documents into topics based on common themes, using techniques like LDA (Latent Dirichlet Allocation)." |
|
|
]) |
|
|
|
|
|
add_paragraph(""" |
|
|
These techniques are essential for tasks like machine translation, sentiment analysis, and chatbot development. NLP models that leverage these techniques can understand and generate human language with remarkable accuracy. |
|
|
""") |
|
|
import streamlit as st |
|
|
|
|
|
|
|
|
sidebar = st.sidebar |
|
|
|
|
|
|
|
|
sidebar.header('π NLP Navigation') |
|
|
|
|
|
|
|
|
sidebar_option = sidebar.radio('Choose a section to explore:', ['π§ What is NLP?', 'πNLP Lifecycle', 'βοΈNLP Techniques']) |
|
|
|
|
|
|
|
|
if 'selected_page' not in st.session_state: |
|
|
st.session_state.selected_page = sidebar_option |
|
|
|
|
|
|
|
|
if sidebar_option != st.session_state.selected_page: |
|
|
st.session_state.selected_page = sidebar_option |
|
|
|
|
|
|
|
|
def set_title(title, color="black"): |
|
|
st.markdown(f"<h1 style='text-align: center; color: {color};'>{title}</h1>", unsafe_allow_html=True) |
|
|
|
|
|
if st.session_state.selected_page == 'π§ What is NLP?': |
|
|
set_title('Understanding Natural Language Processing (NLP)', color="darkgreen") |
|
|
|
|
|
elif st.session_state.selected_page == 'πNLP Lifecycle': |
|
|
set_title('NLP Lifecycle: From Data Collection to Deployment', color="darkgreen") |
|
|
|
|
|
elif st.session_state.selected_page == 'βοΈNLP Techniques': |
|
|
set_title('NLP Techniques: Methods and Applications', color="darkgreen") |
|
|
|
|
|
|
|
|
if st.session_state.selected_page == 'π§ What is NLP?': |
|
|
st.markdown("<h2 style='text-align: center; color: darkgreen;'>π Introduction to NLP</h2>", unsafe_allow_html=True) |
|
|
st.write(""" |
|
|
#### π€ What is NLP? |
|
|
Natural Language Processing (NLP) is a subfield of Artificial Intelligence (AI) focused on the interaction between computers and human languages. Its goal is to enable machines to understand, interpret, and generate human language meaningfully. |
|
|
|
|
|
NLP allows machines to analyze vast amounts of natural language data, such as: |
|
|
- π Text from documents |
|
|
- π£οΈ Speech from conversations |
|
|
- πΌοΈ Images with textual descriptions |
|
|
|
|
|
#### Key Components of NLP: |
|
|
- **Syntax**: The structure of sentences and the arrangement of words. |
|
|
- **Semantics**: The meaning of words and how they are used. |
|
|
- **Pragmatics**: Understanding language in context, including the speaker's intent. |
|
|
- **Discourse**: Analyzing the structure and flow of sentences in a larger context. |
|
|
|
|
|
#### Applications of NLP: |
|
|
- **Machine Translation**: Translating text from one language to another (e.g., Google Translate). |
|
|
- **Speech Recognition**: Converting spoken words into written text (e.g., Siri, Alexa). |
|
|
- **Sentiment Analysis**: Determining the sentiment behind a text (positive, negative, or neutral). |
|
|
- **Text Summarization**: Automatically generating a concise version of a text (e.g., summarizing articles). |
|
|
|
|
|
NLP has significant applications in various fields like healthcare, finance, customer service, and many others. |
|
|
""") |
|
|
|
|
|
|
|
|
elif st.session_state.selected_page == "πNLP Lifecycle": |
|
|
lifecycle_option = sidebar.radio("Select NLP Lifecycle Step:", [ |
|
|
"π Overview of the NLP Life Cycle", |
|
|
"π― Problem Definition", |
|
|
"π Data Collection", |
|
|
"π Simple EDA", |
|
|
"π§Ή Data Preprocessing", |
|
|
"π Feature Engineering", |
|
|
"ποΈββοΈ Model Training", |
|
|
"π
Model Evaluation", |
|
|
"π Deployment" |
|
|
]) |
|
|
|
|
|
if lifecycle_option == "π Overview of the NLP Life Cycle": |
|
|
st.write(""" |
|
|
#### Overview of the NLP Life Cycle: |
|
|
The NLP life cycle outlines a structured approach to building, deploying, and maintaining systems that handle human language. It turns unstructured text into actionable insights, making the process flexible and adaptive to real-world changes. |
|
|
|
|
|
- **The Process Flow**: |
|
|
1. Problem identification and text data collection. |
|
|
2. Data cleaning and preprocessing. |
|
|
3. Model building and testing. |
|
|
4. Continuous monitoring and updates to ensure optimal performance. |
|
|
|
|
|
- **Key Features**: |
|
|
- Flexible and adaptive: The NLP life cycle evolves as language, trends, and data change. |
|
|
- Cross-disciplinary: Combines expertise from linguistics, computer science, and data analysis. |
|
|
- Practical and scalable: Solutions are designed to be implemented in real-world applications. |
|
|
|
|
|
#### Major Stages in the NLP Life Cycle: |
|
|
1. π§ Problem Definition |
|
|
2. π Data Collection |
|
|
3. π Simple EDA (Exploratory Data Analysis) |
|
|
4. π§Ή Data Preprocessing |
|
|
5. π Feature Engineering |
|
|
6. ποΈββοΈ Model Training |
|
|
7. π
Model Evaluation |
|
|
8. βοΈ Model Tuning |
|
|
9. π Deployment |
|
|
10. π οΈ Monitoring and Maintenance |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π― Problem Definition": |
|
|
st.write(""" |
|
|
#### π§ 1. Problem Definition: |
|
|
The first step in the NLP lifecycle is defining the problem. Understanding the objective is key to determining how NLP can be applied. |
|
|
|
|
|
- **Key Questions to Define the Problem**: |
|
|
- π― What is the goal of this analysis (e.g., sentiment classification, topic modeling)? |
|
|
- π What type of data are we dealing with (e.g., reviews, articles, social media posts)? |
|
|
- π What is the desired outcome (e.g., sentiment score, summary, or label)? |
|
|
|
|
|
**Example Problem**: Classifying customer reviews as positive or negative. |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π Data Collection": |
|
|
st.write(""" |
|
|
#### π 2. Data Collection: |
|
|
Data collection is the foundation of the NLP process. It involves gathering relevant textual data from multiple sources. |
|
|
|
|
|
- **Common Data Collection Methods**: |
|
|
- π Public datasets (e.g., Kaggle, government websites). |
|
|
- π APIs for data retrieval (e.g., Twitter API for tweets). |
|
|
- πΈοΈ Web scraping (e.g., using BeautifulSoup or Scrapy). |
|
|
- β Manual data entry in some cases. |
|
|
|
|
|
**Example**: Scraping product reviews from e-commerce websites to analyze customer sentiment. |
|
|
|
|
|
#### Extracting Data from Files: |
|
|
Once data is collected, itβs often in formats like CSV, JSON, or Excel. Tools like **Pandas** in Python can be used to extract and structure this data for analysis. |
|
|
|
|
|
**Steps**: |
|
|
- Identify the file format (e.g., `.csv`, `.json`). |
|
|
- Use functions like `pd.read_csv()` to load the data into a Pandas DataFrame. |
|
|
- Clean and verify the data using methods like `df.head()` or `df.info()` to inspect the first few records and data types. |
|
|
|
|
|
**Code Example**: |
|
|
```python |
|
|
import pandas as pd |
|
|
|
|
|
# Extracting Data from a CSV File |
|
|
df = pd.read_csv('data.csv') |
|
|
print(df.head()) |
|
|
``` |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π Simple EDA": |
|
|
st.write(""" |
|
|
#### π 3. Simple EDA (Exploratory Data Analysis): |
|
|
Simple EDA provides an initial understanding of the data. It helps identify patterns, outliers, and potential issues that may impact the analysis. |
|
|
|
|
|
- **Key Tasks in EDA**: |
|
|
- Check the data balance: Is the data evenly distributed or skewed? |
|
|
- Visualize distributions: Use histograms and boxplots. |
|
|
- Check for missing data: Identify any gaps in the data. |
|
|
- Outlier detection: Spot extreme values and decide whether to keep or remove them. |
|
|
|
|
|
**Example**: In a classification dataset, check for a 70:30 imbalance between positive and negative classes. |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π§Ή Data Preprocessing": |
|
|
st.write(""" |
|
|
#### π§Ή 4. Text Preprocessing: |
|
|
Text preprocessing is essential for converting raw text into a format suitable for analysis or model training. |
|
|
|
|
|
**Steps in Text Preprocessing**: |
|
|
- **Tokenization**: Splitting text into smaller pieces (words or sentences). |
|
|
- **Removing Stopwords**: Eliminate common, less meaningful words (e.g., 'the', 'and'). |
|
|
- **Stemming and Lemmatization**: Reduce words to their root forms (e.g., 'running' to 'run'). |
|
|
- **Removing Special Characters**: Strip out unnecessary symbols, emojis, and URLs. |
|
|
|
|
|
**Code Example**: |
|
|
```python |
|
|
import re |
|
|
|
|
|
# Sample text |
|
|
text = "The quick brown fox π¦ jumps over the lazy dog!" |
|
|
|
|
|
# Remove special characters |
|
|
clean_text = re.sub(r'[^a-zA-Z\s]', '', text) |
|
|
print(clean_text) |
|
|
``` |
|
|
**Output**: "The quick brown fox jumps over the lazy dog" |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π Feature Engineering": |
|
|
st.write(""" |
|
|
#### π 5. Feature Engineering: |
|
|
After preprocessing, the text must be transformed into a numerical representation for machine learning models. |
|
|
|
|
|
**Common Feature Engineering Techniques**: |
|
|
- **Bag of Words (BoW)**: Counts the frequency of words in the text. |
|
|
- **TF-IDF**: Measures word importance based on frequency and rarity. |
|
|
- **Word Embeddings**: Transforms words into dense vectors (e.g., Word2Vec, GloVe). |
|
|
|
|
|
**Example**: Converting the sentence "I love NLP" into a numerical vector using BoW. |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "ποΈββοΈ Model Training": |
|
|
st.write(""" |
|
|
#### ποΈββοΈ 6. Model Training: |
|
|
Once the data is preprocessed, we can train machine learning models on it. The choice of model depends on the NLP task (e.g., classification, sentiment analysis). |
|
|
|
|
|
**Common Models**: |
|
|
- **Text Classification**: Naive Bayes, SVM, or deep learning models. |
|
|
- **Sentiment Analysis**: Logistic Regression, Naive Bayes, BERT. |
|
|
|
|
|
**Example**: Training a Naive Bayes model to classify text into categories like "Sports", "Politics", etc. |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π
Model Evaluation": |
|
|
st.write(""" |
|
|
#### π
7. Model Evaluation: |
|
|
Evaluating the performance of your model is crucial to ensure its reliability. |
|
|
|
|
|
- **Metrics**: Accuracy, Precision, Recall, F1-Score. |
|
|
- **Confusion Matrix**: Visual representation of true positives, false positives, etc. |
|
|
|
|
|
**Example**: Evaluating a sentiment analysis model with accuracy and F1-score on test data. |
|
|
""") |
|
|
|
|
|
elif lifecycle_option == "π Deployment": |
|
|
st.write(""" |
|
|
#### π 8. Deployment: |
|
|
Once the model is trained and evaluated, it is deployed to production for real-time use. |
|
|
|
|
|
**Key Steps**: |
|
|
- **Integration**: Connecting the model with a live application (e.g., a chatbot). |
|
|
- **Monitoring**: Regularly tracking the modelβs performance over time. |
|
|
- **Updating**: Periodically retraining the model with new data. |
|
|
|
|
|
**Example**: Deploying a sentiment analysis model via a REST API to evaluate customer reviews in real-time. |
|
|
""") |
|
|
|
|
|
|
|
|
elif st.session_state.selected_page == "βοΈNLP Techniques": |
|
|
st.write(""" |
|
|
#### βοΈ Common NLP Techniques: |
|
|
There are several techniques in NLP that enable computers to process language in meaningful ways. |
|
|
|
|
|
**1. Tokenization**: Breaking text into individual words or phrases. |
|
|
|
|
|
**2. Named Entity Recognition (NER)**: Identifying entities such as names, locations, dates, etc. |
|
|
|
|
|
**3. Part-of-Speech (POS) Tagging**: Labeling words based on their parts of speech (e.g., noun, verb). |
|
|
|
|
|
**4. Sentiment Analysis**: Determining the sentiment expressed in the text (positive, negative, or neutral). |
|
|
|
|
|
**5. Word Embeddings**: Converting words into high-dimensional vectors to capture semantic meaning. |
|
|
|
|
|
**6. Text Classification**: Assigning predefined labels to text (e.g., spam detection, topic categorization). |
|
|
|
|
|
**7. Machine Translation**: Translating text from one language to another. |
|
|
|
|
|
**8. Text Summarization**: Condensing long texts into shorter, meaningful summaries. |
|
|
""") |
|
|
|
|
|
|
|
|
elif st.session_state.selected_page == "βοΈNLP Techniques": |
|
|
technique_option = sidebar.radio("Select an NLP Technique:", [ |
|
|
"NLP Techniques Overview", |
|
|
"Tokenization", |
|
|
"Stop Words Removal", |
|
|
"Lemmatization", |
|
|
"Stemming", |
|
|
"One-Hot Encoding", |
|
|
"Bag of Words (BoW)", |
|
|
"TF-IDF", |
|
|
"Word Embeddings", |
|
|
"Named Entity Recognition (NER)", |
|
|
"Part-of-Speech (POS) Tagging", |
|
|
"Sentiment Analysis" |
|
|
]) |
|
|
|
|
|
if technique_option == "NLP Techniques Overview": |
|
|
st.write(""" |
|
|
### βοΈ Key NLP Techniques |
|
|
Natural Language Processing (NLP) involves a variety of techniques for processing and analyzing text data. Here are some of the most important methods: |
|
|
|
|
|
1. **Tokenization**: Breaking text into smaller units such as words or sentences. |
|
|
2. **Part-of-Speech (POS) Tagging**: Identifying the grammatical role of words in a sentence (e.g., noun, verb, adjective). |
|
|
3. **Named Entity Recognition (NER)**: Identifying and classifying entities like names, dates, and locations. |
|
|
4. **Dependency Parsing**: Analyzing the syntactic structure of sentences. |
|
|
5. **Sentiment Analysis**: Determining the sentiment expressed in the text (positive, negative, or neutral). |
|
|
6. **Word Embeddings**: Representing words as vectors in continuous space to capture semantic relationships (e.g., Word2Vec, GloVe). |
|
|
|
|
|
**Example**: Sentiment analysis can identify whether customer reviews are positive, negative, or neutral based on the words used in the text. |
|
|
""") |
|
|
|
|
|
elif technique_option == "Tokenization": |
|
|
st.write(""" |
|
|
#### 1. Tokenization |
|
|
Tokenization is the process of breaking text into smaller units, known as tokens. Tokens can be words, sentences, or subwords. This step is essential for most NLP tasks, as it provides the foundation for further analysis. |
|
|
|
|
|
- **Example**: |
|
|
- Sentence: "Natural Language Processing is amazing!" |
|
|
- Tokenized words: `["Natural", "Language", "Processing", "is", "amazing"]` |
|
|
""") |
|
|
|
|
|
elif technique_option == "Stop Words Removal": |
|
|
st.write(""" |
|
|
#### 2. Stop Words Removal |
|
|
Stop words are common words (like 'the', 'and', 'in', etc.) that often do not carry much significant meaning for text analysis. Removing them helps reduce the dimensionality of the text and the noise in the dataset. |
|
|
|
|
|
- **Example**: In the sentence "NLP is fun!", the stop word "is" may be removed, leaving "NLP" and "fun" for analysis. |
|
|
""") |
|
|
|
|
|
elif technique_option == "Lemmatization": |
|
|
st.write(""" |
|
|
#### 3. Lemmatization |
|
|
Lemmatization is the process of reducing a word to its base or root form based on its meaning. It considers the context of the word in a sentence and ensures that the resulting word is a valid word in the language. |
|
|
|
|
|
- **Example**: |
|
|
- "Better" β "Good" |
|
|
- "Running" β "Run" |
|
|
""") |
|
|
|
|
|
elif technique_option == "Stemming": |
|
|
st.write(""" |
|
|
#### 4. Stemming |
|
|
Stemming is similar to lemmatization but is a simpler, rule-based approach. It involves chopping off prefixes or suffixes from words to reduce them to a root form, which may not always be a valid word. |
|
|
|
|
|
- **Example**: |
|
|
- "Running" β "Run" |
|
|
- "Happiness" β "Happi" |
|
|
""") |
|
|
|
|
|
elif technique_option == "One-Hot Encoding": |
|
|
st.write(""" |
|
|
#### 5. One-Hot Encoding |
|
|
One-Hot Encoding is a technique where each word in the vocabulary is represented as a binary vector. Each word is assigned a unique index, and the corresponding vector has a 1 at that index and 0 elsewhere. |
|
|
|
|
|
- **Example**: |
|
|
- Vocabulary: `["cat", "dog", "fish"]` |
|
|
- Encoding for "cat": `[1, 0, 0]` |
|
|
- Encoding for "dog": `[0, 1, 0]` |
|
|
|
|
|
**Pros**: Simple and easy to implement. |
|
|
**Cons**: Results in sparse vectors and can create high-dimensional spaces when the vocabulary is large. |
|
|
""") |
|
|
|
|
|
elif technique_option == "Bag of Words (BoW)": |
|
|
st.write(""" |
|
|
#### 6. Bag of Words (BoW) |
|
|
The Bag of Words model represents text as an unordered collection of words, without considering grammar or word order. This technique is commonly used for text classification and representation. |
|
|
|
|
|
- **Example**: |
|
|
- Text: "I love NLP" |
|
|
- BoW: `{"I": 1, "love": 1, "NLP": 1}` |
|
|
|
|
|
This model simply counts the frequency of each word in the text. |
|
|
""") |
|
|
|
|
|
elif technique_option == "TF-IDF": |
|
|
st.write(""" |
|
|
#### 7. TF-IDF (Term Frequency-Inverse Document Frequency) |
|
|
TF-IDF is a statistical measure used to evaluate the importance of a word within a document, considering its frequency in that document and its inverse frequency across a collection of documents. |
|
|
|
|
|
- **Example**: |
|
|
- In a document about data analysis, the word "data" might have a high TF-IDF score, while it would have a low score in a document about cooking. |
|
|
""") |
|
|
|
|
|
elif technique_option == "Word Embeddings": |
|
|
st.write(""" |
|
|
#### 8. Word Embeddings |
|
|
Word embeddings represent words in a continuous vector space, capturing semantic meanings. Words with similar meanings have similar vector representations. Popular word embedding models include Word2Vec, GloVe, and FastText. |
|
|
|
|
|
- **Example**: The words "king" and "queen" are represented as vectors that are close to each other in the vector space due to their semantic similarity. |
|
|
""") |
|
|
|
|
|
elif technique_option == "Named Entity Recognition (NER)": |
|
|
st.write(""" |
|
|
#### 9. Named Entity Recognition (NER) |
|
|
NER involves identifying and classifying named entities in text, such as people, organizations, locations, and dates. This is often used for information extraction from unstructured text. |
|
|
|
|
|
- **Example**: |
|
|
- Text: "Barack Obama was born in Hawaii." |
|
|
- Extracted Entities: `["Barack Obama" (Person), "Hawaii" (Location)]` |
|
|
""") |
|
|
|
|
|
elif technique_option == "Part-of-Speech (POS) Tagging": |
|
|
st.write(""" |
|
|
#### 10. Part-of-Speech (POS) Tagging |
|
|
POS tagging assigns grammatical labels to words in a sentence, such as noun (NN), verb (VB), or adjective (JJ). This helps understand the syntactic structure of the sentence. |
|
|
|
|
|
- **Example**: |
|
|
- Sentence: "The cat sat on the mat." |
|
|
- POS Tags: `[("The", "DT"), ("cat", "NN"), ("sat", "VBD"), ("on", "IN"), ("the", "DT"), ("mat", "NN")]` |
|
|
""") |
|
|
|
|
|
elif technique_option == "Sentiment Analysis": |
|
|
st.write(""" |
|
|
#### 11. Sentiment Analysis |
|
|
Sentiment analysis involves determining the sentiment of a text, often classifying it as positive, negative, or neutral. It is commonly used for analyzing customer feedback, social media posts, and product reviews. |
|
|
|
|
|
- **Example**: |
|
|
- Text: "I love this product!" |
|
|
- Sentiment: Positive |
|
|
|
|
|
Sentiment analysis helps businesses and organizations understand public perception and customer satisfaction. |
|
|
""") |
|
|
|
|
|
|