Update README.md

README.md

@@ -1,100 +1,111 @@
-# AI vs Human Text Fine-Tuned Classifier
-
-## About the Project
-
-This project aims to develop a robust machine learning model capable of distinguishing between human-written and AI-generated text. With the rapid advancement of large language models (LLMs) such as ChatGPT and Gemini, the ability to identify the origin of a text has become crucial in various domains, including academic integrity, content moderation, misinformation detection, and authorship verification. The project leverages state-of-the-art natural language processing (NLP) techniques and transfer learning to build a binary classifier that can accurately predict whether a given text was authored by a human or generated by an AI.
-
-The workflow encompasses comprehensive exploratory data analysis (EDA), advanced text preprocessing, model selection and fine-tuning, and thorough evaluation. The final model is designed to be easily deployable and accessible for real-world applications.
-
-## About the Dataset
-
-The dataset used in this project is sourced from Kaggle: [AI vs Human Text Dataset](https://www.kaggle.com/datasets/shanegerami/ai-vs-human-text). It contains a large collection of text samples, each labeled as either human-written or AI-generated. The dataset is well-suited for binary classification tasks and provides a diverse range of topics and writing styles, making it ideal for training and evaluating models that need to generalize across different types of content.
-
-- **Features:**
-  - `text`: The actual text sample.
-  - `generated`: Label indicating the source (0 for human, 1 for AI).
-
-The dataset is split into training, validation, and test sets to ensure unbiased evaluation and robust model performance.
-
-## Notebook Summary
-
-The main notebook, `ai_vs_human_text_fine_tuned_classifier.ipynb`, guides users through the entire process of building the classifier:
-
-1. **Problem Definition:** Outlines the motivation and objectives.
-2. **Exploratory Data Analysis (EDA):** Visualizes class distributions, text lengths, lexical richness, punctuation usage, and stopword ratios to uncover patterns and differences between human and AI texts.
-3. **Text Preprocessing:** Applies normalization, stopword removal, noise filtering (removing URLs, emails, hashtags, mentions, numbers), and filters out outlier texts based on length.
-4. **Model Selection:** Utilizes transfer learning with the `distilbert/distilroberta-base` model, enhanced with LoRA (Low-Rank Adaptation) for efficient fine-tuning.
-5. **Training:** Fine-tunes the model on a subset of the data, using stratified splits and advanced training arguments for optimal performance.
-6. **Evaluation:** Assesses the model using accuracy, precision, recall, and F1-score on a held-out test set.
-7. **Deployment:** Demonstrates how to push the trained model and tokenizer to Hugging Face Hub for sharing and reuse.
-
-## Model Results
-
-### Preprocessing
-
-- **Lowercasing and Stripping:** All text is converted to lowercase and stripped of extra whitespace.
-- **Punctuation and Stopword Removal:** Punctuation is removed, and stopwords are filtered out to focus on meaningful content.
-- **Noise Filtering:** Regular expressions are used to remove URLs, emails, hashtags, mentions, and numbers.
-- **Outlier Filtering:** Texts that are extremely short or long (based on quantiles) are removed to ensure consistent input lengths for the model.
-- **Deduplication:** Duplicate texts are dropped to prevent data leakage.
-
-### Training
-
-- **Model Architecture:** The project uses `distilbert/distilroberta-base`, a distilled version of RoBERTa, known for its efficiency and strong performance on text classification tasks.
-- **LoRA Fine-Tuning:** LoRA (Low-Rank Adaptation) is applied to reduce the number of trainable parameters, making the fine-tuning process more memory- and compute-efficient without sacrificing accuracy.
-- **Training Arguments:** The model is trained for 2 epochs with early stopping, regular evaluation, and checkpointing. Batch sizes and learning rates are carefully chosen for stability and speed.
-
-### Evaluation
-
-- **Metrics:** The model is evaluated using accuracy, precision, recall, and F1-score. These metrics provide a comprehensive view of the classifier's performance, especially in distinguishing between the two classes.
-- **Results:** The fine-tuned model demonstrates strong performance, with high accuracy and balanced precision/recall, indicating its effectiveness in real-world scenarios.
-
-## How to Install
-
-Follow these steps to set up the environment using Python's built-in `venv`:
-
-```bash
-# Clone the repository
-git clone https://github.com/DeepActionPotential/FineTextTector
-cd FineTextTector
-
-# Create a virtual environment
-python -m venv venv
-
-# Activate the virtual environment
-# On Windows:
-venv\Scripts\activate
-# On macOS/Linux:
-source venv/bin/activate
-
-
-# Install required packages
-pip install -r requirements.txt
-```
-
-
-
-## How to Use the Software
-
- - ## [Demo-video](images/FineTextTector.mp4)
- - ![Demo-image](images/1.png)
- - ![Demo-image](images/2.png)
-
-## Technologies Used
-
-- **Transformers (Hugging Face):** Core library for model loading, tokenization, and training. Enables transfer learning with state-of-the-art NLP models.
-- **Datasets (Hugging Face):** Efficient data handling, splitting, and preprocessing.
-- **PEFT (Parameter-Efficient Fine-Tuning):** Implements LoRA for memory- and compute-efficient model adaptation.
-- **Optuna:** Automated hyperparameter optimization to fine-tune model performance.
-- **Scikit-learn:** Data splitting, metrics calculation, and utility functions.
-- **Seaborn & Matplotlib:** Data visualization for EDA and result interpretation.
-- **NLTK:** Stopword lists and basic NLP utilities.
-- **Python venv:** Isolated environment management for reproducible installations.
-
-These technologies collectively enable efficient, scalable, and reproducible development of advanced NLP models.
-
-## License
-
-This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
-
----
+
+---
+title: FineTextTector - AI Text Detector
+emoji: 🤖
+colorFrom: indigo
+colorTo: blue
+sdk: streamlit
+sdk_version: 1.30.0
+app_file: app.py
+pinned: false
+license: mit
+---
+
+## About the Project
+
+This project aims to develop a robust machine learning model capable of distinguishing between human-written and AI-generated text. With the rapid advancement of large language models (LLMs) such as ChatGPT and Gemini, the ability to identify the origin of a text has become crucial in various domains, including academic integrity, content moderation, misinformation detection, and authorship verification. The project leverages state-of-the-art natural language processing (NLP) techniques and transfer learning to build a binary classifier that can accurately predict whether a given text was authored by a human or generated by an AI.
+
+The workflow encompasses comprehensive exploratory data analysis (EDA), advanced text preprocessing, model selection and fine-tuning, and thorough evaluation. The final model is designed to be easily deployable and accessible for real-world applications.
+
+## About the Dataset
+
+The dataset used in this project is sourced from Kaggle: [AI vs Human Text Dataset](https://www.kaggle.com/datasets/shanegerami/ai-vs-human-text). It contains a large collection of text samples, each labeled as either human-written or AI-generated. The dataset is well-suited for binary classification tasks and provides a diverse range of topics and writing styles, making it ideal for training and evaluating models that need to generalize across different types of content.
+
+- **Features:**
+  - `text`: The actual text sample.
+  - `generated`: Label indicating the source (0 for human, 1 for AI).
+
+The dataset is split into training, validation, and test sets to ensure unbiased evaluation and robust model performance.
+
+## Notebook Summary
+
+The main notebook, `ai_vs_human_text_fine_tuned_classifier.ipynb`, guides users through the entire process of building the classifier:
+
+1. **Problem Definition:** Outlines the motivation and objectives.
+2. **Exploratory Data Analysis (EDA):** Visualizes class distributions, text lengths, lexical richness, punctuation usage, and stopword ratios to uncover patterns and differences between human and AI texts.
+3. **Text Preprocessing:** Applies normalization, stopword removal, noise filtering (removing URLs, emails, hashtags, mentions, numbers), and filters out outlier texts based on length.
+4. **Model Selection:** Utilizes transfer learning with the `distilbert/distilroberta-base` model, enhanced with LoRA (Low-Rank Adaptation) for efficient fine-tuning.
+5. **Training:** Fine-tunes the model on a subset of the data, using stratified splits and advanced training arguments for optimal performance.
+6. **Evaluation:** Assesses the model using accuracy, precision, recall, and F1-score on a held-out test set.
+7. **Deployment:** Demonstrates how to push the trained model and tokenizer to Hugging Face Hub for sharing and reuse.
+
+## Model Results
+
+### Preprocessing
+
+- **Lowercasing and Stripping:** All text is converted to lowercase and stripped of extra whitespace.
+- **Punctuation and Stopword Removal:** Punctuation is removed, and stopwords are filtered out to focus on meaningful content.
+- **Noise Filtering:** Regular expressions are used to remove URLs, emails, hashtags, mentions, and numbers.
+- **Outlier Filtering:** Texts that are extremely short or long (based on quantiles) are removed to ensure consistent input lengths for the model.
+- **Deduplication:** Duplicate texts are dropped to prevent data leakage.
+
+### Training
+
+- **Model Architecture:** The project uses `distilbert/distilroberta-base`, a distilled version of RoBERTa, known for its efficiency and strong performance on text classification tasks.
+- **LoRA Fine-Tuning:** LoRA (Low-Rank Adaptation) is applied to reduce the number of trainable parameters, making the fine-tuning process more memory- and compute-efficient without sacrificing accuracy.
+- **Training Arguments:** The model is trained for 2 epochs with early stopping, regular evaluation, and checkpointing. Batch sizes and learning rates are carefully chosen for stability and speed.
+
+### Evaluation
+
+- **Metrics:** The model is evaluated using accuracy, precision, recall, and F1-score. These metrics provide a comprehensive view of the classifier's performance, especially in distinguishing between the two classes.
+- **Results:** The fine-tuned model demonstrates strong performance, with high accuracy and balanced precision/recall, indicating its effectiveness in real-world scenarios.
+
+## How to Install
+
+Follow these steps to set up the environment using Python's built-in `venv`:
+
+```bash
+# Clone the repository
+git clone https://github.com/DeepActionPotential/FineTextTector
+cd FineTextTector
+
+# Create a virtual environment
+python -m venv venv
+
+# Activate the virtual environment
+# On Windows:
+venv\Scripts\activate
+# On macOS/Linux:
+source venv/bin/activate
+
+
+# Install required packages
+pip install -r requirements.txt
+```
+
+
+
+## How to Use the Software
+
+ - ## [Demo-video](images/FineTextTector.mp4)
+ - ![Demo-image](images/1.png)
+ - ![Demo-image](images/2.png)
+
+## Technologies Used
+
+- **Transformers (Hugging Face):** Core library for model loading, tokenization, and training. Enables transfer learning with state-of-the-art NLP models.
+- **Datasets (Hugging Face):** Efficient data handling, splitting, and preprocessing.
+- **PEFT (Parameter-Efficient Fine-Tuning):** Implements LoRA for memory- and compute-efficient model adaptation.
+- **Optuna:** Automated hyperparameter optimization to fine-tune model performance.
+- **Scikit-learn:** Data splitting, metrics calculation, and utility functions.
+- **Seaborn & Matplotlib:** Data visualization for EDA and result interpretation.
+- **NLTK:** Stopword lists and basic NLP utilities.
+- **Python venv:** Isolated environment management for reproducible installations.
+
+These technologies collectively enable efficient, scalable, and reproducible development of advanced NLP models.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+---