| # Fine-tuned BERT-base-uncased pre-trained model to classify spam SMS. | |
| My second project in Natural Language Processing (NLP), where I fine-tuned a bert-base-uncased model to classify spam SMS. This is huge improvements from https://github.com/fzn0x/bert-indonesian-english-hate-comments. | |
| ## ✅ Install requirements | |
| Install required dependencies | |
| ```sh | |
| pip install --upgrade pip | |
| pip install -r requirements.txt | |
| ``` | |
| ## ✅ Add BERT virtual env | |
| write the command below | |
| ```sh | |
| # ✅ Create and activate a virtual environment | |
| python -m venv bert-env | |
| source bert-env/bin/activate # On Windows use: bert-env\Scripts\activate | |
| ``` | |
| ## ✅ Install CUDA | |
| Check if your GPU supports CUDA: | |
| ```sh | |
| nvidia-smi | |
| ``` | |
| Then: | |
| ```sh | |
| pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 | |
| PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False | |
| ``` | |
| ## 🔧 How to use | |
| - Check your device and CUDA availability: | |
| ```sh | |
| python check_device.py | |
| ``` | |
| > :warning: Using CPU is not advisable, prefer check your CUDA availability. | |
| - Train the model: | |
| ```sh | |
| python scripts/train.py | |
| ``` | |
| > :warning: Remove unneeded checkpoint in models/pretrained to save your storage after training | |
| - Run prediction: | |
| ```sh | |
| python scripts/predict.py | |
| ``` | |
| ✅ Dataset Location: [`data/spam.csv`](./data/spam.csv), modify the dataset to enhance the model based on your needs. | |
| ## 📚 Citations | |
| If you use this repository or its ideas, please cite the following: | |
| See [`citations.bib`](./citations.bib) for full BibTeX entries. | |
| - Wolf et al., *Transformers: State-of-the-Art Natural Language Processing*, EMNLP 2020. [ACL Anthology](https://www.aclweb.org/anthology/2020.emnlp-demos.6) | |
| - Pedregosa et al., *Scikit-learn: Machine Learning in Python*, JMLR 2011. | |
| - Almeida & Gómez Hidalgo, *SMS Spam Collection v.1*, UCI Machine Learning Repository (2011). [Kaggle Link](https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset) | |
| ## 🧠 Credits and Libraries Used | |
| - [Hugging Face Transformers](https://github.com/huggingface/transformers) – model, tokenizer, and training utilities | |
| - [scikit-learn](https://scikit-learn.org/stable/) – metrics and preprocessing | |
| - Logging silencing inspired by Hugging Face GitHub discussions | |
| - Dataset from [UCI SMS Spam Collection](https://www.kaggle.com/datasets/uciml/sms-spam-collection-dataset) | |
| - Inspiration from [Kaggle Notebook by Suyash Khare](https://www.kaggle.com/code/suyashkhare/naive-bayes) | |
| ## License and Usage | |
| License under [MIT license](./LICENSE). | |
| --- | |
| Leave a ⭐ if you think this project is helpful, contributions are welcome. | |
| --- |