| --- |
| |
| |
| {} |
| --- |
| |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
| This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
|
| ## Model Details |
| This model classifies news headlines as either NBC or Fox News. |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
| - **Developed by:** Jack Bader, Kaiyuan Wang, Pairan Xu |
| - **Taks:** Binary classification (NBC News vs. Fox News) |
| - **Preprocessing:** TF-IDF vectorization applied to the text data |
| - stop_words = "english" |
| - max_features = 1000 |
| - **Model type:** Random Forest |
| - **Freamwork:** Scikit-learn |
| - |
| #### Metrics |
|
|
| <!-- These are the evaluation metrics being used, ideally with a description of why. --> |
|
|
| - Accuracy Score |
|
|
| ### Model Evaluation |
| ```python |
| import pandas as pd |
| import joblib |
| from huggingface_hub import hf_hub_download |
| from sklearn.feature_extraction.text import TfidfVectorizer |
| from sklearn.metrics import classification_report |
| |
| # Mount to drive |
| from google.colab import drive |
| drive.mount('/content/drive') |
| |
| # Load test set |
| test_df = pd.read_csv("/content/drive/MyDrive/test_data_random_subset.csv", encoding="Windows-1252") |
| |
| # Log in w/ huggingface token |
| !huggingface-cli login |
| |
| # Download the model |
| model = hf_hub_download(repo_id = "CIS5190FinalProj/RandomForest", filename = "best_rf_model.pkl") |
| |
| # Download the vectorizer |
| tfidf_vectorizer = hf_hub_download(repo_id = "CIS5190FinalProj/RandomForest", filename = "tfidf_vectorizer.pkl") |
| |
| # Load the model |
| pipeline = joblib.load(model) |
| |
| # Load the vectorizer |
| tfidf_vectorizer = joblib.load(tfidf_vectorizer) |
| |
| # Extract the headlines from the test set |
| X_test = test_df['title'] |
| |
| # Apply transformation to the headlines into numerical features |
| X_test_transformed = tfidf_vectorizer.transform(X_test) |
| |
| # Make predictions using the pipeline |
| y_pred = pipeline.predict(X_test_transformed) |
| |
| # Extract 'labels' as target |
| y_test = test_df['label'] |
| |
| # Print classification report |
| print(classification_report(y_test, y_pred)) |
