README.md

---
title: Retail Food Freshness Classifier
emoji: 🥗
colorFrom: green
colorTo: blue
sdk: streamlit
sdk_version: 1.47.0
app_file: src/campus_triage/app.py
pinned: false
---
# Campus Support Message Triage Assistant

Campus Support Message Triage Assistant is a complete NLP module project that classifies synthetic student support messages by support category and urgency. The system is designed to help a university support office route messages faster while keeping human review in the loop.

## Novelty Statement

This repository uses a new synthetic dataset created specifically for this project. It does not reuse prior coursework, public student support datasets, or real student records.

## Data Source

The dataset is generated by `src/campus_triage/data.py`. It creates at least 1,500 realistic but synthetic messages with `message_id`, `message_text`, `category`, `urgency`, `channel`, `student_type`, and `created_hour`. The generator intentionally includes typos, informal language, short requests, longer emails, urgent wording, and ambiguous messages.

Saved files:

- `data/raw/campus_support_messages.csv`
- `data/processed/train.csv`
- `data/processed/val.csv`
- `data/processed/test.csv`

## Categories and Urgency Labels

Categories: `financial_aid`, `registration`, `housing`, `academic_advising`, `technical_support`, `health_wellness`, `general`

Urgency: `low`, `medium`, `high`

## Setup

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .
```

The Makefile also sets `PYTHONPATH=src`, so the main workflow works without editable install.

## Generate Data

```bash
make data
```

Optional custom size:

```bash
PYTHONPATH=src python scripts/make_dataset.py --rows 1800
```

## Modeling Strategies

All three required modeling strategies are implemented in the repository.

| Strategy | What it does | Where it is implemented | Training artifact | Command |
| --- | --- | --- | --- | --- |
| A. Naive baseline | Majority-class classifier for category and urgency | `build_baseline_model()` in `src/campus_triage/models.py`; trained by `train_baseline()` in `src/campus_triage/train.py` | `models/majority_baseline.joblib` | `make train` |
| B. Classical ML | TF-IDF vectorizer plus Logistic Regression, with separate category and urgency classifiers | `build_text_pipeline()` and `build_classical_model()` in `src/campus_triage/models.py`; trained by `train_classical()` in `src/campus_triage/train.py` | `models/tfidf_logistic_regression.joblib` | `make train` |
| C. Deep learning | DistilBERT sequence classification, with separate category and urgency transformer models | `src/campus_triage/transformer_training.py`; loaded for evaluation by `TransformerTextClassifier` and `load_transformer_dual_classifier()` in `src/campus_triage/models.py` | `models/transformer/category/` and `models/transformer/urgency/` | `make train-transformer` |

The deployed app uses the TF-IDF Logistic Regression model by default because it is fast, small, free-deployment friendly, and appropriate for a student-laptop proof of concept. The deep learning implementation is included and can be trained when hardware/time allow.

## Train Models

Train the baseline and classical models:

```bash
make train
```

Train the optional DistilBERT deep learning model:

```bash
make train-transformer
```

Equivalent direct command:

```bash
PYTHONPATH=src python scripts/train_all_models.py --include-transformer
```

`make train-transformer` downloads `distilbert-base-uncased` from Hugging Face if it is not already cached, so it requires internet access the first time.

## Run Evaluation

```bash
make evaluate
```

Evaluation always includes the baseline and classical models. If trained transformer checkpoints exist in `models/transformer/category/` and `models/transformer/urgency/`, evaluation automatically includes the deep learning model as `distilbert_transformer`.

Outputs:

- `data/outputs/model_comparison.csv`
- `data/outputs/category_confusion_matrix.png`
- `data/outputs/urgency_confusion_matrix.png`
- `data/outputs/classification_reports.txt`
- `data/outputs/error_analysis.csv`

Macro F1 is important because routing classes may be imbalanced. Accuracy can look strong while hiding poor performance on smaller but important classes such as `health_wellness` or `high` urgency. Macro F1 gives each class equal weight, which makes minority urgent cases visible.

## Robustness Experiment

```bash
make experiment
```

The experiment creates a noisy test condition with random character deletion, random typos, lowercasing, extra punctuation, and missing punctuation. It compares clean versus noisy performance for the classical model. If transformer checkpoints exist, it also compares the deep learning model on clean versus noisy test data.

Outputs:

- `data/outputs/robustness_experiment.csv`
- `data/outputs/robustness_plot.png`

Interpretation guidance: transformer models often have better semantic robustness than sparse TF-IDF models, but the project conclusion should be based on the saved clean/noisy macro F1 values after `make train-transformer` is run.

## Launch the App

```bash
make app
```

Equivalent:

```bash
streamlit run main.py
```

The app runs inference only. It lets a user paste a student message, returns category and urgency predictions, shows confidence scores, recommends a routing action, provides a keyword or confidence explanation, includes example messages, and displays proof-of-concept limitations.

## Repository Structure

```text
README.md
requirements.txt
Makefile
setup.py
main.py
.gitignore
src/campus_triage/
  __init__.py
  config.py
  data.py
  features.py
  models.py
  train.py
  evaluate.py
  experiment.py
  predict.py
  app.py
  transformer_training.py
scripts/
  make_dataset.py
  train_all_models.py
  run_experiment.py
models/
data/
  raw/
  processed/
  outputs/
notebooks/
```

## Metrics

Category and urgency are evaluated with accuracy, macro F1, weighted F1, per-class precision, recall, F1, and confusion matrices.

## Hyperparameter Tuning

Classical model hyperparameters are in `build_text_pipeline()` in `src/campus_triage/models.py`. Tune `ngram_range`, `min_df`, `max_features`, `sublinear_tf`, `max_iter`, `class_weight`, and `C`.

Transformer hyperparameters are in `src/campus_triage/transformer_training.py`. Tune `num_train_epochs`, `per_device_train_batch_size`, `per_device_eval_batch_size`, `learning_rate`, and tokenizer `max_length`.

After tuning, rerun:

```bash
make train
make evaluate
make experiment
```

## Error Analysis

`make evaluate` creates `data/outputs/error_analysis.csv` with five mispredictions from the best evaluated model, including message text, true labels, predicted labels, likely root cause, and a concrete mitigation strategy.


## Ethical Considerations

This system should assist, not replace, student support staff. Synthetic data cannot represent all student populations, dialects, disability contexts, crisis language, or institutional policies. High-urgency and health/wellness messages require conservative escalation and human review. Real deployment would require privacy review, bias testing, accessibility review, incident response procedures, and staff training.

## Limitations

- Synthetic training data may overstate real-world performance.
- Confidence scores are model probabilities, not guarantees.
- The app does not integrate with official student systems.
- Transformer training is optional and may need hardware beyond a small laptop.
- The model should not be used for disciplinary, medical, or emergency decisions without human review.

## Future Work

- Add institution-specific labeled examples after privacy review.
- Add calibrated confidence thresholds and manual review queues.
- Evaluate fairness across student type, channel, and language variety.
- Add multilingual message handling.
- Deploy with authentication and audit logging.
- Compare trained transformer checkpoints against the classical model on noisy data.

# External Code Attribution

This project includes code written by the author as well as AI-assisted code generation and publicly available libraries.

### AI Assistance

Portions of this project were developed with assistance from OpenAI ChatGPT (GPT-5.5).

OpenAI ChatGPT:
https://chatgpt.com/

AI assistance included (but was not limited to):

- project architecture suggestions
- code generation
- code refactoring
- debugging
- documentation
- README generation
- evaluation report drafting
- comments and explanations

All AI-generated code was reviewed, tested, and modified by the project author before submission.