README.md

metadata

title: Retail Food Freshness Classifier
emoji: 🥗
colorFrom: green
colorTo: blue
sdk: streamlit
sdk_version: 1.47.0
app_file: src/campus_triage/app.py
pinned: false

Campus Support Message Triage Assistant

Campus Support Message Triage Assistant is a complete NLP module project that classifies synthetic student support messages by support category and urgency. The system is designed to help a university support office route messages faster while keeping human review in the loop.

Novelty Statement

This repository uses a new synthetic dataset created specifically for this project. It does not reuse prior coursework, public student support datasets, or real student records.

Data Source

The dataset is generated by src/campus_triage/data.py. It creates at least 1,500 realistic but synthetic messages with message_id, message_text, category, urgency, channel, student_type, and created_hour. The generator intentionally includes typos, informal language, short requests, longer emails, urgent wording, and ambiguous messages.

Saved files:

• data/raw/campus_support_messages.csv
• data/processed/train.csv
• data/processed/val.csv
• data/processed/test.csv

Categories and Urgency Labels

Categories: financial_aid, registration, housing, academic_advising, technical_support, health_wellness, general

Urgency: low, medium, high

Setup

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .

The Makefile also sets PYTHONPATH=src, so the main workflow works without editable install.

Generate Data

make data

Optional custom size:

PYTHONPATH=src python scripts/make_dataset.py --rows 1800

Modeling Strategies

All three required modeling strategies are implemented in the repository.

Strategy	What it does	Where it is implemented	Training artifact	Command
A. Naive baseline	Majority-class classifier for category and urgency	build_baseline_model() in src/campus_triage/models.py; trained by train_baseline() in src/campus_triage/train.py	models/majority_baseline.joblib	make train
B. Classical ML	TF-IDF vectorizer plus Logistic Regression, with separate category and urgency classifiers	build_text_pipeline() and build_classical_model() in src/campus_triage/models.py; trained by train_classical() in src/campus_triage/train.py	models/tfidf_logistic_regression.joblib	make train
C. Deep learning	DistilBERT sequence classification, with separate category and urgency transformer models	src/campus_triage/transformer_training.py; loaded for evaluation by TransformerTextClassifier and load_transformer_dual_classifier() in src/campus_triage/models.py	models/transformer/category/ and models/transformer/urgency/	make train-transformer

The deployed app uses the TF-IDF Logistic Regression model by default because it is fast, small, free-deployment friendly, and appropriate for a student-laptop proof of concept. The deep learning implementation is included and can be trained when hardware/time allow.

Train Models

Train the baseline and classical models:

make train

Train the optional DistilBERT deep learning model:

make train-transformer

Equivalent direct command:

PYTHONPATH=src python scripts/train_all_models.py --include-transformer

make train-transformer downloads distilbert-base-uncased from Hugging Face if it is not already cached, so it requires internet access the first time.

Run Evaluation

make evaluate

Evaluation always includes the baseline and classical models. If trained transformer checkpoints exist in models/transformer/category/ and models/transformer/urgency/, evaluation automatically includes the deep learning model as distilbert_transformer.

Outputs:

• data/outputs/model_comparison.csv
• data/outputs/category_confusion_matrix.png
• data/outputs/urgency_confusion_matrix.png
• data/outputs/classification_reports.txt
• data/outputs/error_analysis.csv

Macro F1 is important because routing classes may be imbalanced. Accuracy can look strong while hiding poor performance on smaller but important classes such as health_wellness or high urgency. Macro F1 gives each class equal weight, which makes minority urgent cases visible.

Robustness Experiment

make experiment

The experiment creates a noisy test condition with random character deletion, random typos, lowercasing, extra punctuation, and missing punctuation. It compares clean versus noisy performance for the classical model. If transformer checkpoints exist, it also compares the deep learning model on clean versus noisy test data.

Outputs:

• data/outputs/robustness_experiment.csv
• data/outputs/robustness_plot.png

Interpretation guidance: transformer models often have better semantic robustness than sparse TF-IDF models, but the project conclusion should be based on the saved clean/noisy macro F1 values after make train-transformer is run.

Launch the App

make app

Equivalent:

streamlit run main.py

The app runs inference only. It lets a user paste a student message, returns category and urgency predictions, shows confidence scores, recommends a routing action, provides a keyword or confidence explanation, includes example messages, and displays proof-of-concept limitations.

Repository Structure

README.md
requirements.txt
Makefile
setup.py
main.py
.gitignore
src/campus_triage/
  __init__.py
  config.py
  data.py
  features.py
  models.py
  train.py
  evaluate.py
  experiment.py
  predict.py
  app.py
  transformer_training.py
scripts/
  make_dataset.py
  train_all_models.py
  run_experiment.py
models/
data/
  raw/
  processed/
  outputs/
notebooks/

Metrics

Category and urgency are evaluated with accuracy, macro F1, weighted F1, per-class precision, recall, F1, and confusion matrices.

Hyperparameter Tuning

Classical model hyperparameters are in build_text_pipeline() in src/campus_triage/models.py. Tune ngram_range, min_df, max_features, sublinear_tf, max_iter, class_weight, and C.

Transformer hyperparameters are in src/campus_triage/transformer_training.py. Tune num_train_epochs, per_device_train_batch_size, per_device_eval_batch_size, learning_rate, and tokenizer max_length.

After tuning, rerun:

make train
make evaluate
make experiment

Error Analysis

make evaluate creates data/outputs/error_analysis.csv with five mispredictions from the best evaluated model, including message text, true labels, predicted labels, likely root cause, and a concrete mitigation strategy.

Ethical Considerations

This system should assist, not replace, student support staff. Synthetic data cannot represent all student populations, dialects, disability contexts, crisis language, or institutional policies. High-urgency and health/wellness messages require conservative escalation and human review. Real deployment would require privacy review, bias testing, accessibility review, incident response procedures, and staff training.

Limitations

• Synthetic training data may overstate real-world performance.
• Confidence scores are model probabilities, not guarantees.
• The app does not integrate with official student systems.
• Transformer training is optional and may need hardware beyond a small laptop.
• The model should not be used for disciplinary, medical, or emergency decisions without human review.

Future Work

• Add institution-specific labeled examples after privacy review.
• Add calibrated confidence thresholds and manual review queues.
• Evaluate fairness across student type, channel, and language variety.
• Add multilingual message handling.
• Deploy with authentication and audit logging.
• Compare trained transformer checkpoints against the classical model on noisy data.

External Code Attribution

This project includes code written by the author as well as AI-assisted code generation and publicly available libraries.

AI Assistance

Portions of this project were developed with assistance from OpenAI ChatGPT (GPT-5.5).

OpenAI ChatGPT: https://chatgpt.com/

AI assistance included (but was not limited to):

• project architecture suggestions
• code generation
• code refactoring
• debugging
• documentation
• README generation
• evaluation report drafting
• comments and explanations

All AI-generated code was reviewed, tested, and modified by the project author before submission.