README.md

---
library_name: skops
license: mit
pipeline_tag: tabular-regression
tags:
- scikit-learn
- tabular-regression
- engagement-prediction
- viralst
- poc
model-index:
- name: raviearjun/engagement-predictor
  results:
  - task:
      type: tabular-regression
    dataset:
      name: viralst-engagement-synthetic-poc
      type: synthetic-poc
    metrics:
    - type: mean_absolute_error
      value: 0.01142
    - type: root_mean_squared_error
      value: 0.01458
    - type: r_squared
      value: 0.71023
---

# engagement-predictor

A regression model (`scikit-learn` `Ridge` inside a `Pipeline`) that predicts
`engagement_score` for brand-creator campaign content, part of the **Viralst**
application.

## Overview

Given campaign brief attributes (tone, hook type, target audience, creator
tier, objective, etc. — typically extracted from a brief document via an LLM
parser) and planned content execution details (duration, upload time,
caption), the model predicts the expected `engagement_score`
(`(likes + comments + shares) / views`).

It is served through the `POST /engagement/predict` endpoint of the Viralst
`ai/` service.

## Training data and methodology

The model is trained on a **synthetically generated dataset** (2,000 rows,
300 unique briefs, 30 unique brands) produced by
`training/generate_poc_data.py`. Feature-to-target relationships were
deliberately encoded (with variable noise per row) so the training pipeline
— feature engineering, model selection, evaluation, and serving — could be
validated end-to-end ahead of onboarding real campaign data. Results below
characterize how well the model recovers the encoded synthetic relationships,
not real-world campaign performance.

## Training procedure

- Model: `sklearn.linear_model.Ridge` inside a `Pipeline` (`OneHotEncoder`
  for single-value categorical features, `StandardScaler` for numeric and
  multi-hot features).
- Alpha selected via grid search using `GroupKFold` (grouped by `brief_id`),
  so evaluation reflects generalization to unseen briefs rather than
  leaking the same brief across train/test folds.
- Serialized with `skops.io.dump` (not raw pickle) for safer loading.

## Evaluation results

```
{
  "alpha": 10.0,
  "n_rows": 2000,
  "n_groups": 300,
  "cv_mae": 0.011419824566734465,
  "cv_rmse": 0.01458130687031906,
  "cv_r2": 0.7102257580449983,
  "baseline_mae": 0.021489796999999998,
  "baseline_rmse": 0.027087352583401354
}
```

Mean-predictor baseline RMSE: `0.027087352583401354`. The final model outperforms
this baseline by a wide margin on the synthetic dataset described above.

## Feature schema

See `config.json` in this repo for the full list and order of feature
columns the model expects. The canonical definition lives in
`src/engagement_predictor/features.py` in the application repository
(https://github.com/<your-org>/viralst).

## Training dataset

The raw synthetic dataset used to train this model is included in this repo
under `dataset/` (`brands.csv`, `briefs.csv`, `contents.csv` — 30
brands, 300 briefs, 2000 content rows). It was generated by
`training/generate_poc_data.py`; see "Training data and methodology" above
for why it is synthetic and what it does and does not represent.

## How to use

```python
import pandas as pd
import skops.io as sio
from huggingface_hub import hf_hub_download

path = hf_hub_download(repo_id="raviearjun/engagement-predictor", filename="engagement_ridge.skops")
untrusted = sio.get_untrusted_types(file=path)
model = sio.load(path, trusted=untrusted)

# X must be a DataFrame with columns matching config.json -> feature_columns
prediction = model.predict(X)
```

## Limitations

- Trained on synthetic data; does not reflect real-world engagement
  distributions, platform trends, or actual creator/audience behavior.
- Feature quality depends on upstream LLM extraction from brief documents;
  prediction quality degrades if that extraction is inaccurate or incomplete.
- Not yet validated against real campaign data from brands outside the
  training set.

## Reproducibility

Training environment (see `requirements.txt` in this repo for exact pins):

```
scikit-learn==1.7.2
skops==0.14.0
numpy==1.26.4
pandas==2.3.3
```