README.md

---
license: mit
tags:
  - matrix-factorization
  - preference-model
  - recommendation
  - cloud-local
  - numpy
library_name: custom
pipeline_tag: text-classification
metrics:
  - accuracy: 0.677
  - AUC: 0.734
---

# Cloud vs Local — MF preference model

Simple Matrix Factorization model that predicts **which model to use for a prompt**:
`cloud` or `local`. Trained on preference data of the form *(user, prompt, chosen)*
where `chosen ∈ {cloud, local}`.

![MF pipeline diagram](mf_model_diagram.png)

## 🚨 Use real preference data for better results

This model is **trained on synthetic demo data** (12,000 generated ratings) — it exists
to validate the pipeline and the tooling, **not** to be the final product. For a model
that actually routes your traffic, retrain on your **real preference logs**:

1. Export a CSV with at least three columns: `user_id`, `prompt`, `chosen`
   (values `cloud` or `local`).
2. Open `mf_cloud_vs_local.ipynb` in Jupyter and run the last code cell:
   `train_on(load_preference_data("your_real_data.csv"))`.
3. The whole pipeline — EDA, stratified train/val/test split, MF training,
   evaluation, bundle export — runs unchanged on your file.

Real data gives you: trustworthy metrics, per-topic routing insights from your own
prompts, and a cleaner cold-start story (see Limitations).

## Model

`r̂(u,i) = μ + b_u + b_i + ⟨p_u, q_i⟩` — user bias + prompt bias + dot product of latent
factor vectors (k = 8), trained with binary cross-entropy + L2 regularization via
minibatch SGD (NumPy), early-stopped on validation AUC.

Decision rule: `p(cloud) = σ(r̂) ≥ 0.5 → cloud`, else `local`.

## Metrics (held-out test, n = 1,200 — synthetic data)

| metric    | value   | baseline (always cloud) |
|-----------|---------|--------------------------|
| accuracy  | 0.677   | 0.483                    |
| AUC       | 0.734   | 0.500                    |

The model recovers the latent topic structure: privacy-sensitive and
latency-sensitive prompts are routed to `local`; complex-reasoning and long-context
prompts to `cloud`.

## Files in this repo

- `README.md` — this model card
- `mf_cloud_vs_local.ipynb` — full source notebook (data gen, EDA, training, eval, bundle, diagram, inference demo)
- `mf_inference.py` — sample inference script (CLI, loads the bundle with `SimpleMF.from_pretrained`)
- `mf_model_diagram.png` — training + inference pipeline diagram
- `config.json` — hyperparameters + metrics
- `mf_params.npz` — `mu, bu, bi, P, Q, user_ids, prompt_ids` (weights + vocab)
- `test_predictions.csv` — per-pair predictions on the held-out test set

## Usage

```bash
pip install numpy datasets huggingface_hub

# score specific (user, prompt) pairs
python mf_inference.py --model_dir mf_bundle --user_id U0007 \
    --prompt_ids P0001,P0400,P0572 --csv preference_data_synthetic.csv

# rank all known prompts for a user
python mf_inference.py --model_dir mf_bundle --user_id U0007 --top_k 5 \
    --csv preference_data_synthetic.csv
```

Clone this repo and run `mf_inference.py` locally, or open `mf_cloud_vs_local.ipynb`
to retrain on your own preference data.

## Limitations

- **Cold start**: prompts not seen in training have no `q_i` embedding — the model
  cannot score brand-new prompt text. Extend with a text encoder (e.g. embed prompt
  text into the latent space) for production cold start.
- **Synthetic data**: metrics above are on synthetic ratings; expect different numbers
  on real preference logs — see "Use real preference data" above.
- No leakage checks were needed for synthetic data; run a near-duplicate check between
  train/val/test when real data arrives.