---
language:
  - pt
  - multilingual
license: mit
tags:
  - recommenders
  - text-retrieval
  - product-recommendation
  - sentence-transformers
datasets:
  - synthetic-ecommerce
metrics:
  - auc
  - ndcg
  - mrr
---

# foundational-model

A semantic product recommendation model that matches user profiles (free text) to products. Uses a frozen multilingual MiniLM encoder with trainable projection heads and chunk attention for user encoding.

## Model description

- **Architecture**: Dual-encoder (user encoder + item encoder)
- **Base model**: [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) (frozen)
- **Trainable params**: ~148k (projection head + chunk attention)
- **Input**: User profile text + product name + description
- **Output**: Cosine similarity scores for ranking

## Intended use

Product recommendation from user free-text profiles (e.g. "Marcos, gosto de videogames e de música, sou de Rio de janeiro"). Trained on synthetic e-commerce interactions in Portuguese.

## How to use

```python
from transformers import AutoTokenizer
import torch
from huggingface_hub import hf_hub_download

# Download checkpoint
checkpoint = hf_hub_download(repo_id="oristides/foundational-model", filename="pytorch_model.bin")

# Load model (requires model_arch1.RecSysModel - see repo for architecture)
from model.model_arch1 import RecSysModel
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")
model = RecSysModel()
model.load_state_dict(torch.load(checkpoint, map_location="cpu"))
model.eval()

# Encode user and items, then: scores = user_emb @ item_embs.T
```

Or use the `recommender` CLI in this repo: `uv run projects/reneguirecsys/model/recommender.py "your profile" -k 10`

## Training

- **Loss**: In-batch multi-negative cross-entropy
- **Split**: Leave-one-out per user
- **Eval metrics**: AUC, NDCG@10, MRR
- **Max sequence length**: 256 (user chunks), 128 (items)

## Citation

```bibtex
@misc{oristides-foundational-model-2025,
  author = {oristides},
  title = {Foundational Model for Product Recommendation},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/oristides/foundational-model}
}
```

## License

MIT