File size: 2,295 Bytes
339d442 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
language:
- pt
- multilingual
license: mit
tags:
- recommenders
- text-retrieval
- product-recommendation
- sentence-transformers
datasets:
- synthetic-ecommerce
metrics:
- auc
- ndcg
- mrr
---
# foundational-model
A semantic product recommendation model that matches user profiles (free text) to products. Uses a frozen multilingual MiniLM encoder with trainable projection heads and chunk attention for user encoding.
## Model description
- **Architecture**: Dual-encoder (user encoder + item encoder)
- **Base model**: [paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) (frozen)
- **Trainable params**: ~148k (projection head + chunk attention)
- **Input**: User profile text + product name + description
- **Output**: Cosine similarity scores for ranking
## Intended use
Product recommendation from user free-text profiles (e.g. "Marcos, gosto de videogames e de música, sou de Rio de janeiro"). Trained on synthetic e-commerce interactions in Portuguese.
## How to use
```python
from transformers import AutoTokenizer
import torch
from huggingface_hub import hf_hub_download
# Download checkpoint
checkpoint = hf_hub_download(repo_id="oristides/foundational-model", filename="pytorch_model.bin")
# Load model (requires model_arch1.RecSysModel - see repo for architecture)
from model.model_arch1 import RecSysModel
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")
model = RecSysModel()
model.load_state_dict(torch.load(checkpoint, map_location="cpu"))
model.eval()
# Encode user and items, then: scores = user_emb @ item_embs.T
```
Or use the `recommender` CLI in this repo: `uv run projects/reneguirecsys/model/recommender.py "your profile" -k 10`
## Training
- **Loss**: In-batch multi-negative cross-entropy
- **Split**: Leave-one-out per user
- **Eval metrics**: AUC, NDCG@10, MRR
- **Max sequence length**: 256 (user chunks), 128 (items)
## Citation
```bibtex
@misc{oristides-foundational-model-2025,
author = {oristides},
title = {Foundational Model for Product Recommendation},
year = {2025},
publisher = {Hugging Face},
url = {https://huggingface.co/oristides/foundational-model}
}
```
## License
MIT
|