File size: 5,856 Bytes

---
library_name: transformers
tags:
- recommendation
- retrieval
- prag
- pytorch
license: mit
language:
- en
---

# PRAG

This is PRAG, a LLM  model trained for multi recommendation tasks and domains. It uses a two-tower architecture with a shared model to embed both user queries and product items into a common vector space for efficient retrieval.

## Model Details

### Model Description
<!-- Provide a longer summary of what this model is. -->
The PRAG model is designed for product recommendation and retrieval tasks. It maps text inputs (queries or item descriptions) to high-dimensional embeddings. The model is optimized using contrastive learning, where the goal is to maximize the cosine similarity between a query and its corresponding relevant item while minimizing similarity with irrelevant items.

- **Developed by:** @[dodo](https://huggingface.co/do2do2), @[quocdat32461997](https://huggingface.co/tendatngo)
- **Model type:** LLM
- **Language(s) (NLP):** English

### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** https://github.com/do2do2-ai
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

### Downstream Use [optional]
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
This model can be used directly for:
- Semantic search in e-commerce catalogs.
- Building recommendation systems by matching user intent (queries) to product descriptions.
- Zero-shot product retrieval tasks.

### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
- Non-English text (not explicitly trained/validated).
- General-purpose text embedding outside the e-commerce/recommendation domain.
- Tasks requiring generative capabilities.

## Bias, Risks, and Limitations

- **Domain Specificity:** Optimized for product data with similar features and characteristics, but performance may vary on different domains or datasets.
- **Language:** Limited to English.
- **Biases:** May inherit biases present in the Amazon Reviews 2023 dataset.

### Recommendations

Users should evaluate the model on their specific product domain before deployment. Consider fine-tuning if the target domain's vocabulary differs significantly from Amazon's.

## How to Get Started with the Model

You can use the model via the recommendation API. Below is an example of how to make an authenticated request to get product recommendations.

```python
import os
import requests
import json

# Configuration
api_key = "YOUR_API_KEY"
recommend_url = "https://trydodo.xyz" 

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}",
}

# Define context, template, and product catalog
payload = {
    "context": {
        "previous_purchases": ["electronics", "books"],
        "budget": 100.0,
    },
    "template": "Recommend next product to customer. Previous purchases: {previous_purchases}, Budget less than: {budget}",
    "catalog": {
        "product_1": "Wireless headphones - Premium noise-cancelling",
        "product_2": "Laptop stand - Adjustable aluminum ergonomic",
        "product_3": "Python book - Complete guide for beginners",
        "product_4": "Smartphone case - Shockproof cover with kickstand",
        "product_5": "USB-C hub - 7-in-1 adapter with 4K HDMI",
    },
}

try:
    response = requests.post(
        url=f"{recommend_url}/api/recommend/recommend",
        params={
            "model_key": "prag_v1",
            "num_results": 5,
        },
        headers=headers,
        json=payload,
    )
    response.raise_for_status()
    
    recommendations = response.json()
    print(f"Recommendations: {json.dumps(recommendations, indent=2)}")

except Exception as e:
    print(f"Request failed: {e}")
```

## Training Details

The model was trained following standard large language model (LLM) training practices for retrieval systems.

### Training Data

The model is trained on diverse product-related datasets, including product reviews, metadata, and user interaction logs. The data is preprocessed to emphasize semantic relationships between queries and items.

### Training Procedure

#### Preprocessing

- Standard tokenization using a LLM tokenizer.
- Input sequences are formatted to represent user intent (queries) and product characteristics (items).
- Dynamic padding and truncation are applied to optimize training efficiency.

#### Training Hyperparameters

The training follows casual LLM training practices:
- **Batch Size:** Scaled to maximize GPU utilization.
- **Loss Function:** Contrastive learning objective.
- **Temperature:** Tuned to balance retrieval precision and recall.

## Evaluation

### Testing Data, Factors & Metrics


### Results

| Models | Recall@5/10 (Toys and Games) | Recall@5/10 (All Beauty) | NDCG@5/10 (Toys and Games) | NDCG@5/10 (All Beauty) |
| :--- | :--- | :--- | :--- | :--- |
| PRAG-v1 | 0.0452/0.0921 | 0.0452/0.0921 | 0.0267/0.0414 | 0.0267/0.0414 |
| [TIGER](https://papers.neurips.cc/paper_files/paper/2023/file/20dcab0f14046a5c6b02b61da9f13229-Paper-Conference.pdf) | 0.0531/0.0712 | 0.0454/0.0648 | 0.0371/0.0432 | 0.0321/0.0384 |

## Technical Specifications

### Model Architecture and Objective

- **Backbone:** Standard LLM for text representation.
- **Objective:** Contrastive learning objective

### Software

- **Framework:** PyTorch
- **Library:** Transformers

## Model Card Authors

[quocdat32461997](https://huggingface.co/tendatngo)