| --- |
| library_name: transformers |
| tags: |
| - recommendation |
| - e-commerce |
| - sequential-modeling |
| - next-item-prediction |
| - behavioral-modeling |
| --- |
| |
| # Commerce Intent |
|
|
| ## Model Overview |
|
|
| Commerce Intent is a pretrained sequential behavioral model for e-commerce session understanding. It is trained to predict the next item in a user session based on historical interaction sequences. |
|
|
| The model learns representations from multi-modal structured signals, including: |
|
|
| - Item ID |
| - Brand |
| - Category |
| - Event type (view, cart and purchase) |
| - Normalized price |
| - Positional order within the session |
|
|
| It is designed as a foundation model for downstream recommendation and behavioral modeling tasks. |
|
|
| --- |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| Commerce Intent models user behavior within a session as an autoregressive sequence modeling problem. Given a sequence of past interactions, the model predicts the next likely item. |
|
|
| The architecture consists of: |
|
|
| - Multi-embedding token fusion (item, brand, category, event) |
| - Continuous price projection |
| - Positional encoding |
| - Transformer encoder with causal masking |
| - Linear head for next-item prediction |
|
|
| This model is pretrained and can be fine-tuned for recommendation, ranking, or conversion modeling tasks. |
|
|
| - **Developed by:** infinity6 |
| - **Model type:** Sequential autoregressive transformer |
| - **Language(s):** Structured e-commerce interaction data (non-NLP) |
| - **License:** Apache 2.0 |
| - **Finetuned from model:** None (trained from scratch) |
|
|
| --- |
|
|
| ## Dependencies |
|
|
| This model depends on the external package: |
|
|
| - https://github.com/infinity6-ai/i6model_ecomm |
| |
| The package contains the custom architecture required to correctly load and run the model. You must install it before using Commerce Intent. |
| |
| ### Installation |
| |
| Clone the repository: |
| |
| ```bash |
| git clone https://github.com/infinity6-ai/i6model_ecomm.git |
| cd i6model_ecomm |
| pip install . |
| ``` |
| |
| --- |
| |
| ## Intended Use |
| |
| ### Direct Use |
| |
| The model can be used directly for: |
| |
| - Next-item prediction |
| - Session-based recommendation |
| - Behavioral embedding extraction |
| - Purchase intent modeling |
| - Real-time ranking systems |
| |
| Example: |
| |
| ```python |
| import torch |
| |
| from i6modelecomm.model import i6modelecomm |
| |
| model = i6modelecomm.CommerceIntent.from_pretrained( |
| "infinity6/ecomm_shop_intent_pretrained" |
| ) |
| |
| # TODO: map items and remap categories. |
|
|
| # TODO: freeze layers and train with your data. |
|
|
| model.eval() |
|
|
| D = 'cpu' |
|
|
| # batch_size | seq_len = 3 |
| itms = torch.tensor([[12, 45, 78]], dtype=torch.long).to(D) |
| brds = torch.tensor([[3, 7, 2]], dtype=torch.long).to(D) |
| cats = torch.tensor([[8, 8, 15]], dtype=torch.long).to(D) |
| prcs = torch.tensor([[29.9, 35.0, 15.5]], dtype=torch.float).to(D) |
| evts = torch.tensor([[1, 1, 2]], dtype=torch.long).to(D) |
|
|
| # mask |
| mask = torch.tensor([[1, 1, 1]], dtype=torch.bool).to(D) |
|
|
| with torch.no_grad(): |
| outputs = model( |
| itms=itms, # items |
| brds=brds, # brands |
| cats=cats, # categories |
| prcs=prcs, # prices |
| evts=evts, # events |
| attention_mask=mask, |
| labels=None # inference only -- no loss computation |
| ) |
| |
| # logits tem shape (B, L-1, num_itm) |
| logits = outputs.logits |
| |
| print("Logits shape:", logits.shape) |
| ``` |
| |
| Inputs must include: |
| |
| - `itms` |
| - `brds` |
| - `cats` |
| - `evts` |
| - `prcs` |
| - `attention_mask` |
|
|
| --- |
|
|
| ### Downstream Use |
|
|
| The model can be fine-tuned for: |
|
|
| - Conversion prediction |
| - Cart abandonment modeling |
| - Customer lifetime value modeling |
| - Cross-sell / upsell recommendation |
| - Personalized search ranking |
|
|
| --- |
|
|
| ### Out-of-Scope Use |
|
|
| This model is not suitable for: |
|
|
| - Natural language tasks |
| - Image tasks |
| - Generative text modeling |
| - Multi-user graph modeling without adaptation |
| - Cold-start scenarios without item mappings and category remapping |
|
|
| --- |
|
|
| ## Bias, Risks, and Limitations |
|
|
| - The model reflects behavioral biases present in historical e-commerce data. |
| - Popularity bias may emerge due to item frequency distribution. |
| - Model performance depends on session length and interaction quality. |
| - Cold-start performance for unseen items is limited. |
| - It does not encode demographic or identity-aware fairness constraints. |
|
|
| ### Recommendations |
|
|
| - Monitor recommendation fairness and popularity skew. |
| - Retrain periodically to reflect new item distributions. |
| - Apply business constraints in production systems. |
| - Use A/B testing before large-scale deployment. |
|
|
| --- |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| The model was trained on large-scale anonymized e-commerce interaction logs containing: |
|
|
| - Session-based user interactions |
| - Item identifiers |
| - Brand identifiers |
| - Category identifiers |
| - Event types |
| - Timestamped behavioral sequences |
| - Price values (log-normalized and standardized) |
|
|
| Sessions shorter than a minimum threshold were filtered. |
|
|
| --- |
|
|
| ## Training Data |
|
|
| ### Data Sources and Preparation |
|
|
| The model was trained on a unified, large-scale corpus of e-commerce interaction data, aggregating and normalizing multiple public datasets to create a robust foundation for sequential behavior modeling. |
|
|
| The training data combines the following sources: |
|
|
| | Dataset | Description | Key Statistics | |
| |---------------------------------------------------------------|------------------------------------------------------------------|-------------------| |
| | **E-commerce behavior data from multi category store** | Real event logs from a multi-category e-commerce platform | ~285M records | |
| | **E-commerce Clickstream and Transaction Dataset (Kaggle)** | Sequential event data including views and clicks | ~500K+ events | |
| | **E-Commerce Behavior Dataset – Agents for Data** | Product interactions from ~18k users across multiple event types | ~2M interactions | |
| | **Retail Rocket clickstream dataset** | Industry-standard dataset with views, carts, and purchases | ~2.7M events | |
| | **SIGIR 2021 / Coveo Session data challenge** | Navigation sessions with clicks, adds, purchase + metadata | ~30M events | |
| | **JDsearch dataset** | Real interactionswith search queries from JD.com platform | ~26M interactions | |
|
|
| ### Data Unification and Normalization |
|
|
| All datasets underwent a rigorous unification and normalization process: |
|
|
| - **Schema Alignment**: Standardized field names and types across all sources (item_id, brand_id, category_id, event_type, timestamp, price) |
| - **Event Type Normalization**: Mapped varied event nomenclature to a standardized taxonomy (view, cart, purchase) |
| - **ID Harmonization**: Created consistent ID spaces for items, brands, and categories through cross-dataset mapping |
| - **Temporal Alignment**: Unified timestamp formats and established consistent session windows |
| - **Price Normalization**: Applied log-normalization (log1p) followed by standardization using global statistics |
| - **Session Construction**: Reconstructed user sessions based on temporal proximity and interaction patterns |
| - **Quality Filtering**: Removed sessions below minimum length threshold and filtered anomalous interactions |
|
|
| This diverse and comprehensive training corpus enables the model to learn robust representations of e-commerce behavior patterns across different platforms, markets, and interaction types, serving as a strong foundation for downstream fine-tuning tasks. |
|
|
| --- |
|
|
| ### Preprocessing |
|
|
| - Missing categorical values replaced with `UNK` |
| - Price values transformed via `log1p` |
| - Standardization using global mean and standard deviation |
| - Session truncation to fixed-length sequences |
| - Right padding with attention masking |
|
|
| --- |
|
|
| ### Training Objective |
|
|
| Next-item autoregressive prediction using cross-entropy loss with padding ignored. |
|
|
| --- |
|
|
| ### Training Regime |
|
|
| - **Precision:** FP32 |
| - **Optimizer:** AdamW |
| - **Learning Rate:** 1e-3 with warmup |
| - **Gradient Clipping:** 5.0 |
| - **Causal masking applied** |
|
|
| --- |
|
|
| ## Evaluation |
|
|
| ### Metrics |
|
|
| - Cross-Entropy Loss |
| - Perplexity |
| - Recall@20 |
|
|
| ### Results |
|
|
| On the evaluation split, the model achieved: |
|
|
| - **Perplexity:** 24.04 |
| - **Recall@20:** 0.6823 |
|
|
| These results indicate strong next-item prediction performance in session-based e-commerce interaction modeling. |
|
|
| ### Summary |
|
|
| The model demonstrates: |
|
|
| - Low predictive uncertainty (Perplexity 24.04) |
| - High ranking quality for next-item recommendation (Recall@20 of 68.23%) |
|
|
| Performance may vary depending on dataset distribution, session length, and preprocessing configuration. |
|
|
| --- |
|
|
| ## Environmental Impact |
|
|
| - **Hardware:** GPU NVIDIA H100 NVL (94GB PCIe 5.0) |
| - **Precision:** FP32 |
| - **Training Duration:** Several hours (varies by configuration) |
| - **Carbon Impact:** ≈45 kg CO₂e (estimated based on energy consumption of 30h on H100 GPU) |
|
|
| --- |
|
|
| ## Limitations |
|
|
| - No long-term user modeling beyond session scope |
| - Does not include user-level embeddings |
| - Requires predefined categorical vocabularies |
| - Limited generalization to unseen item IDs |
|
|
