ChisomChibuike/amazon-rating-sentiment

Model Card: Persona-Driven Review & Rating Predictor

This model card provides an overview of the fine-tuned model designed to predict user ratings and text reviews based on user personas and item identifiers.

---

Model Details

• Developed by: Chisom Chibuike
• Model Type: Large Language Model (Autoregressive Transformer), fine-tuned via Parameter-Efficient Fine-Tuning (PEFT/QLoRA).
• Base Model: Llama 3.1 8B Instruct
• Language(s): English (primarily e-commerce review vernacular)
• License: Meta Llama 3.1 Community License
• Finetuning Date: May 2026

---

Intended Use

Primary Use Case

• Task A (Persona-to-Review Mapping): Predicting a user's likely numerical rating, and review text for a specific item, given their structured behavioral persona profile.
• Simulation & Testing: Enabling e-commerce platforms to simulate how different customer cohorts might react to new products or inventory changes.

Out-of-Scope Uses

• Generating fake reviews for real products to manipulate commercial ratings.
• Deploying without safety filtering to interact directly with end-users without monitoring for hallucinations.

---

Training Data & Inputs

The model was fine-tuned on a merged dataset consisting of product reviews (amazon review) and generated user persona profiles (the user persona profile was generated with the aid of an LLM, using the user information from amazon review database). see here for dataset.

• Input Context:
  • persona: A structured JSON object containing a user's values, writing style markers, common complaints, and rating distribution.
  • title: The textual headline of the product being review.
• Target Outputs:
  • average_rating: Numerical target (1.0 to 5.0).
  • text: The detailed review body reflecting the persona's tone, detail level, and vocabulary markers.

---

Evaluation Metrics

During training and validation, model performance was tracked using the following metrics:

Component	Metric	Target Objective
Rating Prediction	Mean Absolute Error (MAE)	Minimize deviation from true average_rating
Text Generation	ROUGE-L	Capture style matching and alignment with persona markers

---

Limitations & Biases

Here are the honest limitation of what we currently have.

Critical Caveat: The model's predictive power is strictly bounded by the data present in the training set (heavily weighted toward categories like All Beauty and Skincare in the initial samples).

• Domain Bias: Performance may degrade significantly if evaluated on items outside the primary training categories (e.g., items like Electronics or Automotive parts ).
• Cold Start Problem: The model requires a rich, established persona profile. If a persona contains minimal evidence or empty arrays, prediction accuracy drops.
• Style Homogenization: The model may default to generic "helpful reviewer" archetypes if the unique writing style signals in the input persona are weak.

See this link for github repository.