--- library_name: peft license: mit language: - en base_model: - MightyOctopus/pricer-merged-model-A-v1 --- # Model Card for Model ID ### Model Description pricer-lora-ft-v3 is a fine-tuned large language model (with the base model: MightyOctopus/pricer-merged-model-A-v1) specialized in numeric price prediction for consumer products(e.g. Amazon products etc). The model predicts approximate product prices from textual metadata such as product title, description, and category. It demonstrates that a fully open source LLM can be adapted for structured numeric regression tasks traditionally handled by classical ML models. - **Developed by:** MyungHwan Hong (MightyOctopus) - **Funded by:** Self-funded / independent research - **Shared by:** MyungHwan Hong - **Model type:** Causal Language Model (Numeric Prediction / Regression via Text-to-Number) - **Language(s) (NLP):** English - **License:** MIT - **Finetuned from model:** meta-llama/Llama-3.1-8B ### Model Sources - **Repository:** [More Information Needed] - **Research Note:** (https://docs.google.com/document/d/1PwuOCS6wgO3MqKexnEdAqpVswXMqGilqKEuFyhUGk7M/edit?tab=t.0) - **Demo:** [More Information Needed] ### Direct Use - Predicting approximate Amazon product prices from text metadata - Research on LLM-based numeric regression - Benchmarking open-source LLMs against frontier models (e.g., GPT-4o-mini) - Educational experiments on LoRA fine-tuning and evaluation ### Downstream Use - Price estimation pipelines (non-production) - Feature generation for pricing analytics - Comparative studies with classical ML regressors ### Out-of-Scope Use - Real-time or production pricing systems - Financial decision-making - Legal, medical, or safety-critical applications - Use as an authoritative price source ## Bias, Risks, and Limitations - Predictions are approximate, not exact - Performance depends on similarity to training data distribution - The model may hallucinate prices for unfamiliar or novel products - Prices may reflect historical or dataset-specific biases - Not robust to rapid market price changes ### Recommendations Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. - Treat outputs as estimates, not ground truth - Validate predictions against real pricing data - Avoid using the model in high-stakes or commercial systems - Be aware of dataset and temporal bias ## How to Get Started with the Model Use the code below to get started with the model. ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel import torch TOKENIZER_MODEL = "meta-llama/Llama-3.1-8B" BASE_MODE_ID = "MightyOctopus/pricer-merged-model-A-v1" FINE_TUNED_ADAPTER = "MightyOctopus/pricer-lora-ft-v3" tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODE_ID, torch_dtype=torch.bfloat16, device_map="auto" ) tokenizer.pad_token = tokenizer.eos_token base_model.generation_config.pad_token_id = tokenizer.pad_token_id fine_tuned_model = PeftModel.from_pretrained( base_model, FINE_TUNED_ADAPTER ) fine_tuned_model.eval() prompt = """Product: Title: Stainless Steel Electric Kettle 1.7L Category: Home & Kitchen Description: Fast boiling electric kettle with auto shut-off. Price is $""" inputs = tokenizer(prompt, return_tensors="pt").to(fine_tuned_model.device) with torch.no_grad(): outputs = fine_tuned_model.generate( **inputs, max_new_tokens=10, temperature=0.2 ) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` ## Training Details ### Training Data - Amazon product metadata dataset - Fields include title, description, category, and ground-truth price - Prices normalized via structured text prompts - Dataset split into training, validation, and test sets ### Training Procedure #### Preprocessing Preprocessing - Text normalization - Structured prompt formatting - Numeric price represented as plain text output - Loss applied only to answer tokens using response masking #### Training Hyperparameters **Training regime:** Supervised Fine-Tuning (SFT) with LoRA Key Hyperparameters: - Optimizer: AdamW - Learning Rate: 2e-5 - Epochs: 2 - Batch Size: 16 - Gradient Accumulation: 2 - Effective Batch Size: 3 - LoRA Rank: 32 - LoRA Alpha: 64 - LoRA Dropout: 0.0 - Weight Decay: 0.0 - Precision: bfloat16 #### Speeds, Sizes, Times - Base model: 8B parameters - LoRA parameters: ~0.5% of base model - Training time: Approx. 21 hours on single GPU ## Evaluation ### Testing Data - Held-out Amazon product samples - Products not seen during training ### Factors - Product category - Price range distribution - Description length - Token length ### Metrics - Mean Absolute Error (MAE) - Root Mean Squared Logarithmic Error (RMSLE) - Hit Rate (prediction within ±20% of ground truth) ### Results | Model | MAE ($) | RMSLE | Hit Rate | | ----------------------- | ---------- | -------- | --------- | | GPT-4o-mini (Zero-shot) | ~84.56 | 0.70 | 49.4% | | **pricer-lora-ft-v3** | **~67.40** | **0.59** | **62.0%** | #### GPT 4o Mini ![Screenshot 2025-12-17 at 4.34.49 PM](https://cdn-uploads.huggingface.co/production/uploads/6845b5bc786e0c9d32aad167/8WzzRxU_mZScltMQc23kD.png) #### pricer-lora-ft--v3 ![Screenshot 2025-12-17 at 4.35.28 PM](https://cdn-uploads.huggingface.co/production/uploads/6845b5bc786e0c9d32aad167/0dx54TlZtrOmRHAAygWVN.png) #### Summary ## Model Examination The model demonstrates that fine-tuned open-source LLMs can outperform frontier zero-shot models on specialized numeric tasks when trained with domain-specific data and structured prompts. ## Environmental Impact Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). - **Hardware Type:** NVIDIA GPU (e.g., A100) - **Hours used:** ~21 hours - **Cloud Provider:** Google Colab / Hugging Face - **Compute Region:** Unknown (Colab GPUs used) - **Carbon Emitted:** Not estimated ## Technical Specifications ### Model Architecture and Objective - Transformer-based causal language model - Objective: Next-token prediction optimized for numeric accuracy - Loss applied selectively to price tokens ### Compute Infrastructure - Single-GPU fine-tuning - LoRA-based parameter-efficient training #### Hardware - NVIDIA GPU (A100) #### Software - Transformers - TRL - PEFT 0.14.0 - PyTorch ## Citation **BibTeX:** @misc{hong2025pricer, author = {Hong, MyungHwan}, title = {Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model}, year = {2025}, url = {https://huggingface.co/MightyOctopus/pricer-lora-ft-v3} } **APA:** MyungHwan Hong, (2025). Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model. Hugging Face. https://huggingface.co/MightyOctopus/pricer-lora-ft-v3 ## Glossary [More Information Needed] ## More Information [More Information Needed] ## Model Card Authors MyungHwan Hong ## Model Card Contact Hugging Face: MightyOctopus ### Framework versions - PEFT 0.14.0