| | --- |
| | library_name: peft |
| | license: mit |
| | language: |
| | - en |
| | base_model: |
| | - MightyOctopus/pricer-merged-model-A-v1 |
| | --- |
| | |
| | # Model Card for Model ID |
| |
|
| | <!-- Provide a quick summary of what the model is/does. --> |
| |
|
| |
|
| | ### Model Description |
| |
|
| | pricer-lora-ft-v3 is a fine-tuned large language model (with the base model: MightyOctopus/pricer-merged-model-A-v1) specialized in numeric price prediction for consumer products(e.g. Amazon products etc). |
| | The model predicts approximate product prices from textual metadata such as product title, description, and category. |
| | It demonstrates that a fully open source LLM can be adapted for structured numeric regression tasks traditionally handled by classical ML models. |
| |
|
| |
|
| |
|
| | - **Developed by:** MyungHwan Hong (MightyOctopus) |
| | - **Funded by:** Self-funded / independent research |
| | - **Shared by:** MyungHwan Hong |
| | - **Model type:** Causal Language Model (Numeric Prediction / Regression via Text-to-Number) |
| | - **Language(s) (NLP):** English |
| | - **License:** MIT |
| | - **Finetuned from model:** meta-llama/Llama-3.1-8B |
| |
|
| | ### Model Sources |
| |
|
| | <!-- Provide the basic links for the model. --> |
| |
|
| | - **Repository:** [More Information Needed] |
| | - **Research Note:** (https://docs.google.com/document/d/1PwuOCS6wgO3MqKexnEdAqpVswXMqGilqKEuFyhUGk7M/edit?tab=t.0) |
| | - **Demo:** [More Information Needed] |
| |
|
| |
|
| | ### Direct Use |
| |
|
| | - Predicting approximate Amazon product prices from text metadata |
| |
|
| | - Research on LLM-based numeric regression |
| |
|
| | - Benchmarking open-source LLMs against frontier models (e.g., GPT-4o-mini) |
| |
|
| | - Educational experiments on LoRA fine-tuning and evaluation |
| |
|
| | ### Downstream Use |
| |
|
| | - Price estimation pipelines (non-production) |
| |
|
| | - Feature generation for pricing analytics |
| |
|
| | - Comparative studies with classical ML regressors |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | - Real-time or production pricing systems |
| |
|
| | - Financial decision-making |
| |
|
| | - Legal, medical, or safety-critical applications |
| |
|
| | - Use as an authoritative price source |
| |
|
| |
|
| | ## Bias, Risks, and Limitations |
| |
|
| | - Predictions are approximate, not exact |
| |
|
| | - Performance depends on similarity to training data distribution |
| |
|
| | - The model may hallucinate prices for unfamiliar or novel products |
| |
|
| | - Prices may reflect historical or dataset-specific biases |
| |
|
| | - Not robust to rapid market price changes |
| |
|
| |
|
| | ### Recommendations |
| |
|
| | Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
| |
|
| | - Treat outputs as estimates, not ground truth |
| |
|
| | - Validate predictions against real pricing data |
| |
|
| | - Avoid using the model in high-stakes or commercial systems |
| |
|
| | - Be aware of dataset and temporal bias |
| |
|
| |
|
| | ## How to Get Started with the Model |
| |
|
| | Use the code below to get started with the model. |
| |
|
| | ```python |
| | from transformers import AutoModelForCausalLM, AutoTokenizer |
| | from peft import PeftModel |
| | import torch |
| | |
| | TOKENIZER_MODEL = "meta-llama/Llama-3.1-8B" |
| | BASE_MODE_ID = "MightyOctopus/pricer-merged-model-A-v1" |
| | FINE_TUNED_ADAPTER = "MightyOctopus/pricer-lora-ft-v3" |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_MODEL) |
| | base_model = AutoModelForCausalLM.from_pretrained( |
| | BASE_MODE_ID, |
| | torch_dtype=torch.bfloat16, |
| | device_map="auto" |
| | ) |
| | |
| | tokenizer.pad_token = tokenizer.eos_token |
| | base_model.generation_config.pad_token_id = tokenizer.pad_token_id |
| | |
| | fine_tuned_model = PeftModel.from_pretrained( |
| | base_model, |
| | FINE_TUNED_ADAPTER |
| | ) |
| | |
| | fine_tuned_model.eval() |
| | |
| | prompt = """Product: |
| | Title: Stainless Steel Electric Kettle 1.7L |
| | Category: Home & Kitchen |
| | Description: Fast boiling electric kettle with auto shut-off. |
| | |
| | Price is $""" |
| | |
| | inputs = tokenizer(prompt, return_tensors="pt").to(fine_tuned_model.device) |
| | |
| | with torch.no_grad(): |
| | outputs = fine_tuned_model.generate( |
| | **inputs, |
| | max_new_tokens=10, |
| | temperature=0.2 |
| | ) |
| | |
| | print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| | ``` |
| |
|
| |
|
| | ## Training Details |
| |
|
| | ### Training Data |
| |
|
| | - Amazon product metadata dataset |
| |
|
| | - Fields include title, description, category, and ground-truth price |
| |
|
| | - Prices normalized via structured text prompts |
| |
|
| | - Dataset split into training, validation, and test sets |
| |
|
| | ### Training Procedure |
| |
|
| | <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
| |
|
| | #### Preprocessing |
| |
|
| | Preprocessing |
| |
|
| | - Text normalization |
| |
|
| | - Structured prompt formatting |
| |
|
| | - Numeric price represented as plain text output |
| |
|
| | - Loss applied only to answer tokens using response masking |
| |
|
| |
|
| | #### Training Hyperparameters |
| |
|
| | **Training regime:** |
| | Supervised Fine-Tuning (SFT) with LoRA |
| |
|
| | Key Hyperparameters: |
| |
|
| | - Optimizer: AdamW |
| |
|
| | - Learning Rate: 2e-5 |
| |
|
| | - Epochs: 2 |
| |
|
| | - Batch Size: 16 |
| |
|
| | - Gradient Accumulation: 2 |
| |
|
| | - Effective Batch Size: 3 |
| |
|
| | - LoRA Rank: 32 |
| |
|
| | - LoRA Alpha: 64 |
| |
|
| | - LoRA Dropout: 0.0 |
| |
|
| | - Weight Decay: 0.0 |
| |
|
| | - Precision: bfloat16 |
| |
|
| | #### Speeds, Sizes, Times |
| |
|
| | - Base model: 8B parameters |
| |
|
| | - LoRA parameters: ~0.5% of base model |
| |
|
| | - Training time: Approx. 21 hours on single GPU |
| |
|
| | ## Evaluation |
| |
|
| | ### Testing Data |
| |
|
| | - Held-out Amazon product samples |
| |
|
| | - Products not seen during training |
| |
|
| | ### Factors |
| |
|
| | - Product category |
| |
|
| | - Price range distribution |
| |
|
| | - Description length |
| |
|
| | - Token length |
| |
|
| | ### Metrics |
| |
|
| | - Mean Absolute Error (MAE) |
| |
|
| | - Root Mean Squared Logarithmic Error (RMSLE) |
| |
|
| | - Hit Rate (prediction within ±20% of ground truth) |
| |
|
| |
|
| | ### Results |
| |
|
| | | Model | MAE ($) | RMSLE | Hit Rate | |
| | | ----------------------- | ---------- | -------- | --------- | |
| | | GPT-4o-mini (Zero-shot) | ~84.56 | 0.70 | 49.4% | |
| | | **pricer-lora-ft-v3** | **~67.40** | **0.59** | **62.0%** | |
| |
|
| | #### GPT 4o Mini |
| |
|
| |  |
| |
|
| | #### pricer-lora-ft--v3 |
| |
|
| |  |
| |
|
| | #### Summary |
| |
|
| |
|
| |
|
| | ## Model Examination |
| |
|
| | <!-- Relevant interpretability work for the model goes here --> |
| |
|
| | The model demonstrates that fine-tuned open-source LLMs can outperform frontier zero-shot models on specialized numeric tasks when trained with domain-specific data and structured prompts. |
| |
|
| | ## Environmental Impact |
| |
|
| | <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
| |
|
| | Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
| |
|
| | - **Hardware Type:** NVIDIA GPU (e.g., A100) |
| | - **Hours used:** ~21 hours |
| | - **Cloud Provider:** Google Colab / Hugging Face |
| | - **Compute Region:** Unknown (Colab GPUs used) |
| | - **Carbon Emitted:** Not estimated |
| |
|
| | ## Technical Specifications |
| |
|
| | ### Model Architecture and Objective |
| |
|
| | - Transformer-based causal language model |
| |
|
| | - Objective: Next-token prediction optimized for numeric accuracy |
| |
|
| | - Loss applied selectively to price tokens |
| |
|
| | ### Compute Infrastructure |
| |
|
| | - Single-GPU fine-tuning |
| |
|
| | - LoRA-based parameter-efficient training |
| |
|
| | #### Hardware |
| |
|
| | - NVIDIA GPU (A100) |
| |
|
| | #### Software |
| |
|
| | - Transformers |
| |
|
| | - TRL |
| |
|
| | - PEFT 0.14.0 |
| |
|
| | - PyTorch |
| |
|
| | ## Citation |
| |
|
| | <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
| |
|
| | **BibTeX:** |
| |
|
| | @misc{hong2025pricer, |
| | author = {Hong, MyungHwan}, |
| | title = {Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model}, |
| | year = {2025}, |
| | url = {https://huggingface.co/MightyOctopus/pricer-lora-ft-v3} |
| | } |
| |
|
| |
|
| | **APA:** |
| |
|
| | MyungHwan Hong, (2025). Pricer LoRA Fine-Tuned LLaMA 3.1 8B Model. Hugging Face. https://huggingface.co/MightyOctopus/pricer-lora-ft-v3 |
| |
|
| | ## Glossary |
| |
|
| | <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> |
| |
|
| | [More Information Needed] |
| |
|
| | ## More Information |
| |
|
| | [More Information Needed] |
| |
|
| | ## Model Card Authors |
| |
|
| | MyungHwan Hong |
| |
|
| | ## Model Card Contact |
| |
|
| | Hugging Face: MightyOctopus |
| |
|
| | ### Framework versions |
| |
|
| | - PEFT 0.14.0 |
