DikovAlexandr/Qwen2.5-3B-LoAR-Classifier

Qwen2.5-3B — LoRA adapter for Marketplace Review Category Classification

Model Details

This repository contains a LoRA adapter trained to classify marketplace customer reviews into product categories. The adapter was fine-tuned on a downstream sequence-classification task and is intended to be used with the Qwen/Qwen2.5-3B base model using the PEFT library.

Model Description

• Adapter type: LoRA (PEFT)

• Base model: Qwen/Qwen2.5-3B

• Task: Multiclass text classification (product category of marketplace reviews)

• Labels (classes):

  • бытовая техника (home appliances)
  • обувь (shoes)
  • одежда (clothing)
  • посуда (tableware / kitchenware)
  • текстиль (textiles)
  • товары для детей (kids / baby products)
  • украшения и аксессуары (jewelry & accessories)
  • электроника (electronics)
  • нет товара (NO_PRODUCT)

• Developed by: [More Information Needed]

• Funded by [optional]: [More Information Needed]

• Shared by [optional]: [More Information Needed]

• Model type: [More Information Needed]

• Language(s) (NLP): [More Information Needed]

• License: [More Information Needed]

• Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

How to load (inference)

Install prerequisites:

pip install transformers peft safetensors huggingface_hub
# If you want 8-bit:
pip install bitsandbytes

Training Details

Training Data

LORA_R = 64

LORA_ALPHA = 128

LORA_DROPOUT = 0.10

NUM_EPOCHS = 5

PER_DEVICE_BATCH_SIZE = 1

GRAD_ACCUM_STEPS = 8

LEARNING_RATE = 2e-4

WEIGHT_DECAY = 0.01

WARMUP_RATIO = 0.03

MAX_SEQ_LENGTH = 256

SEED = 42

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Framework versions

• PEFT 0.15.2