README.md

license: mit
language:
  - en
base_model:
  - MatteoKhan/Mistral-LLaMA-Fusion
library_name: transformers
tags:
  - fine-tuned
  - cosmetic-domain
  - lora
  - mistral
  - llama
  - rtx4060-optimized
💄 CosmeticAdvisor: Expert Model for Beauty & Cosmetic Queries
📌 Overview
Mistral-LLaMA-Fusion-Cosmetic is a domain-specialized language model, fine-tuned on a dataset focused on cosmetic-related queries. Built from the powerful Mistral-LLaMA-Fusion, this version benefits from LoRA-based fine-tuning and GPU optimization on a RTX 4060.

🔗 Created by: Matteo Khan
🎓 Affiliation: Apprentice at TW3 Partners (Generative AI Research)
📍 License: MIT

🔗 Connect on LinkedIn(https://www.linkedin.com/in/matteo-khan-a10309263/)
🔍 Base Model

🧠 Model Details
Architecture: Mistral + LLaMA fusion

Technique: Fine-tuned with LoRA (Low-Rank Adaptation)

Base Model: MatteoKhan/Mistral-LLaMA-Fusion

Training Dataset: Proprietary dataset (Parquet) of user queries in the cosmetic and beauty domain

Training Hardware: RTX 4060 (8GB VRAM), 3 epochs

🎯 Intended Use
This model is optimized for:

✅ Responding to beauty & cosmetic product questions

✅ Assisting in cosmetic product recommendation

✅ Enhancing chatbots in beauty domains

✅ Cosmetic-focused creative content generation

🛠️ Technical Details
Fine-tuning Method: LoRA (r=8, α=16, dropout=0.05)

Quantization: 4-bit NF4 via bitsandbytes

Training Strategy: Gradient checkpointing + mixed precision (fp16)

Sequence Length: 256 tokens

Batch Strategy: Batch size 1 + gradient accumulation 16

🧪 Training Configuration (LoRA)
python
Copier
Modifier
peft_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    inference_mode=False,
    r=8,
    lora_alpha=16,
    lora_dropout=0.05,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    bias="none",
)
🚀 How to Use
python
Copier
Modifier
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "MatteoKhan/CosmeticAdvisor"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = "What skincare products are best for oily skin?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
⚠️ Limitations
May hallucinate or provide incorrect information

Knowledge is limited to cosmetic domain-specific data

Should not replace professional dermatological advice

🧾 Citation
If you use this model in your research, please cite:

bibtex
Copier
Modifier
@misc{mistralllama2025cosmetic,
  title={Mistral-LLaMA-Fusion-Cosmetic},
  author={Matteo Khan},
  year={2025},
  note={Fine-tuned for cosmetic domain},
  url={https://huggingface.co/MatteoKhan/CosmeticAdvisor}
}