nikhiltharlada/mistral-7b-quotes-final

Mistral-7B Motivational Quotes Generator

This model is a QLoRA fine-tuned version of Mistral-7B-v0.1 designed to generate short motivational and inspirational quotes.

It was trained using 4-bit quantization + LoRA adapters, allowing efficient training and inference on consumer GPUs such as NVIDIA T4 (Colab).

---

Model Details

Field	Value
Developer	Nikhil Tharlada
Base Model	mistralai/Mistral-7B-v0.1
Model Type	Causal Language Model (PEFT / LoRA)
Task	Motivational Quote Generation
Language	English
License	Apache-2.0

---

Model Links

• Adapter Repository: https://huggingface.co/nikhiltharlada/mistral-7b-quotes-final
• Mistral Paper: https://arxiv.org/abs/2310.06825
• LoRA Paper: https://arxiv.org/abs/2106.09685

---

Papers & References

Mistral 7B Paper

Mistral 7B
https://arxiv.org/abs/2310.06825

LoRA: Low-Rank Adaptation of Large Language Models

Hu et al., 2021
https://arxiv.org/abs/2106.09685

QLoRA: Efficient Finetuning of Quantized LLMs

Dettmers et al., 2023
https://arxiv.org/abs/2305.14314

---

Intended Use

Direct Use

This model is optimized for generating motivational quotes when prompted with the instruction:

<s>[INST] Give a motivational quote [/INST] {quote}</s>

Example outputs:

• "Small steps today create massive change tomorrow."
• "Your limits exist only where you accept them."

Out-of-Scope Use

This model is NOT intended for:

• factual question answering
• legal / medical advice
• decision-making systems
• production use without moderation

---

Bias, Risks & Limitations

• The model reflects biases from the Goodreads quotes dataset
• It may hallucinate or misattribute quotes
• No built-in content moderation
• Best suited for creative generation only

---

How to Use the Model

Install Dependencies

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

model_id = "mistralai/Mistral-7B-v0.1"
adapter_id = "nikhiltharlada/mistral-7b-quotes-final"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "<s>[INST] Give a motivational quote [/INST]"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Dataset

Abirate/english_quotes

Source: Goodreads

Contains:

• Quote text
• Author
• Tags

We used only the quote text for training.

---

Data Formatting

Each training sample was converted to Mistral instruction format:

<s>[INST] Give a motivational quote [/INST] {quote}</s>

Tokenization

• Max length: 512 tokens
• Padding: Right padding using EOS token

---

Training Configuration

Parameter	Value
Training Method	QLoRA (4-bit NF4)
LoRA Rank	16
Learning Rate	2e-4
Optimizer	AdamW
Epochs	1
Batch Size	1
Gradient Accumulation	16
Effective Batch Size	16

---

Hardware

Component	Value
GPU	NVIDIA Tesla T4
VRAM	16 GB
Platform	Google Colab
Training Time	~1 hour

QLoRA reduced memory usage by ~4× compared to full fine-tuning.

---

Evaluation

Metrics Used

• Training Loss
• Perplexity
• Manual qualitative evaluation

Results

• Successful adaptation to motivational quote style
• Strong instruction following when using [INST] format
• Consistent short inspirational outputs

---

Model Architecture Highlights

Mistral-7B introduces:

Sliding Window Attention (SWA)

Each layer attends to the previous 4096 tokens, enabling:

• Linear compute scaling
• Long context capability

Grouped-Query Attention (GQA)

Provides:

• Reduced memory usage
• Faster decoding

Total parameters: 7.3B

Only LoRA adapters were trained → base model knowledge preserved.

---

Environmental Impact

Estimated using the ML CO₂ calculator.

Metric	Value
GPU	NVIDIA T4 (70W)
Training Time	~2 hours
Estimated CO₂	< 0.05 kg CO₂eq

This demonstrates the efficiency of QLoRA training.

---

Citation

@article{mistral2023,
  title={Mistral 7B},
  author={Jiang, Albert Q. and others},
  journal={arXiv preprint arXiv:2310.06825},
  year={2023}
}

---

Model Card Authors

Nikhil Tharlada