README.md

metadata

license: llama2
base_model: unsloth/llama-2-13b
tags:
  - llama-2
  - unsloth
  - fine-tuned
  - instruction-following
  - text-generation
datasets:
  - Stormtrooperaim/llama2-Lemon
language:
  - en
library_name: transformers
pipeline_tag: text-generation

Llama-2-13B Lemon Fine-tuned Model

This is a fine-tuned version of Llama-2-13B trained on the llama2-Lemon dataset to improve instruction-following capabilities. The model has been optimized using Unsloth for efficient training with LoRA adapters.

Model Details

• Base Model: unsloth/llama-2-13b
• Training Dataset: Stormtrooperaim/llama2-Lemon
• Training Method: QLoRA with Unsloth
• Model Size: 13B parameters

Usage

This model uses the Llama-2 prompt format:

<s>[INST] Your instruction here [/INST]

Training Details

• Trained with 4-bit quantization using Unsloth
• LoRA rank: 16
• Learning rate: 2e-4
• Optimizer: AdamW 8-bit