Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+license: apache-2.0
+base_model: Llama-2-7B-bf16-sharded
+model-index:
+- name: llama_ft
+  results: []
+---
+
+
+# llama_ft
+
+This model is a fine-tuned version of [Llama-2-7B-bf16-sharded](https://huggingface.co/TinyPixel/Llama-2-7B-bf16-sharded) on a grocery cart dataset.
+
+## Intended uses & limitations
+
+The model helps to tell to what type of grocery does the following items belong to.
+
+
+## Training procedure
+Fine tuning techniques like Qlora and PEFT have been used to train the model on the dataset on a single gpu , and the adapters are then finally merged with the model.
+
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.float16
+
+The loading configurations of the model
+
+### Training hyperparameters
+
+The following are the LORA configs-->
+
+    lora_alpha = 16
+    lora_dropout = 0.1
+    lora_r = 64
+
+    peft_config = LoraConfig(
+    lora_alpha=lora_alpha,
+    lora_dropout=lora_dropout,
+    r=lora_r,
+    bias="none",
+    task_type="CAUSAL_LM",
+    target_modules=["q_proj","v_proj"]
+    )
+
+The following are the training configs -->
+
+    per_device_train_batch_size = 4
+    gradient_accumulation_steps = 4
+    optim = "paged_adamw_32bit"
+    save_steps = 10
+    logging_steps = 1
+    learning_rate = 2e-4
+    max_grad_norm = 0.3
+    max_steps = 120
+    warmup_ratio = 0.03
+    lr_scheduler_type = "constant"
+