Update README.md

README.md

@@ -1,3 +1,15 @@
 ---
 license: mit
 ---
+This model is a part of Project InterACT (Multi model AI system) involving an object detection model and an LLM
+
+This is a model built by finetuning the Llama-2-7b-chat model on custom dataset: Jithendra-k/InterACT_LLM.
+
+Points to consider for Finetuning Llama-2_7B_chat model:
+=> Free Google Colab offers a 15GB Graphics Card (Limited Resources --> Barely enough to store Llama 2–7b’s weights)
+=> We also considered the overhead due to optimizer states, gradients, and forward activations
+=> Full fine-tuning is not possible in our case due to computation: we used parameter-efficient fine-tuning (PEFT) techniques like LoRA or QLoRA.
+=> To drastically reduce the VRAM usage, we fine-tuned the model in 4-bit precision, which is why we've used QLoRA technique.
+=> We only trained with 5 epochs considering our computation, time and early stopping.
+
+Code to finetune a Llama-2_7B_chat model: https://colab.research.google.com/drive/1ZTdSKu2mgvQ1uNs0Wl7T7gniuoZJWs24?usp=sharing