Update README.md

README.md

@@ -5,9 +5,84 @@ tags: []
 
 # Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
-
+Patched LLama 3.2 8B from LLaMA 3.2 11B Model
+
+Here’s the complete, refined code for patching the weights:
+```python
+# Import required libraries
+from transformers import AutoProcessor, AutoTokenizer, AutoModelForImageTextToText, AutoModelForCausalLM
+
+# Load the 11B Vision-Instruct model
+processor = AutoProcessor.from_pretrained("meta-llama/Llama-3.2-11B-Vision-Instruct")
+model = AutoModelForImageTextToText.from_pretrained("meta-llama/Llama-3.2-11B-Vision-Instruct")
+
+# Load the 8B text-only model
+s_tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
+s_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
+
+# Prepare input text for testing
+input_text = "Write me a poem about Machine Learning."
+input_ids = s_tokenizer(input_text, return_tensors="pt")
+
+# Test the original 8B model
+outputs = s_model.generate(**input_ids, do_sample=False, max_new_tokens=10)
+print("8B Model Output:", s_tokenizer.decode(outputs[0]))
+
+# Patch weights from the 11B model into the 8B model
+model_weight = model.state_dict()
+s_model_dict = s_model.state_dict()
+skip_layer = 0  # Track skipped layers
+
+for key in s_model_dict.keys():
+    if "layers." in key:
+        layer_idx = int(key.split("layers.")[1].split(".")[0])  # Extract layer index
+        try:
+            s_model_dict[key] = model_weight[
+                "language_model." + key.replace(f"layers.{layer_idx}.", f"layers.{layer_idx + skip_layer}.")
+            ]
+        except KeyError:
+            skip_layer += 1
+            s_model_dict[key] = model_weight[
+                "language_model." + key.replace(f"layers.{layer_idx}.", f"layers.{layer_idx + skip_layer}.")
+            ]
+    else:
+        s_model_dict[key] = model_weight["language_model." + key]
+
+# Test the patched 8B model
+outputs = s_model.generate(**input_ids, do_sample=False, max_new_tokens=10)
+print("Patched 8B Model Output:", s_tokenizer.decode(outputs[0]))
+
+# Test the original 11B model
+outputs = model.generate(**input_ids, do_sample=False, max_new_tokens=10)
+print("11B Model Output:", s_tokenizer.decode(outputs[0]))
+
+```
+
+### **Example Outputs**
+
+**Prompt:** "Write me a poem about Machine Learning."
+
+**Outputs:**
+1. **8B Model Output (Before Patching):**
+   ```
+   <|begin_of_text|>Write me a poem about Machine Learning.
+   Artificial minds, born from code,
+   Learning
+   ```
+
+2. **Patched 8B Model Output:**
+   ```
+   <|begin_of_text|>Write me a poem about Machine Learning.
+   In silicon halls, where data reigns
+   ```
+
+3. **11B Model Output:**
+   ```
+   <|begin_of_text|>Write me a poem about Machine Learning.
+   In silicon halls, where data reigns
+   ```
 
+---
 
 ## Model Details