Update README.md

README.md

@@ -78,6 +78,78 @@ Solution:
 
 the output of the model doesn't have (for now) any formatting, it's just reasoning as output
 
+# Code Example
+
+- Using transformers:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Load the tokenizer and model
+model_name = "secemp9/TraceBack-12b"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+# Move the model to the desired device
+device = 'cuda' if torch.cuda.is_available() else 'cpu'
+model.to(device)
+
+# Define the messages
+messages = [
+    {"role": "user", "content": """Instruction:
+how many r in strawberry
+
+
+Solution:
+There are **three** "r"s in "strawberry."
+"""}
+]
+
+# Step 1: Apply chat template to get formatted text as a string
+formatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+# Step 2: Tokenize the formatted text into a dictionary of tensors
+inputs = tokenizer(formatted_text, return_tensors="pt").to(device)
+
+# Generate the response
+outputs = model.generate(**inputs, max_new_tokens=32000)
+
+# Decode and print the output
+generated_text = tokenizer.decode(outputs[0])
+print(generated_text)
+```
+
+- unsloth
+```python
+from unsloth import FastLanguageModel
+
+# Load the model and tokenizer
+model, tokenizer = FastLanguageModel.from_pretrained("secemp9/TraceBack-12b")
+
+# Define the messages (replace "stuff_here" with your actual input)
+messages = [
+    {"role": "user", "content": """Instruction:
+how many r in strawberry
+
+
+Solution:
+There are **three** "r"s in "strawberry."
+"""}
+]
+
+# Step 1: Apply chat template to get formatted text as a string
+formatted_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+# Step 2: Tokenize the formatted text into a dictionary of tensors
+inputs = tokenizer(formatted_text, return_tensors="pt").to(model.device)
+
+# Generate the response
+outputs = model.generate(**inputs, max_new_tokens=32000)
+
+# Decode and print the output
+generated_text = tokenizer.decode(outputs[0])
+print(generated_text)
+```
 # Axolotl config
 
 For this, I basically tried to convert my unsloth code to an axolotl config file. I also used deepspeed. Configuration below: