Update README.md

README.md

@@ -75,6 +75,113 @@ print(output[0]['generated_text'])
 
 ```
 
+### Outputs
+microsoft/Phi-3-medium-128k-instruct
+```python
+ To train a language model using Lora and SFT (Supervised Fine-tuning), you can follow these steps:
+
+1. Install the required libraries:
+
+```python
+!pip install transformers
+```
+
+2. Import the necessary libraries:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM, Trainer, TrainingArguments
+```
+
+3. Load the pre-trained Lora model and tokenizer:
+
+```python
+tokenizer = AutoTokenizer.from_pretrained("microsoft/llama-65b-lora")
+model = AutoModelForCausalLM.from_pretrained("microsoft/llama-65b-lora")
+```
+
+4. Load the SFT dataset:
+
+```python
+# Replace with the path to your SFT dataset
+train_dataset =...
+```
+
+5. Define the training arguments:
+
+```python
+training_args = TrainingArguments(
+    output_dir="./results",
+    num_train_epochs=3,
+    per_device_train_batch_size=4,
+    per_device_eval_batch_size=4,
+    warmup_steps=500,
+    weight_decay=0.01,
+    logging_dir='./logs',
+    logging_steps=10,
+)
+```
+
+6. Create a custom Trainer for Lora and SFT:
+
+```python
+class LoraSFTTrainer(Trainer):
+    def compute_loss(self, model, inputs):
+        labels = inputs.pop("labels")
+        outputs = model(**inputs, use_cache=False)
+        lm_logits = outputs.logits
+        loss_fct = torch.nn.CrossEntropyLoss()
+        masked_lm_loss = loss_fct(lm_logits.view(-1, self.model.config.vocab_size), labels.view(-1))
+        return masked_lm_loss
+```
+
+7. Initialize the trainer and train the model:
+
+```python
+trainer = LoraSFTTrainer(
+    model=model,
+    args=training_args,
+    train_dataset=train_dataset,
+    eval_dataset=None,
+)
+
+trainer.train()
+```
+
+This code will train the Lora model using the SFT dataset. You can adjust the training arguments and the dataset path according to your needs.
+
+```
+REILX/Phi-3-medium-128k-code-instruct
+```python
+ import torch
+from transformers import RobertaForCausalLM, RobertaTokenizer
+
+# Load the model and tokenizer
+model = RobertaForCausalLM.from_pretrained('roberta-base')
+tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
+
+# Load the data
+data = [
+    "This is a sample sentence.",
+    "Another sample sentence."
+]
+
+# Tokenize the data
+input_ids = [tokenizer.encode(sentence, add_special_tokens=True) for sentence in data]
+
+# Train the model
+model.train()
+for input_id in input_ids:
+    outputs = model(input_id, labels=input_id)
+    loss = outputs.loss
+    loss.backward()
+    optimizer.step()
+
+# Save the model
+model.save_pretrained('my_model')
+
+```
+
 ### Training hyperparameters
 
 The following hyperparameters were used during training: