PM234 commited on
Commit
d678d93
·
verified ·
1 Parent(s): 619dfdc

Update code example in Readme

Browse files
Files changed (1) hide show
  1. README.md +14 -19
README.md CHANGED
@@ -12,13 +12,14 @@ tags:
12
 
13
  # Model Card for Model ID
14
  <!-- Provide a quick summary of what the model is/does. -->
15
- This is a LoRA adapter-based fine-tuned version of DeepSeek-R1-Distill-Llama-8B, optimized for Medical Question Answering (MedQA) using PEFT, LoRA adapters, and bnb-4bit quantization. The fine-tuning was performed on a curated dataset containing medical questions and answers from trusted sources.
16
 
17
 
18
  ## Model Details
19
 
20
  ### Model Description
21
 
 
22
  <!-- Provide a longer summary of what this model is. -->
23
 
24
 
@@ -77,27 +78,21 @@ Users (both direct and downstream) should be made aware of the risks, biases and
77
 
78
  Use the code below to get started with the model.
79
 
80
- from transformers import AutoModelForCausalLM, AutoTokenizer
81
-
82
- from peft import PeftModel
83
-
84
- base_model = "unsloth/DeepSeek-R1-Distill-Llama-8B-bnb-4bit"
85
-
86
- adapter_model = "PM234/DeepSeek-R1-MedExpert-LoRA-8B-bnb4bit"
87
-
88
- #### Load tokenizer from the base model
89
- tokenizer = AutoTokenizer.from_pretrained(base_model)
90
 
91
- #### Load base model
92
- model = AutoModelForCausalLM.from_pretrained(
93
- base_model,
94
- torch_dtype="auto",
95
- device_map="auto"
96
- )
97
 
98
- #### Load LoRA adapter on top of the base model
99
- model = PeftModel.from_pretrained(model, adapter_model)
100
 
 
 
 
 
 
 
101
 
102
  [More Information Needed]
103
 
 
12
 
13
  # Model Card for Model ID
14
  <!-- Provide a quick summary of what the model is/does. -->
15
+ This is a LoRA adapter-based fine-tuned version of DeepSeek-R1-Distill-Llama-8B, optimized for Medical Question Answering (MedQA) using PEFT, LoRA adapters, and bnb-4bit quantization. The fine-tuning was performed on a curated dataset of 10k examples containing medical questions and answers from trusted sources.
16
 
17
 
18
  ## Model Details
19
 
20
  ### Model Description
21
 
22
+
23
  <!-- Provide a longer summary of what this model is. -->
24
 
25
 
 
78
 
79
  Use the code below to get started with the model.
80
 
81
+ ```python
82
+ from unsloth import FastLanguageModel
 
 
 
 
 
 
 
 
83
 
84
+ # Load model + adapters directly
85
+ model, tokenizer = FastLanguageModel.from_pretrained("PM234/DeepSeek-R1-MedExpert-LoRA-8B-bnb4bit")
 
 
 
 
86
 
87
+ # Prep for inference
88
+ FastLanguageModel.for_inference(model)
89
 
90
+ # Example:
91
+ test_input = "Below is an instruction...\n### Instruction: Answer the following medical question.\n### Input: What is the primary source of energy for the human body?\n### Response:"
92
+ inputs = tokenizer(test_input, return_tensors="pt").to("cuda")
93
+ outputs = model.generate(**inputs, max_new_tokens=20)
94
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True)) # "Glucose"
95
+ ```
96
 
97
  [More Information Needed]
98