Mr-FineTuner commited on
Commit
7d02527
·
verified ·
1 Parent(s): 9229ee1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -0
README.md CHANGED
@@ -8,6 +8,23 @@ tags:
8
 
9
  <!-- Provide a quick summary of what the model is/does. -->
10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
 
13
  ## Model Details
 
8
 
9
  <!-- Provide a quick summary of what the model is/does. -->
10
 
11
+ model = FastLanguageModel.get_peft_model(
12
+ model,
13
+ r = 64, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
14
+
15
+ target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
16
+ "gate_proj", "up_proj", "down_proj",],
17
+
18
+ lora_alpha = 8,
19
+
20
+ lora_dropout = 0.2, # Supports any, but = 0 is optimized
21
+
22
+ bias = "none", # Supports any, but = "none" is optimized
23
+ use_gradient_checkpointing = "unsloth",
24
+ random_state = 3407,
25
+ use_rslora = False, #
26
+ loftq_config = None, # And LoftQ
27
+ )
28
 
29
 
30
  ## Model Details