ZhiyangQi97 commited on
Commit
53dfd76
·
verified ·
1 Parent(s): 4518ac1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -3
README.md CHANGED
@@ -40,19 +40,23 @@ The base model is [tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3](https://hug
40
 
41
  ---
42
 
43
- ## ⚙️ Usage Instructions
44
 
45
- This repository contains **only the LoRA adapter**. You must load the original base model and then apply this adapter:
 
46
 
47
  ```python
48
  from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
49
  from peft import PeftModel
50
 
 
51
  base_model_id = "tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3"
52
- adapter_id = "your-username/kokorochat-high-lora"
53
 
 
54
  tokenizer = AutoTokenizer.from_pretrained(base_model_id)
55
 
 
56
  base_model = AutoModelForCausalLM.from_pretrained(
57
  base_model_id,
58
  device_map="auto",
@@ -60,6 +64,7 @@ base_model = AutoModelForCausalLM.from_pretrained(
60
  quantization_config=BitsAndBytesConfig(load_in_4bit=True)
61
  )
62
 
 
63
  model = PeftModel.from_pretrained(base_model, adapter_id)
64
  model = model.merge_and_unload()
65
  ```
 
40
 
41
  ---
42
 
43
+ ## ⚙️ Usage Instructions (LoRA Adapter)
44
 
45
+ This repository only contains the **adapter weights**.
46
+ You must load the original base model and then apply this adapter.
47
 
48
  ```python
49
  from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
50
  from peft import PeftModel
51
 
52
+ # === Base + Adapter Paths ===
53
  base_model_id = "tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3"
54
+ adapter_id = "your-username/kokorochat-lora"
55
 
56
+ # === Load Tokenizer ===
57
  tokenizer = AutoTokenizer.from_pretrained(base_model_id)
58
 
59
+ # === Load Base Model ===
60
  base_model = AutoModelForCausalLM.from_pretrained(
61
  base_model_id,
62
  device_map="auto",
 
64
  quantization_config=BitsAndBytesConfig(load_in_4bit=True)
65
  )
66
 
67
+ # === Load & Merge LoRA ===
68
  model = PeftModel.from_pretrained(base_model, adapter_id)
69
  model = model.merge_and_unload()
70
  ```