sixfingerdev commited on
Commit
48f3b0a
·
verified ·
1 Parent(s): a59c10d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -10
README.md CHANGED
@@ -74,20 +74,31 @@ Ensure you have a GPU with sufficient VRAM for 4-bit inference.
74
  ## Example Usage
75
 
76
  Generate text using the adapter:
 
 
 
 
77
 
78
- 'prompt = "Soru: Yapay zeka nedir?\nCevap:"'
79
- 'inputs = tokenizer(prompt, return_tensors="pt")'
 
 
 
80
 
81
- 'with torch.no_grad():'
82
- ' outputs = model.generate('
83
- ' **inputs,'
84
- ' max_new_tokens=50,'
85
- ' do_sample=True,'
86
- ' temperature=0.7'
87
- ' )'
88
 
89
- 'print(tokenizer.decode(outputs[0], skip_special_tokens=True))'
 
90
 
 
 
 
 
 
 
 
 
91
  ---
92
 
93
  ## Notes
 
74
  ## Example Usage
75
 
76
  Generate text using the adapter:
77
+ ```python
78
+ from transformers import AutoTokenizer, AutoModelForCausalLM
79
+ from peft import PeftModel
80
+ import torch
81
 
82
+ # Base model
83
+ base_model = AutoModelForCausalLM.from_pretrained(
84
+ "unsloth/llama-3.1-8b-bnb-4bit",
85
+ device_map="auto"
86
+ )
87
 
88
+ # LoRA adapter
89
+ model = PeftModel.from_pretrained(base_model, "sixfingerdev/SixFinger-8B")
 
 
 
 
 
90
 
91
+ # Tokenizer
92
+ tokenizer = AutoTokenizer.from_pretrained("unsloth/llama-3.1-8b-bnb-4bit")
93
 
94
+ # Örnek text generation
95
+ prompt = "Soru: Yapay zeka nedir?\nCevap:"
96
+ inputs = tokenizer(prompt, return_tensors="pt")
97
+ with torch.no_grad():
98
+ outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)
99
+
100
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
101
+ ```
102
  ---
103
 
104
  ## Notes