RushabhShah122000 commited on
Commit
94ae6b0
·
verified ·
1 Parent(s): 1cc90be

Added code to run

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -19,4 +19,24 @@ language:
19
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
19
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
+
23
+ # Code to run
24
+ ```python
25
+ from llama_cpp import Llama
26
+
27
+ llm = Llama.from_pretrained(
28
+ repo_id="RushabhShah122000/model",
29
+ filename="unsloth.Q8_0.gguf",
30
+ )
31
+
32
+ output = llm(
33
+ "Bedtime story",
34
+ max_tokens=512,
35
+ echo=True
36
+ )
37
+ story_text = output['choices'][0]['text']
38
+ print(story_text)
39
+ ```
40
+
41
+
42
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)