muzerai commited on
Commit
8b98ab7
·
verified ·
1 Parent(s): 6b22f1b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -25,6 +25,44 @@ The following models were included in the merge:
25
 
26
  This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ```
29
  import torch
30
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
25
 
26
  This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method
27
 
28
+ Cuda
29
+
30
+ ```
31
+ from transformers import AutoModelForCausalLM, AutoTokenizer
32
+
33
+ model_name = "muzerai/SmolLM3-3B-Merged-AIJOAH"
34
+ device = "cuda" # for GPU usage or "cpu" for CPU usage
35
+
36
+ # load the tokenizer and the model
37
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
38
+ model = AutoModelForCausalLM.from_pretrained(
39
+ model_name,
40
+ ).to(device)
41
+
42
+ # prepare the model input
43
+ prompt = "Give me a brief explanation of gravity in simple terms."
44
+ messages_think = [
45
+ {"role": "user", "content": prompt}
46
+ ]
47
+
48
+ text = tokenizer.apply_chat_template(
49
+ messages_think,
50
+ tokenize=False,
51
+ add_generation_prompt=True,
52
+ )
53
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
54
+
55
+ # Generate the output
56
+ generated_ids = model.generate(**model_inputs, max_new_tokens=32768)
57
+
58
+ # Get and decode the output
59
+ output_ids = generated_ids[0][len(model_inputs.input_ids[0]) :]
60
+ print(tokenizer.decode(output_ids, skip_special_tokens=True))
61
+
62
+ ```
63
+
64
+
65
+ MAC
66
  ```
67
  import torch
68
  from transformers import AutoModelForCausalLM, AutoTokenizer