arnavgrg
/

codallama-7b-instruct-nf4-fp16-upscaled

Text Generation

text-generation-inference

Model card Files Files and versions

arnavgrg commited on Dec 5, 2023

Commit

70ff731

·

1 Parent(s): 172cd26

Update README.md

Files changed (1) hide show

README.md +5 -1

README.md CHANGED Viewed

@@ -15,5 +15,9 @@ To use this model, you can just load it via `transformers` in fp16:
 import torch
 from transformers import AutoModelForCausalLM
-model = AutoModelForCausalLM.from_pretrained("arnavgrg/codallama-7b-instruct-nf4-fp16-upscaled", torch_dtype=torch.float16)
 ```

 import torch
 from transformers import AutoModelForCausalLM
+model = AutoModelForCausalLM.from_pretrained(
+  "arnavgrg/codallama-7b-instruct-nf4-fp16-upscaled",
+  device_map="auto",
+  torch_dtype=torch.float16
+)
 ```