Spaces:

Steph254
/

demo_1

Runtime error

Steph254 commited on Mar 18

Commit

3e8ef05

verified ·

1 Parent(s): cbfbc0d

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -25,7 +25,7 @@ def load_llama_model(model_name, is_guard=False):
     model = AutoModelForCausalLM.from_pretrained(
         model_name,
         token=HUGGINGFACE_TOKEN,
-        torch_dtype=torch.float16,  # Use float16 for optimized performance
         low_cpu_mem_usage=True
     )

     model = AutoModelForCausalLM.from_pretrained(
         model_name,
         token=HUGGINGFACE_TOKEN,
+        torch_dtype=torch.float32,
         low_cpu_mem_usage=True
     )