Spaces:

daniloedu
/

different_llms

Runtime error

daniloedu commited on Aug 3, 2023

Commit

b96dda2

1 Parent(s): 306c046

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -15,7 +15,7 @@ class Client:
             model_name,
             device_map=device_map,
             torch_dtype=torch.float16,
-            load_in_8bit=True,
             rope_scaling={"type": "dynamic", "factor": 2}
         )

             model_name,
             device_map=device_map,
             torch_dtype=torch.float16,
+            load_in_8bit=False,
             rope_scaling={"type": "dynamic", "factor": 2}
         )