RanjithaRuttala
/

PEFT_FP16_starcoder2-3b

Model card Files Files and versions

RanjithaRuttala commited on Dec 3, 2025

Commit

c4ffa19

·

verified ·

1 Parent(s): 4791ca0

Update handler.py

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -20,7 +20,7 @@ class EndpointHandler:
             torch_dtype=torch.bfloat16,     # ✅ Changed to bfloat16
             trust_remote_code=True,
             device_map="auto",
-            attn_implementation="flash_attention_2"  # ✅ Faster + stable
         )
         self.model.eval()
         print("✅ Model loaded successfully!")

             torch_dtype=torch.bfloat16,     # ✅ Changed to bfloat16
             trust_remote_code=True,
             device_map="auto",
+            # attn_implementation="flash_attention_2"  # ✅ Faster + stable
         )
         self.model.eval()
         print("✅ Model loaded successfully!")