Use `attn_implementation` instead of `_attn_implementation`
#21
by
qubvel-hf
- opened
README.md
CHANGED
|
@@ -84,7 +84,7 @@ processor = AutoProcessor.from_pretrained(model_path)
|
|
| 84 |
model = AutoModelForImageTextToText.from_pretrained(
|
| 85 |
model_path,
|
| 86 |
torch_dtype=torch.bfloat16,
|
| 87 |
-
|
| 88 |
).to("cuda")
|
| 89 |
```
|
| 90 |
|
|
|
|
| 84 |
model = AutoModelForImageTextToText.from_pretrained(
|
| 85 |
model_path,
|
| 86 |
torch_dtype=torch.bfloat16,
|
| 87 |
+
attn_implementation="flash_attention_2"
|
| 88 |
).to("cuda")
|
| 89 |
```
|
| 90 |
|