Update README.md
Browse files
README.md
CHANGED
|
@@ -19,14 +19,21 @@ For more details, please refer to our ||coming soon release blog post||. Or try
|
|
| 19 |
|
| 20 |
Load the model and prepare it for inference. We use [FlexAttention for inference](https://pytorch.org/blog/flexattention-for-inference/), so calling `.compile()` is critical for fast decoding. Our `compile` implementation also handles warmup, so you can start making requests directly once it returns.
|
| 21 |
|
| 22 |
-
```
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
|
|
|
|
|
|
|
|
|
| 30 |
```
|
| 31 |
|
| 32 |
-
* TODO: Add usage examples
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
Load the model and prepare it for inference. We use [FlexAttention for inference](https://pytorch.org/blog/flexattention-for-inference/), so calling `.compile()` is critical for fast decoding. Our `compile` implementation also handles warmup, so you can start making requests directly once it returns.
|
| 21 |
|
| 22 |
+
```python
|
| 23 |
+
import torch
|
| 24 |
+
from transformers import AutoModelForCausalLM
|
| 25 |
+
|
| 26 |
+
moondream = AutoModelForCausalLM.from_pretrained(
|
| 27 |
+
"moondream/moondream3-preview",
|
| 28 |
+
trust_remote_code=True,
|
| 29 |
+
dtype=torch.bfloat16,
|
| 30 |
+
device_map={"": "cuda"},
|
| 31 |
+
)
|
| 32 |
+
moondream.compile()
|
| 33 |
```
|
| 34 |
|
| 35 |
+
* TODO: Add usage examples
|
| 36 |
+
* Query
|
| 37 |
+
* Caption
|
| 38 |
+
* Detect
|
| 39 |
+
* Point
|