vikhyatk commited on
Commit
638927a
·
verified ·
1 Parent(s): d1b7c10

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -9
README.md CHANGED
@@ -19,14 +19,21 @@ For more details, please refer to our ||coming soon release blog post||. Or try
19
 
20
  Load the model and prepare it for inference. We use [FlexAttention for inference](https://pytorch.org/blog/flexattention-for-inference/), so calling `.compile()` is critical for fast decoding. Our `compile` implementation also handles warmup, so you can start making requests directly once it returns.
21
 
22
- ```
23
- moondream = AutoModelForCausalLM.from_pretrained(
24
- "moondream/moondream3-preview",
25
- trust_remote_code=True,
26
- dtype=torch.bfloat16,
27
- device_map={"": "cuda"},
28
- )
29
- moondream.compile()
 
 
 
30
  ```
31
 
32
- * TODO: Add usage examples
 
 
 
 
 
19
 
20
  Load the model and prepare it for inference. We use [FlexAttention for inference](https://pytorch.org/blog/flexattention-for-inference/), so calling `.compile()` is critical for fast decoding. Our `compile` implementation also handles warmup, so you can start making requests directly once it returns.
21
 
22
+ ```python
23
+ import torch
24
+ from transformers import AutoModelForCausalLM
25
+
26
+ moondream = AutoModelForCausalLM.from_pretrained(
27
+ "moondream/moondream3-preview",
28
+ trust_remote_code=True,
29
+ dtype=torch.bfloat16,
30
+ device_map={"": "cuda"},
31
+ )
32
+ moondream.compile()
33
  ```
34
 
35
+ * TODO: Add usage examples
36
+ * Query
37
+ * Caption
38
+ * Detect
39
+ * Point