MANO066 commited on
Commit
7cb8ea6
·
verified ·
1 Parent(s): ff9727c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md CHANGED
@@ -96,3 +96,75 @@ print("=============================")
96
  print(f"🔗 Reference Image: {IMAGE_URL}")
97
  print("=============================\n")
98
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  print(f"🔗 Reference Image: {IMAGE_URL}")
97
  print("=============================\n")
98
 
99
+
100
+ # Install OpenVINO Optimum for export & inference
101
+ pip install --upgrade --upgrade-strategy eager "optimum[openvino]"
102
+ ```
103
+
104
+
105
+ ## 🖼️ OpenVINO Export
106
+
107
+ ```bash
108
+ optimum-cli export openvino \
109
+ --model MANO066/Intervl3-1b-safety \
110
+ Intervl3-1b-safety \
111
+ --trust-remote-code \
112
+ --weight-format fp16
113
+ ```
114
+
115
+ ---
116
+
117
+ ## 🖼️ OpenVINO Inference Example
118
+
119
+ ```python
120
+ import requests
121
+ from PIL import Image
122
+ from io import BytesIO
123
+ import numpy as np
124
+ from pathlib import Path
125
+ from IPython.display import display
126
+ import openvino_genai as ov_genai
127
+
128
+ model_dir = "Intervl3-1b-safety"
129
+ ov_model = ov_genai.VLMPipeline(model_dir, device="CPU")
130
+
131
+ config = ov_genai.GenerationConfig()
132
+ config.max_new_tokens = 100
133
+
134
+ def load_image(image_file):
135
+ if isinstance(image_file, str) and image_file.startswith("http"):
136
+ response = requests.get(image_file)
137
+ image = Image.open(BytesIO(response.content)).convert("RGB")
138
+ else:
139
+ image = Image.open(image_file).convert("RGB")
140
+ image_data = np.array(image.getdata()).reshape(1, image.size[1], image.size[0], 3).astype(np.byte)
141
+ return image, ov.Tensor(image_data)
142
+
143
+ EXAMPLE_IMAGE = Path("example_image1.jpg")
144
+ EXAMPLE_IMAGE_URL = "https://huggingface.co/OpenGVLab/InternVL2-2B/resolve/main/examples/image1.jpg"
145
+
146
+ if not EXAMPLE_IMAGE.exists():
147
+ img_data = requests.get(EXAMPLE_IMAGE_URL).content
148
+ with EXAMPLE_IMAGE.open("wb") as handler:
149
+ handler.write(img_data)
150
+
151
+ def streamer(subword: str) -> bool:
152
+ print(subword, end="", flush=True)
153
+
154
+ question = "Please describe the image shortly"
155
+ image, image_tensor = load_image(EXAMPLE_IMAGE)
156
+ display(image)
157
+
158
+ print(f"User: {question}\n")
159
+ print("Assistant:")
160
+ output = ov_model.generate(question, image=image_tensor, generation_config=config, streamer=streamer)
161
+ ```
162
+
163
+ ---
164
+
165
+ ### ✅ Notes
166
+
167
+ * PyTorch + Transformers workflow is **GPU optimized**.
168
+ * OpenVINO workflow is **CPU optimized**, lightweight, and fast.
169
+ * OpenVINO export converts model weights to **FP16**, reducing memory usage.
170
+ * Choose the workflow depending on your hardware and inference needs.