Image-Text-to-Text
MLX
Safetensors
cohere_compass
mlx-vlm
openmed
openmedkit
apple-silicon
on-device
vision
multimodal
clinical
medical
privacy
native-resolution
conversational
Instructions to use OpenMed/North-Micro-Vision-Instruct-bf16-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use OpenMed/North-Micro-Vision-Instruct-bf16-mlx with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("OpenMed/North-Micro-Vision-Instruct-bf16-mlx") config = load_config("OpenMed/North-Micro-Vision-Instruct-bf16-mlx") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
| { | |
| "family": "cohere-compass", | |
| "format_version": 1, | |
| "precision": "bf16", | |
| "quantization": null, | |
| "runtime": { | |
| "local_only_recommended": true, | |
| "mlx_vlm_revision": "dd79a5d8caf3edafd6fa9e6326d7ce4977ddcbfc", | |
| "python": "mlx-vlm", | |
| "swift": "OpenMedKit Cohere Compass runtime pending", | |
| "validated_multimodal_context": 8192 | |
| }, | |
| "source_model": "CohereLabs/North-Micro-Vision-Instruct", | |
| "source_revision": "373bda96ac70bf89f99f7048f420cf00dc07c149", | |
| "task": "image-text-to-text", | |
| "validation": { | |
| "cases": [ | |
| "text_privacy", | |
| "text_fact_extraction", | |
| "image_clinical_document", | |
| "image_chart" | |
| ], | |
| "report": "openmed-validation.json", | |
| "status": "passed", | |
| "strict_weight_load": true, | |
| "text_and_image": true | |
| }, | |
| "weights": { | |
| "format": "safetensors", | |
| "path": "model.safetensors" | |
| } | |
| } | |