Instructions to use Jeethu/North-Micro-Vision-Instruct-PARO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jeethu/North-Micro-Vision-Instruct-PARO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Jeethu/North-Micro-Vision-Instruct-PARO")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Jeethu/North-Micro-Vision-Instruct-PARO")
model = AutoModelForMultimodalLM.from_pretrained("Jeethu/North-Micro-Vision-Instruct-PARO", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Jeethu/North-Micro-Vision-Instruct-PARO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jeethu/North-Micro-Vision-Instruct-PARO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Jeethu/North-Micro-Vision-Instruct-PARO

SGLang

How to use Jeethu/North-Micro-Vision-Instruct-PARO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jeethu/North-Micro-Vision-Instruct-PARO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jeethu/North-Micro-Vision-Instruct-PARO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jeethu/North-Micro-Vision-Instruct-PARO",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Jeethu/North-Micro-Vision-Instruct-PARO with Docker Model Runner:
```
docker model run hf.co/Jeethu/North-Micro-Vision-Instruct-PARO
```

North-Micro-Vision-Instruct-PARO / README.md

Jeethu

Upload folder using huggingface_hub

5082a36 verified 9 days ago

preview code

Raw

History Blame Contribute Delete

4.42 kB

	---
	library_name: transformers
	license: apache-2.0
	pipeline_tag: image-text-to-text
	language:
	- en
	- de
	- fr
	- es
	- it
	- pt
	- hi
	- ja
	- ko
	- zh
	- ar
	tags:
	- vision
	- multimodal
	- conversational
	- multilingual
	- native-resolution
	base_model:
	- CohereLabs/North-Micro-Vision-Instruct
	base_model_relation: quantized
	---

	# Jeethu/North-Micro-Vision-Instruct

	Pairwise Rotation Quantization for Efficient Reasoning LLM Inference

	<p>
	<a href="https://arxiv.org/abs/2511.10645"><img src="https://img.shields.io/badge/arXiv-2511.10645-b31b1b.svg" alt="Paper"></a>
	<a href="https://paroquant.z-lab.ai"><img src="https://img.shields.io/badge/Blog-ParoQuant-blue" alt="Blog"></a>
	<a href="https://huggingface.co/collections/z-lab/paroquant"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Models-yellow" alt="Models"></a>
	<a href="https://pypi.org/project/paroquant/"><img src="https://img.shields.io/pypi/v/paroquant" alt="PyPI"></a>
	</p>

	ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant.

	Jeethu/North-Micro-Vision-Instruct is a 4-bit [CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) quantized with ParoQuant.

	## Evaluation

	The following evaluations were run on 2026-08-14 with deterministic greedy decoding. The source checkpoint was evaluated in its native BF16 dtype. This ParoQuant checkpoint uses INT4 language projections (group size 128, `krot=8`) with FP16 retained tensors, including the vision encoder. For context, the published [MLX affine 4-bit checkpoint](https://huggingface.co/mlx-community/North-Micro-Vision-Instruct-4bit) uses group size 64 with BF16 retained tensors.

	### Vision

	\| Benchmark \| Samples \| Source BF16 \| ParoQuant INT4 / FP16 \| MLX affine 4-bit / BF16 \|
	\| --- \| ---: \| ---: \| ---: \| ---: \|
	\| [ChartQA](https://huggingface.co/datasets/lmms-lab/ChartQA) relaxed accuracy \| 100 \| 81.00% \| 81.00% \| 81.00% \|
	\| [MMStar](https://huggingface.co/datasets/Lin-Chen/MMStar) accuracy \| 1,500 \| 50.53% \| 50.33% \| 51.07% \|

	ChartQA uses a fixed seed-0 stratified sample of 50 `human_test` and 50 `augmented_test` examples. Scoring follows the VLMEvalKit relaxed rule: case-insensitive exact text matching or a 5% relative tolerance for numeric answers. MMStar uses the complete validation split and extracts the selected option from deterministic generations.

	Against source BF16, ParoQuant changes ChartQA by 0.00 percentage points (paired bootstrap 95% CI: -3.00 to +3.00) and MMStar by -0.20 points (95% CI: -1.80 to +1.40). A targeted multi-image color-ordering, synthetic OCR, and object-counting smoke suite was also passed exactly by source BF16 and ParoQuant. MLX was semantically correct on all three cases and exact on two; its color response was verbose.

	### Text

	\| Benchmark \| Samples / tokens \| Source BF16 \| ParoQuant INT4 / FP16 \| MLX affine 4-bit / BF16 \|
	\| --- \| ---: \| ---: \| ---: \| ---: \|
	\| [WikiText-2](https://huggingface.co/datasets/Salesforce/wikitext) perplexity (lower is better) \| 32,704 tokens \| 30.882 \| 31.106 \| 33.506 \|
	\| [ARC-Challenge](https://huggingface.co/datasets/allenai/ai2_arc) accuracy \| 1,172 \| 73.21% \| 70.56% \| 69.88% \|
	\| [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) accuracy \| 2,000 \| 49.50% \| 48.20% \| 49.00% \|

	WikiText-2 perplexity uses 64 non-overlapping sequences of 512 tokens from the test split. ARC-Challenge uses the complete labeled test split. HellaSwag uses a fixed seed-0 sample from the validation split. ARC-Challenge and HellaSwag are zero-shot greedy chat multiple-choice evaluations with identical prompts and tokenization across backends; they are not canonical `lm-eval` log-likelihood scores.

	Against source BF16, the ParoQuant ARC-Challenge delta is -2.65 percentage points (paired bootstrap 95% CI: -4.18 to -1.11), while its HellaSwag delta is -1.30 points (95% CI: -2.80 to +0.20).

	Dataset revisions were pinned to `9e63b7df1592a1c2158e735cc1725454aef0d6d9` (ChartQA), `bc98d668301da7b14f648724866e57302778ab27` (MMStar), `210d026faf9955653af8916fad021475a3f00453` (ARC), `218ec52e09a7e7462a5400043bb9a69a41d06b76` (HellaSwag), and `b08601e04326c79dfdd32d625aee71d232d685c3` (WikiText).