---
library_name: transformers
license: apache-2.0
pipeline_tag: image-text-to-text
language:
  - en
  - de
  - fr
  - es
  - it
  - pt
  - hi
  - ja
  - ko
  - zh
  - ar
tags:
  - vision
  - multimodal
  - conversational
  - multilingual
  - native-resolution
base_model:
  - CohereLabs/North-Micro-Vision-Instruct
base_model_relation: quantized
---

# Jeethu/North-Micro-Vision-Instruct

**Pairwise Rotation Quantization for Efficient Reasoning LLM Inference**

<p>
  <a href="https://arxiv.org/abs/2511.10645"><img src="https://img.shields.io/badge/arXiv-2511.10645-b31b1b.svg" alt="Paper"></a>
  <a href="https://paroquant.z-lab.ai"><img src="https://img.shields.io/badge/Blog-ParoQuant-blue" alt="Blog"></a>
  <a href="https://huggingface.co/collections/z-lab/paroquant"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Models-yellow" alt="Models"></a>
  <a href="https://pypi.org/project/paroquant/"><img src="https://img.shields.io/pypi/v/paroquant" alt="PyPI"></a>
</p>

ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant.

Jeethu/North-Micro-Vision-Instruct is a 4-bit [CohereLabs/North-Micro-Vision-Instruct](https://huggingface.co/CohereLabs/North-Micro-Vision-Instruct) quantized with ParoQuant.

## Evaluation

The following evaluations were run on 2026-08-14 with deterministic greedy decoding. The source checkpoint was evaluated in its native BF16 dtype. This ParoQuant checkpoint uses INT4 language projections (group size 128, `krot=8`) with FP16 retained tensors, including the vision encoder. For context, the published [MLX affine 4-bit checkpoint](https://huggingface.co/mlx-community/North-Micro-Vision-Instruct-4bit) uses group size 64 with BF16 retained tensors.

### Vision

| Benchmark | Samples | Source BF16 | ParoQuant INT4 / FP16 | MLX affine 4-bit / BF16 |
| --- | ---: | ---: | ---: | ---: |
| [ChartQA](https://huggingface.co/datasets/lmms-lab/ChartQA) relaxed accuracy | 100 | 81.00% | 81.00% | 81.00% |
| [MMStar](https://huggingface.co/datasets/Lin-Chen/MMStar) accuracy | 1,500 | 50.53% | 50.33% | 51.07% |

ChartQA uses a fixed seed-0 stratified sample of 50 `human_test` and 50 `augmented_test` examples. Scoring follows the VLMEvalKit relaxed rule: case-insensitive exact text matching or a 5% relative tolerance for numeric answers. MMStar uses the complete validation split and extracts the selected option from deterministic generations.

Against source BF16, ParoQuant changes ChartQA by 0.00 percentage points (paired bootstrap 95% CI: -3.00 to +3.00) and MMStar by -0.20 points (95% CI: -1.80 to +1.40). A targeted multi-image color-ordering, synthetic OCR, and object-counting smoke suite was also passed exactly by source BF16 and ParoQuant. MLX was semantically correct on all three cases and exact on two; its color response was verbose.

### Text

| Benchmark | Samples / tokens | Source BF16 | ParoQuant INT4 / FP16 | MLX affine 4-bit / BF16 |
| --- | ---: | ---: | ---: | ---: |
| [WikiText-2](https://huggingface.co/datasets/Salesforce/wikitext) perplexity (lower is better) | 32,704 tokens | 30.882 | **31.106** | 33.506 |
| [ARC-Challenge](https://huggingface.co/datasets/allenai/ai2_arc) accuracy | 1,172 | 73.21% | **70.56%** | 69.88% |
| [HellaSwag](https://huggingface.co/datasets/Rowan/hellaswag) accuracy | 2,000 | 49.50% | 48.20% | 49.00% |

WikiText-2 perplexity uses 64 non-overlapping sequences of 512 tokens from the test split. ARC-Challenge uses the complete labeled test split. HellaSwag uses a fixed seed-0 sample from the validation split. ARC-Challenge and HellaSwag are zero-shot greedy chat multiple-choice evaluations with identical prompts and tokenization across backends; they are not canonical `lm-eval` log-likelihood scores.

Against source BF16, the ParoQuant ARC-Challenge delta is -2.65 percentage points (paired bootstrap 95% CI: -4.18 to -1.11), while its HellaSwag delta is -1.30 points (95% CI: -2.80 to +0.20).

Dataset revisions were pinned to `9e63b7df1592a1c2158e735cc1725454aef0d6d9` (ChartQA), `bc98d668301da7b14f648724866e57302778ab27` (MMStar), `210d026faf9955653af8916fad021475a3f00453` (ARC), `218ec52e09a7e7462a5400043bb9a69a41d06b76` (HellaSwag), and `b08601e04326c79dfdd32d625aee71d232d685c3` (WikiText).