Meet North Micro Vision: A 2.4B Native-Resolution Vision-Language Model

Community Article

Published August 12, 2026

Upvote

All co-authors listed below

Today we are releasing North-Micro-Vision-Instruct, a 2.4B-parameter open-weight vision-language model (VLM) with native-resolution image support, under the Apache 2.0 license.

North Micro Vision is our smallest VLM to date. It is designed as a compact foundation for specialized multimodal applications, combining broad image-understanding capabilities with native-resolution support. Its 2.4B-parameter scale makes it practical to fine-tune for different visual domains and workflows. With the right inference stack and quantization, models at this scale can also support experimentation beyond server-only deployments, including on laptops and edge or mobile-class hardware.

The release reflects Cohere's broader work on sovereign AI by pairing model development with clear licensing, open weights, and transparent evaluation.

Model weights 🤗: CohereLabs/North-Micro-Vision-Instruct
Public vLLM support: Coming soon.

Figure 1. North Micro Vision's performance across core vision capabilities compared with other compact open-weight models. Scores are averaged across the benchmarks in each capability group; see the benchmarking methodology and detailed results.

What North Micro Vision Is Built For

Capability at a compact scale: North Micro Vision balances broad visual capabilities with a size suited to local, edge-aware, and specialized deployments.
Dense, native-resolution visual inputs: The model preserves the aspect ratio and fine detail of documents, tables, charts, screenshots, and forms instead of first reducing every input to a small square image.
Multilingual visual understanding: Training covers multiple languages and visual domains—including documents, charts, and natural images—supporting multilingual evaluation and domain adaptation.
Fine-tuning and customization: The model provides a compact foundation for adaptation to domain-specific data, tasks, and deployment constraints.

Architecture

These goals shaped a three-part architecture: a native-resolution vision encoder, a projector, and a compact language model.

Figure 2. High-level North Micro Vision architecture, consisting of a native-resolution vision encoder, a projector, and a language model.

North Micro Vision combines a custom-trained 400M-parameter native-resolution vision encoder with an in-house 2B-parameter language model North Micro LLM. The language model follows our Command A+ architecture [1], interleaving three sliding-window attention layers that use rotary positional embeddings with one global attention layer without positional embeddings [2]. The vision encoder combines 2D RoPE with learned 1D positional embeddings to preserve spatial structure across native-resolution inputs.

The projector maps visual features into the language model's embedding space. Following DeepStack [3], patch embeddings from multiple vision-encoder layers are injected into corresponding early LLM layers, giving the language model access to visual representations at different levels of abstraction.

VLM Training Process

We trained North Micro Vision in four stages, with Stage 2 split into two resolution phases. Stage 1 adapted the vision encoder and projector; Stages 2.1 and 2.2 increased resolution while jointly training the encoder, projector, and language model; Stage 3 instruction-tuned the full model; and Stage 4 used a simplified variant of Mixed Preference Optimization (MPO) to improve safety, formatting, and response quality.

Native-Resolution Vision Encoder Training

We continued pre-training the vision encoder from the SigLIP 2 SO400M checkpoint google/siglip2-so400m-patch16-384 [5]. The curriculum progressively increased image resolution and context length and incorporated Continual Rotary Position Embedding (C-RoPE)—a combination of 2D RoPE and bilinearly interpolated learned 1D positional embeddings—to support native-resolution inputs [4].

The native-resolution encoder curriculum comprised three stages:

Stage 1 (10M examples): We started at a fixed resolution of 384 × 384 pixels, the default input resolution of the base SigLIP 2 SO400M model. Keeping the language model frozen, we trained only the vision encoder and vision-language projector using a mixture of dense captions (60%) and OCR data (40%).
Stage 2.1 (13M examples): We increased the resolution to 1024 × 1024 and trained the vision encoder, projector, and language model together. The data mixture was evenly split between dense captions and OCR examples.
Stage 2.2 (10M examples): We increased the native-resolution cap to 1654 × 2339 pixels, corresponding to an A4 page at 200 dpi. We retained the 50/50 data mixture and continued training the full model. The resulting aligned checkpoint can process a single A4 document page at up to 200 dpi while preserving its aspect ratio.

Across these stages, next-token prediction remained the primary objective, with patch-token alignment [4] included as an auxiliary loss weighted at 0.05 relative to next-token prediction. The alignment objective encouraged image-patch features to capture language-relevant information.

Figure 3. ImageNet-1K classification performance of the North Micro Vision encoder (North-Micro-Vision VE) compared with other vision encoders at native resolution.

This OCR- and document-heavy curriculum produced an encoder that is strong at document understanding while preserving useful representations for natural images (Figure 3).

Multimodal Instruction Tuning

We then instruction-tuned the aligned model at native resolution using a diverse mixture of multimodal instruction-following data:

Native OCR:                17.8%
Charts and tables:         17.8%
Grounding and counting:    13.3%
OCR QA:                    13.3%
General VQA:               11.2%
Captioning and knowledge:   8.9%
Text-only:                  8.9%
Mathematics:                4.4%
Science:                    4.4%

Stage 3 (50M examples): The mixture emphasized OCR, charts and tables, visual grounding and counting, and general VQA, while retaining text-only data to help preserve language performance. We jointly trained the vision encoder, projector, and language model using next-token prediction.

Longer native-resolution training in Stage 3 improved performance across many evaluated capabilities, with the largest gains in document understanding and visual grounding. Document tasks benefit from preserving small text and layout details, while grounding benefits from retaining precise spatial information.

Preference Tuning

Stage 4 (500k examples): We applied a simplified variant of Mixed Preference Optimization (MPO), inspired by Wang et al. [6]. The original MPO objective combines a DPO preference loss, a BCO quality loss, and an SFT generation loss. We found that a simpler combination worked well in our setting, so we dropped the BCO quality loss and combined DPO with an auxiliary next-token prediction/SFT loss weighted at 15% relative to DPO. Using a diverse set of binary multimodal preferences, we trained only the language model while keeping the vision encoder and projector frozen.

This final stage introduced safety training, improved formatting, and made lightweight chat refinements. Across the full curriculum, the model first learned to preserve and align fine-grained visual information, then to follow multimodal instructions, and finally to favor preferred responses.

Training Data

The curriculum drew on publicly available datasets and an in-house, large-scale multilingual document corpus. The in-house document corpus supported the synthesis and curation of data for OCR, document understanding, chart understanding, captioning, HTML table generation, and visual grounding across languages.

For grounding examples, we normalized bounding boxes from all datasets to [x1,y1,x2,y2] on a 0–1000 scale before training. This provided a consistent representation across source datasets that use different coordinate formats.

Data Ablations

Vision data quality: We ablated both data-quality filtering and the training mixture. For quality filtering, an internal judging pipeline scored sampled examples and identified batch-level issues such as low diversity, repeated systematic errors, extraction artifacts, noisy formatting, weak grounding, and low overall training-signal quality. After we removed the lowest-scoring data, applying increasingly aggressive thresholds had only a marginal effect on final performance.

Vision data mix: Small changes to the recipe did not produce large differences as long as the mixture remained balanced across capabilities, languages, and domains. After low-quality examples had been removed, coverage and balance mattered more than narrowly tuning the proportions.

Benchmarking Methodology

Vision encoder evaluation. To check that our document-heavy training preserved general visual representations, we evaluated the frozen vision encoder on ImageNet-1K [4]. We trained an attentive-pooling classifier over native-resolution final-layer features without resizing images to fixed dimensions. This provides a complementary evaluation on natural images.

Model evaluation. We evaluated North Micro Vision across standard vision-language benchmarks covering general VQA, multilingual and multi-image understanding, chart and document understanding, OCR, STEM, grounding and counting, robustness and hallucination, and text capabilities. The suite measures both the tasks most likely to benefit from native-resolution inputs and the broader capabilities expected from a general-purpose VLM.

Figure 1 reports the average score for each capability group using the following benchmarks:

General VQA: MMBench (DEV_EN_V11), MMStar, RealWorldQA, and GQA (TestDev_Balanced).
Multilingual: Multilingual MMBench (DEV) and MMMB.
Multi-image: BLINK.
Chart / Document / OCR: ChartQA (Test), DocVQA (VAL), InfoVQA (VAL), OCRBench v2 (en), OCRBench, AI2D (TEST), and CharXiv (DQ).
STEM: MMMU (DEV_VAL).
Grounding / Counting: RefCOCO (avg) and CountBench.
Robustness / Hallucination: HallusionBench.

We ran evaluations with VLMEvalKit [7]. We evaluated North Micro Vision with an internal vLLM implementation; its recommended inference settings are temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, and repetition_penalty=1.0. Public vLLM support is coming soon. For each comparison model, we used its vLLM implementation where available and otherwise used Hugging Face inference, following the authors' recommended vision and text inference settings and maximum image resolutions. Because the comparison focuses on instruction-tuned, non-reasoning models, we capped generation at 1,024 tokens.

For text-only evaluations, we ported the OpenCompass tasks [8] into VLMEvalKit so that they could run through the same evaluation harness.

Benchmark Results

The detailed results below report per-benchmark scores across the capability groups described above, together with text-only evaluations.

	North-Micro-Vision-Instruct	Ministral-3-3B-Instruct	LFM2.5-VL-1.6B	Phi-3.5-vision-instruct	Gemma-4-E2B-it	Qwen3-VL-2B-Instruct	Qwen3.5-2B-Instruct	SmolVLM2.2B
Size	2.4B	3.8B	1.6B	4.2B	5.1B	2.2B	2.1B	2.2B
License	Apache 2.0	Apache 2.0	LFM v1.0	MIT	Apache 2.0	Apache 2.0	Apache 2.0	Apache 2.0
General VQA
MMBench_{DEV_EN_V11}	0.687	0.692	0.696	0.731	0.693	0.744	0.760	0.674
MMStar	0.518	0.531	0.508	0.495	0.529	0.506	0.614	0.460
RealWorldQA	0.622	0.583	0.642	0.580	0.507	0.646	0.693	0.567
GQA_{TestDev_Balanced}	0.574	0.544	0.395	0.650	0.387	0.572	0.539	0.000^‡
Multilingual
MTL_{MMBench_DEV}	0.636	0.674	0.623	0.619	0.648	0.664	0.669	0.454
MMMB	0.728	0.734	0.717	0.686	0.743	0.723	0.745	0.577
Multi-image
BLINK	0.527	0.471	0.484	0.561	0.468	0.514	0.563	0.420
Chart / Document / OCR
ChartQA_Test	0.808	0.791	0.739	0.821	0.422	0.693	0.775	0.682
DocVQA_VAL	0.921	0.896	0.877	0.860	0.732	0.825	0.926	0.799
InfoVQA_VAL	0.652	0.589	0.627	0.561	0.380	0.622	0.731	0.383
OCRBench_{v2_en}	0.367	0.414	0.415	0.339	0.435	0.417	0.481	0.304
OCRBench	0.792	0.735	0.802	0.642	0.719	0.751	0.861	0.727
AI2D_TEST	0.775	0.741	0.728	0.790	0.712	0.713	0.752	0.697
CharXiv_DQ	0.600	0.766	0.516	0.637	0.751	0.595	0.761	0.482
STEM
MMMU_{DEV_VAL}	0.329	0.508	0.380	0.432	0.477	0.379	0.474	0.399
Grounding / Counting
RefCOCO_avg^†	0.732	0.317	0.581	0.451	0.084	0.304	0.785	0.018
CountBench	0.725	0.737	0.910	0.645	0.534	0.848	0.805	0.764
Robustness / Hallucination
HallusionBench	0.615	0.652	0.601	0.585	0.598	0.673	0.655	0.600
Text
MMLU_test	0.504	0.660	0.464	0.355	0.692	0.630	0.543	0.084
MMLU-Pro_test	0.307	0.475	0.199	0.286	0.441	0.428	0.298	0.099
Multi-If	0.373	0.470	0.443	0.304	0.687	0.523	0.464	0.236
IFEval	0.749	0.725	0.776	0.543	0.869	0.734	0.679	0.501

^† P@1 over averaged over RefCOCO_val, RefCOCO_testA, RefCOCO_testB, RefCOCO+_val, RefCOCO+_testA, RefCOCO+_testB, RefCOCOg_val, RefCOCOg_test.

^‡ SmolVLM2.2B's GQA output was scored as 0.000 under VLMEvalKit's answer-extraction rules.

Conclusion

North Micro Vision combines a compact 2.4B-parameter architecture with native-resolution image processing and a training curriculum focused on documents, charts, OCR, and grounding. Its benchmark profile is strongest on document understanding and visual grounding, while its open weights and Apache 2.0 license provide a practical foundation for further adaptation. Explore the model and weights on Hugging Face.

Ecosystem Support

Fast Inference 🚀

MLX-VLM model weights - Community-contributed by Prince Canuma and Neywa.

Fine-tuning

In partnership with NVIDIA, we're also shipping an AutoModel recipe for North Micro Vision, so developers can fine-tune and deploy it on NVIDIA GPUs right out of the box.

NVIDIA AutoModel recipe - Fine-tune and deploy North Micro Vision on NVIDIA GPUs.
Axolotl fine-tuning support - Community-supported fine-tuning using the Axolotl framework.

Extended Author List

North Micro Vision Team: David Rau, Florian Schneider

Pre-training Team: Alex McKinney, Bas Büller, Björn Bebensee, David Cairuz da Silva, Dwarak Talupuru, Felipe Cruz Salinas, Giannis Chatziveroglou, Kilian Haefeli, Nikolas Gritsch, Prashant Kumar, Roman Castagné, Sammie Bae, Sylvie Shi, Tim Chung, Tony Liu (alphabetical order)

Inference Code: Clifton Poth

Product Management: Dennis Zhao

Product Marketing: Tim Ang

Management and Leadership: Nils Reimers, Carlos Lassance, Acyr Locatelli, Nick Frosst, Phil Blunsom, Joelle Pineau, Ivan Zhang, Aidan Gomez

Citation

@misc{cohere_north_micro_vision_instruct,
    title = {{North Micro Vision}: A 2.4B Native-Resolution Vision-Language Model},
    url = {https://huggingface.co/blog/CohereLabs/meet-north-micro-vision-instruct},
    author = {{Team Cohere}},
    month = {August},
    year = {2026}
}