README.md

metadata

license: apple-amlr
license_link: https://huggingface.co/apple/LensVLM-9B/blob/main/LICENSE
base_model_relation: quantized
language: en
library_name: mlx
pipeline_tag: image-text-to-text
tags:
  - mlx
  - text-generation-inference
  - apple-silicon
  - 4-bit
  - 8-bit
  - mlx-vlm
base_model:
  - apple/LensVLM-9B

LensVLM-9B-MLX

LensVLM-9B is a 9-billion-parameter vision-language model from Apple, built on Qwen3.5-9B, introduced in the paper "LensVLM: Selective Context Expansion for Compressed Visual Representation of Text." Its core mechanism scans a compressed image representation of text — at configurable compression ratios of 5x, 10x, or 15x — and then selectively expands only the pages relevant to a given question back to their uncompressed form via learned tools, allowing the model to process very long documents without holding the entire uncompressed text in context. It's run via the accompanying ml-lensvlm codebase with a simple demo script accepting a text file and a question, and is released under the Apple Machine Learning Research Model License (with the accompanying source code separately licensed under the Apple Sample Code License).

+-- prithivMLmods/LensVLM-9B-MLX (main)
    +-- / (Root: bf16)
    +-- 4bit/ (Quantized: 4-bit)
    +-- 8bit/ (Quantized: 8-bit)

Use with mlx

Install the required library:

pip install -U mlx-vlm

Model Note: LensVLM-9B is a 9-billion parameter vision-language model developed by Apple, specialized in compressed visual text processing and document-level multi-modal understanding.

BF16 Variant (Base Weights)

The unquantized BF16 weights reside directly in the root repository directory:

CLI (Terminal)

python -m mlx_vlm generate \
  --model prithivMLmods/LensVLM-9B-MLX \
  --max-tokens 512 \
  --temperature 0.0 \
  --prompt "Read and analyze the contents of this document in detail." \
  --image <path_to_image>

Python API

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path)
config = load_config(model_path)

image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))

output = generate(
    model, 
    processor, 
    formatted_prompt, 
    image=image, 
    max_tokens=512, 
    temperature=0.0
)
print(output.text)

8-bit Variant

Target the 8bit subfolder:

CLI (Terminal)

python -m mlx_vlm generate \
  --model prithivMLmods/LensVLM-9B-MLX/8bit \
  --max-tokens 512 \
  --temperature 0.0 \
  --prompt "Read and analyze the contents of this document in detail." \
  --image <path_to_image>

Python API

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path, subfolder="8bit")
config = load_config(model_path, subfolder="8bit")

image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))

output = generate(
    model, 
    processor, 
    formatted_prompt, 
    image=image, 
    max_tokens=512, 
    temperature=0.0
)
print(output.text)

4-bit Variant

Target the 4bit subfolder:

CLI (Terminal)

python -m mlx_vlm generate \
  --model prithivMLmods/LensVLM-9B-MLX/4bit \
  --max-tokens 512 \
  --temperature 0.0 \
  --prompt "Read and analyze the contents of this document in detail." \
  --image <path_to_image>

Python API

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "prithivMLmods/LensVLM-9B-MLX"
model, processor = load(model_path, subfolder="4bit")
config = load_config(model_path, subfolder="4bit")

image = ["<path_to_image>"]
prompt = "Read and analyze the contents of this document in detail."
formatted_prompt = apply_chat_template(processor, config, prompt, num_images=len(image))

output = generate(
    model, 
    processor, 
    formatted_prompt, 
    image=image, 
    max_tokens=512, 
    temperature=0.0
)
print(output.text)

License and Attribution

• Model: apple/LensVLM-9B
• Base Model: Qwen/Qwen3.5-9B
• License: This MLX compression follows the Apple AMLR License of the original LensVLM-9B model.
• MLX-VLM: https://github.com/Blaizzy/mlx-vlm
• MLX: https://github.com/ml-explore/mlx