WeVisOne-VL-GGML

GGUF exports of WeVisOne-VL models (2B and 4B variants) for efficient local inference with llama.cpp.

This repository contains all model variants organized by size.

Model Organization

2B Models

File	Type	Precision	Size
mmproj-wevisone-vlm-2b-f16.gguf	Multimodal Projector	FP16	0.76 GB
wevisone-mask-decoder-2b-f16.gguf	Mask Decoder	FP16	0.02 GB
wevisone-mask-decoder-2b-f32.gguf	Mask Decoder	FP32	0.04 GB
wevisone-vlm-2b-f16.gguf	Vision-Language Model	FP16	3.80 GB
wevisone-vlm-2b-q4_K_M.gguf	Vision-Language Model	Q4_K_M (4-bit)	1.20 GB
wevisone-vlm-2b-q8_0.gguf	Vision-Language Model	Q8_0 (8-bit)	2.02 GB

4B Models

File	Type	Precision	Size
mmproj-wevisone-vlm-4b-f16.gguf	Multimodal Projector	FP16	0.78 GB
wevisone-mask-decoder-4b-f16.gguf	Mask Decoder	FP16	0.02 GB
wevisone-mask-decoder-4b-f32.gguf	Mask Decoder	FP32	0.04 GB
wevisone-vlm-4b-f16.gguf	Vision-Language Model	FP16	8.23 GB
wevisone-vlm-4b-q4_K_M.gguf	Vision-Language Model	Q4_K_M (4-bit)	2.53 GB
wevisone-vlm-4b-q8_0.gguf	Vision-Language Model	Q8_0 (8-bit)	4.38 GB

Precision Tiers

VLM variants (merged mmproj + LLM, llama.cpp qwen3vl arch)

Tier	Intended use	Expected parity
FP16	Default serving tier	Token-identical in fixed-mode benchmarks
Q8_0	Memory-constrained serving (≈2x smaller)	Separate quantized tier, ratio ≤ 2.5 noise band
Q4_K_M	Highest cost/performance; batch/edge serving	Quality tier only — verify on parity suite

Pixel-decoder variants (arch wevisone_mask)

Tier	Intended use
FP32	The only parity-gated tier — released decoder executes in fp32 by construction
FP16	Memory-tight edge deployments; MAD may drift, gate before use

Note: Q4_K is deliberately NOT offered for the decoder: GroupNorm/MSDA statistics and the alpha-cascade residual path degrade beyond published MAD thresholds under 4-bit weights.

Usage

Running with llama.cpp

# Build llama.cpp with qwen3vl support
cd llama.cpp && make

# Run inference with 2B model
./llama-cli \
  -m wevisone-vlm-2b-q8_0.gguf \
  --mmproj mmproj-wevisone-vlm-2b-f16.gguf \
  --image your_image.jpg \
  -p "Describe this image."

# Run inference with 4B model
./llama-cli \
  -m wevisone-vlm-4b-q8_0.gguf \
  --mmproj mmproj-wevisone-vlm-4b-f16.gguf \
  --image your_image.jpg \
  -p "Describe this image."

Mask Decoder (Segmentation)

The mask decoder files (wevisone-mask-decoder-*.gguf) are used for segmentation tasks via the WeVisOne-VL C++ runtime.

Original Model

These GGUF exports are based on:

• tencent/WeVisOne-VL-2B
• tencent/WeVisOne-VL-4B

Model Card

For detailed information about parity acceptance criteria, benchmarking data, and technical specifications, please refer to the original MODEL_CARD.md.

License

This model is released under the same license as the original WeVisOne-VL model.