victorucd/EVIE-Preview-4.5B-MLX-4bit

EVIE-Preview-4.5B · MLX 4-bit

An unofficial Apple Silicon adaptation of Tencent EVIE-Preview-4.5B, converted by Victor Uceda (victorucd). This is a visual document retriever, not a chat or text-generation model. It returns one normalized 128-dimensional vector per token and ranks pages with directional MaxSim.

The Qwen3.5 language backbone uses 4-bit affine round-to-nearest quantization (group size 64). The vision tower and retrieval projection remain BF16. There is no calibration dataset, fine-tuning, or retraining.

Compatibility and architecture

• Tested with mlx-vlm==0.6.16 and mlx==0.32.2 on Apple Silicon.
• Standard mlx_vlm.load(...) loads the included evie_mlx.py through MLX-VLM's model_file mechanism.
• No monkeypatches of MLX-VLM, and no private package is required.
• This is bundled custom retrieval code, not yet a built-in MLX-VLM model family. Review the code and pin a Hub revision in reproducible/security-sensitive deployments; loading model_file executes that Python module.
• Full-attention layers are bidirectional; GatedDeltaNet retains the source model's recurrent computation.
• The public encoders process one item at a time, without padding or KV caches. Batched padded forward calls and text generation are rejected. Text and image queries are supported; video is not supported by this adapter.

Quick start

pip install "mlx-vlm==0.6.16" "mlx==0.32.2"

Download the checkpoint and use the included helper functions:

import sys
from huggingface_hub import snapshot_download
from mlx_vlm import load

model_id = "victorucd/EVIE-Preview-4.5B-MLX-4bit"
path = snapshot_download(model_id)  # add revision="<commit hash>" to pin
sys.path.insert(0, path)
from evie_mlx import encode_images, encode_queries, score

model, processor = load(path, strict=True)
model.eval()
documents = encode_images(model, processor, ["page_1.png", "page_2.png"])
queries = encode_queries(model, processor, ["What is the revenue for 2025?"])
scores = score(queries, documents)
print(scores)
print("Best page:", scores.argmax(axis=1))

load(model_id, strict=True) also works directly. The snapshot step above makes the helper module easy to import. For image-to-image retrieval, encode the query with encode_images, not encode_queries. visual_only=True optionally removes image prompt tokens; use the same explicit choice for query and corpus. The public text-to-image benchmark below retains all non-padding document tokens, following the source evaluation.

The CLI is also available after downloading:

python evie_mlx.py --model . --query "Quarterly revenue" --image page_1.png --image page_2.png

Public validation

The following datasets were used only for evaluation, not for training, fine-tuning, or quantization calibration: TabFQuAD V1 and Economics Reports V2.

Complete public task	Queries	Candidate pages	MLX Q4 nDCG@5	Source BF16 reported nDCG@5	End-to-end time
TabFQuAD V1	280	70	97.59	97.32	12.66 min
Economics Reports V2	232	452	68.43	68.53	25.00 min

Scores are on a 0–100 scale. The BF16 column is quoted from the pinned source model card, not a BF16 rerun on this machine. Differences are small and should not be interpreted as a statistically significant improvement or as a controlled estimate of quantization loss.

Measured on an Apple Silicon Mac, macOS 26.5.2, single-item encoding: 37.69 minutes for both tasks, including download, image/query encoding and scoring. Image encoding took 706.60 s and 1454.96 s; query encoding took 42.16 s and 27.79 s; scoring took 2.56 s and 1.96 s, respectively. Peak memory reported by MLX was 3.74 GiB (not total system RSS). The raw FP16 document embeddings occupied 41.36 MB and 85.78 MB; V1 includes its repeated page rows before aggregation.

The two queries on the source model's four public example pages selected pages 1 and 2, as expected. The release also passed strict weight loading, unit-norm checks, query-order invariance, exact masked MaxSim, and a check that no global MLX-VLM functions were modified. Detailed smoke scores are in smoke_test.json.

This is a small validation of two complete public tasks, not a full ViDoRe leaderboard submission or a claim of SOTA for the quantized model. Dataset revisions, metrics, timing and package versions are in eval_results.json. No private documents, datasets, or results are included in this release.

Protocol: 768-token visual budget, official query text plus ten <|endoftext|> augmentation tokens, full non-padding document embeddings, FP32 normalization and scoring, FP16 embedding storage. V1 uses all 280 released rows, deduplicated to 70 candidate pages by filename; V2 uses the complete released corpus and graded qrels. Queries are not subsampled.

To reproduce the pilot from a downloaded model directory:

pip install -r requirements.txt
python evaluate_vidore.py --model . --work ./vidore-pilot --tasks tabfquad economics --max-seconds 6900

The evaluator checks its time budget and stops if free disk space falls below 8 GiB. It downloads only the two selected datasets, not the approximately 55 GB hydrated full benchmark. Dataset files and generated embeddings are local outputs, not part of this model repository.

Conversion provenance

• Source: tencent/EVIE-Preview-4.5B@1c43543b000b33e458070e13eae6a344938d8aa1.
• Backbone: Qwen3.5-4B; retrieval design: ColPali Engine.
• Quantization: MLX-VLM RTN, affine 4-bit, group size 64, BF16 vision/projection.
• conversion.json records the source, versions, weight size and SHA256; SHA256SUMS.txt covers the release files.

To repeat the conversion on an Apple Silicon machine with at least 20 GiB free:

python convert_evie.py --output ./EVIE-Preview-4.5B-MLX-4bit --work ./conversion-cache

The script uses the pinned public source and refuses to overwrite an existing checkpoint.

Limitations

Quantization can change rankings. Only the listed tasks and public smoke test were run for this release; the source model's broader benchmark claims must not be attributed to this Q4 checkpoint. Relevance scores are uncalibrated and are meaningful for ranking candidates for the same query, not as probabilities or cross-query confidence values. Runtime depends on page resolution, thermal state, unified memory and concurrent workload. The model is intended for retrieval research; evaluate domain-specific quality and downstream risks before deployment.

Attribution and license

Apache-2.0. Original model credit belongs to Tencent; backbone credit to Qwen; retrieval tooling credit to ILLUIN/ColPali Engine. This adaptation relies on Apple MLX and MLX-VLM. See NOTICE.txt and LICENSE.txt. This personal release is not an official Tencent or mlx-community release.

@misc{tencent2026evie,
  title = {EVIE-Preview-4.5B},
  author = {{Tencent}},
  year = {2026},
  howpublished = {\url{https://huggingface.co/tencent/EVIE-Preview-4.5B}}
}
@misc{uceda2026eviemlx,
  title = {EVIE-Preview-4.5B MLX 4-bit adaptation},
  author = {Victor Uceda},
  year = {2026},
  howpublished = {\url{https://huggingface.co/victorucd/EVIE-Preview-4.5B-MLX-4bit}}
}