File size: 2,871 Bytes
b49d8d3 099942b b49d8d3 099942b b49d8d3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: apache-2.0
base_model: naver/v-splade-efficient
tags:
- mlx
- v-splade
- splade
- visual-document-retrieval
- sparse-retrieval
- multimodal
pipeline_tag: visual-document-retrieval
---
# v-splade-efficient-mlx
MLX (bfloat16) conversion of [`naver/v-splade-efficient`](https://huggingface.co/naver/v-splade-efficient)
(V-SPLADE, [arXiv:2605.30917](https://arxiv.org/abs/2605.30917)) for Apple Silicon,
produced by [NomaDamas/SPLADE-mlx](https://github.com/NomaDamas/SPLADE-mlx).
V-SPLADE is an inference-free sparse retriever for visual document retrieval:
document pages (rendered PDFs, slides, scans) are encoded by a ModernVBERT
backbone (SigLIP vision tower + pixel-shuffle connector + ModernBERT text
encoder) with a SPLADE MLM head into a 50,368-dim vocabulary-space sparse
vector, while queries are resolved by a learned Bag-of-Words lookup with no
neural encoding at all.
**Contents**: `weights.safetensors` (document encoder, bfloat16),
`query_lookup.npy` (inference-free query table, fp32), `config.json`,
plus tokenizer/processor configs for self-contained loading.
**Changes from upstream**: PyTorch checkpoint converted to MLX safetensors
(parameter re-mapping, conv weight transposed to NHWC, cast to bfloat16); the
query lookup table `softplus(embedding @ projection + bias)` is precomputed
with special tokens zeroed. No training or fine-tuning was performed.
**Quality** (see repo REPORT.md for methodology):
- Separate fp32 conversion parity vs the PyTorch reference: max |logit delta|
1.5e-04 on
real document-page inputs, sparse-vector cosine 1.000000, top-64 term
overlap 100%; the query table matches the shipped Sentence Transformers
static embedding to 1.2e-07.
- ViDoRe `docvqa_test_subsampled` nDCG@5 (fp32): 0.4098 (torch) ->
0.4098 (MLX), delta +0.0000 (gate: ±0.002).
This repository itself stores **bfloat16** document-encoder weights. The fp32 numbers
above describe a separate fp32 conversion and must not be attributed to this linked
bfloat16 artifact.
## Usage
```python
from splade_mlx.convert_vsplade import load_vsplade
import mlx.core as mx
from PIL import Image
model, query_encoder, processor = load_vsplade("NomaDamas/v-splade-efficient-mlx")
# documents (page images)
enc = processor(text=["User:<image><end_of_utterance>\nAssistant:"],
images=[[Image.open("page.png")]], return_tensors="np")
d = model.encode(mx.array(enc["input_ids"]), mx.array(enc["attention_mask"]),
enc["pixel_values"]) # (1, 50368)
# queries: inference-free lookup, no neural network
q = processor.tokenizer(["total revenue 2023"], return_tensors="np")
qw = query_encoder.encode(q["input_ids"], q["attention_mask"]) # (1, 50368)
score = d @ qw.T
```
## License
Apache-2.0, same as the upstream checkpoint (© NAVER Corp).
This repository is not affiliated with or endorsed by NAVER.
|