File size: 1,742 Bytes
efe3e30 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
language: en
license: apache-2.0
tags:
- splade
- sparse-retrieval
- information-retrieval
- beir
pipeline_tag: feature-extraction
---
# SPLADEX — best_proxy checkpoint
This is the **best_proxy** checkpoint from an inference-free SPLADE-v3-doc training run
on a combined BEIR-style documentation dataset (NumPy, Pandas, Pybind11, etc.).
## Model description
- **Architecture**: SPLADE-v3-doc (MLM head, `log1p(ReLU(logits)).max()` document pooling)
- **Query encoder**: Inference-free — static per-token weights (`static_query_weights.pt`)
initialized from IDF and learned during training.
- **Document encoder**: Full SPLADE document side.
- **Selection criterion**: Best window-average selection proxy during training
(top-1 ranking score − sparsity budget penalty).
## Files
| File | Description |
|------|-------------|
| `config.json` | Model config (HF format) |
| `model.safetensors` / `pytorch_model.bin` | Document-encoder weights |
| `tokenizer*` | Tokenizer files |
| `static_query_weights.pt` | Learned static query token weights |
| `trainer_state.pt` | Optimizer / scheduler state + training metrics at best step |
## Usage
```python
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM
repo = "Cdn13/splade-multi-static-doc"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForMaskedLM.from_pretrained(repo)
# Load static query weights
sqw = torch.load("static_query_weights.pt", map_location="cpu")
query_weights = sqw["query_weights"] # shape: [vocab_size]
```
> **Note**: The query representation is `presence(token) * query_weights[token]`,
> computed without any forward pass through the model.
|