---
language: en
license: apache-2.0
tags:
  - splade
  - sparse-retrieval
  - information-retrieval
  - beir
pipeline_tag: feature-extraction
---

# SPLADEX — best_proxy checkpoint

This is the **best_proxy** checkpoint from an inference-free SPLADE-v3-doc layer-pruned training run
pruned from `Cdn13/splade-multi-static-doc`.

## Model description

- **Architecture**: SPLADE-v3-doc (MLM head, `log1p(ReLU(logits)).max()` document pooling)
- **Query encoder**: Inference-free — static per-token weights (`static_query_weights.pt`)
  initialized from IDF and learned during training.
- **Document encoder**: Full SPLADE document side (pruned transformer layers).
- **Selection criterion**: Best window-average selection proxy during training
  (top-1 ranking score − sparsity budget penalty).

## Files

| File | Description |
|------|-------------|
| `config.json` | Model config (HF format) |
| `model.safetensors` / `pytorch_model.bin` | Document-encoder weights |
| `tokenizer*` | Tokenizer files |
| `static_query_weights.pt` | Learned static query token weights |
| `trainer_state.pt` | Optimizer / scheduler state + training metrics at best step |

## Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

repo = "Cdn13/static-splade-trained-pruned"
tokenizer = AutoTokenizer.from_pretrained(repo)
model     = AutoModelForMaskedLM.from_pretrained(repo)

# Load static query weights
sqw = torch.load("static_query_weights.pt", map_location="cpu")
query_weights = sqw["query_weights"]  # shape: [vocab_size]
```

> **Note**: The query representation is `presence(token) * query_weights[token]`,
> computed without any forward pass through the model.