| ---
|
| language: en
|
| license: apache-2.0
|
| tags:
|
| - splade
|
| - sparse-retrieval
|
| - information-retrieval
|
| - beir
|
| pipeline_tag: feature-extraction
|
| ---
|
|
|
| # SPLADEX — best_proxy checkpoint
|
|
|
| This is the **best_proxy** checkpoint from an inference-free SPLADE-v3-doc layer-pruned training run
|
| pruned from `Cdn13/splade-multi-static-doc`.
|
|
|
| ## Model description
|
|
|
| - **Architecture**: SPLADE-v3-doc (MLM head, `log1p(ReLU(logits)).max()` document pooling)
|
| - **Query encoder**: Inference-free — static per-token weights (`static_query_weights.pt`)
|
| initialized from IDF and learned during training.
|
| - **Document encoder**: Full SPLADE document side (pruned transformer layers).
|
| - **Selection criterion**: Best window-average selection proxy during training
|
| (top-1 ranking score − sparsity budget penalty).
|
|
|
| ## Files
|
|
|
| | File | Description |
|
| |------|-------------|
|
| | `config.json` | Model config (HF format) |
|
| | `model.safetensors` / `pytorch_model.bin` | Document-encoder weights |
|
| | `tokenizer*` | Tokenizer files |
|
| | `static_query_weights.pt` | Learned static query token weights |
|
| | `trainer_state.pt` | Optimizer / scheduler state + training metrics at best step |
|
|
|
| ## Usage
|
|
|
| ```python
|
| import torch
|
| from transformers import AutoTokenizer, AutoModelForMaskedLM
|
|
|
| repo = "Cdn13/static-splade-trained-pruned"
|
| tokenizer = AutoTokenizer.from_pretrained(repo)
|
| model = AutoModelForMaskedLM.from_pretrained(repo)
|
|
|
| # Load static query weights
|
| sqw = torch.load("static_query_weights.pt", map_location="cpu")
|
| query_weights = sqw["query_weights"] # shape: [vocab_size]
|
| ```
|
|
|
| > **Note**: The query representation is `presence(token) * query_weights[token]`,
|
| > computed without any forward pass through the model.
|
| |
