README.md

---
license: cc-by-nc-4.0
base_model: nvidia/llama-nv-embed-reasoning-3b
library_name: sentence-transformers
pipeline_tag: feature-extraction
tags:
  - sentence-transformers
  - dense-retrieval
  - information-retrieval
  - oblique-retrieval
  - authorship
  - stylometry
language:
  - en
---

# OBLIQ-IR-3B

A 3B single-vector dense retriever for **oblique retrieval** — queries whose relevance is decided by a
latent attribute (an implicit stance, an abstract proof strategy, an authorial style, a lossy recollection
of a rhetorical exchange) that has little or no surface expression in the document.

From *OBLIQ-IR: Training a Dense Retriever for Oblique Queries* (EMNLP 2026).

**This repository holds the merged full-weight model** — load it and go, no base download, no PEFT.

| | |
|---|---|
| 🧩 LoRA adapter only (48 MB) | [`DataScience-UIBK/OBLIQ-IR-3B-LoRA`](https://huggingface.co/DataScience-UIBK/OBLIQ-IR-3B-LoRA) |
| 📊 Training data, runs, results | [`DataScience-UIBK/OBLIQ-IR-Data`](https://huggingface.co/datasets/DataScience-UIBK/OBLIQ-IR-Data) |
| 💻 Code | [github.com/DataScienceUIBK/obliq-ir](https://github.com/DataScienceUIBK/obliq-ir) |
| 🧪 Benchmark | [`dianetc/OBLIQ-Bench`](https://huggingface.co/datasets/dianetc/OBLIQ-Bench) |

## Usage

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DataScience-UIBK/OBLIQ-IR-3B", trust_remote_code=True)

q = model.encode(["query: "   + "A passage in the same restless, aphoristic voice as this one."])
d = model.encode(["passage: " + doc for doc in corpus])
scores = model.similarity(q, d)
```

**The `query: ` and `passage: ` prefixes are required** — the model was trained with them.

> **Do not prepend a task instruction.** The training file carries a per-task instruction string next to each
> query, but the training script consumes only the query text, and the evaluation behind every number in the
> paper encodes the raw query with the `query: ` prefix alone. Adding an instruction moves you
> off-distribution. Task identity comes from the training mixture, not a runtime prefix.

To match the paper's evaluation, encode documents at 1024 tokens and queries at 256:

```python
model.max_seq_length = 1024
d = model.encode(["passage: " + x for x in corpus])
model.max_seq_length = 256
q = model.encode(["query: " + x for x in queries])
```

## Results

NDCG@10 Gold on OBLIQ-Bench. "Full" adds TourRank (Y=5) on Math/Twitter/Congress; on Writing the selected
policy is the identity, so this model alone *is* the full system there.

| Pipeline | Writing | Math | Twitter | Congress |
|---|---|---|---|---|
| BM25 | .077 | .022 | .000 | .000 |
| Qwen3-Embed-4B | .033 | .095 | .032 | .040 |
| Gemini-2-Embedding | .164 | .144 | .068 | .059 |
| GPT-5.2 Multi-Hop Agent | .061 | .161 | .141 | .183 |
| OBLIQ-IR (no distillation) | .096 | .148 | .158 | .196 |
| OBLIQ-IR dense | **.211** | .140 | .151 | .187 |
| **OBLIQ-IR full** | **.211** | **.171** | **.177** | **.281** |
| *Oracle GPT-5.2 Tournament (not attainable)* | *.515* | *.279* | *.331* | *.913* |

Gains are large and significant on Writing (+0.116) and Congress (+0.085, both *p*<0.001) and marginal on
Math (*p*=0.08) and Twitter (*p*=0.06). We do not claim the fifth task, WildChat-Errors.

## Merged vs. adapter: a small numerical caveat

The published numbers were produced by the **adapter** path (base weights + LoRA computed separately). This
repository stores `W + BA·scaling` folded into bf16 weights. Because bf16 carries only about three
significant digits, folding is slightly lossy, so this model does not reproduce the adapter bit-for-bit:

| Task | Metric | This merged model | Adapter (paper) | Δ |
|---|---|---|---|---|
| Math | NDCG@10 | 0.1407 | 0.1403 | +0.0004 |
| Twitter | NDCG@10 | 0.1516 | 0.1508 | +0.0008 |

Top-1 results agree on 151/151 math queries and 275/281 twitter queries; deeper in the ranking the order
shifts slightly. The differences are far below run-to-run variance, but they can move the **third decimal**.

**For exact reproduction of the paper's tables, use
[`OBLIQ-IR-3B-LoRA`](https://huggingface.co/DataScience-UIBK/OBLIQ-IR-3B-LoRA)**, which regenerates the
published rankings bit-for-bit. Use this repository when you want convenience.

## Training

Two signals mixed into one encoder shared by all tasks. **(1) Per-mechanism synthetic queries:** an
instruction LM describes a document's latent attribute, then writes a query through one of three lenses
matched to the OBLIQ query mechanisms — descriptive, analogue, tip-of-the-tongue — with four BM25 hard
negatives each. **(2) kNN-graph distillation from a frozen authorship encoder** (writing only): the k=3
nearest-neighbour graph of [Rivera-Soto et al. (2021)](https://huggingface.co/rrivera1849/LUAR-MUD) over the
writing corpus supplies anchor–positive pairs. Only the graph's *topology* is used; the teacher's similarity
scores are discarded. That lifts Writing from .096 to .211. Neither stage touches the OBLIQ qrels.

| Setting | Value |
|---|---|
| Base | `nvidia/llama-nv-embed-reasoning-3b` (Llama-3.2-3B bi-encoder) |
| Adapter (before merging) | LoRA r=16, α=32, dropout 0.05 on q/k/v/o/gate/up/down (~24M params) |
| Pooling | mean over tokens, then L2 normalise → 3072-dim |
| Loss | CachedMultipleNegativesRankingLoss, τ = 1/0.02 |
| LR | 2e-4, 3% warmup, linear decay |
| Batch | 4 per device × 40 GPUs × 2 grad-accum = 320 anchors |
| Seq length | 1024 (queries truncated to 256 at evaluation) |
| Schedule | 1 epoch, 40× H100, ~50 min |
| Training rows | 154,361 (149,361 synthetic + 5,000 authorship kNN pairs) |

## Licence and intended use

Released under **CC BY-NC 4.0**. This repository redistributes weights derived from
`nvidia/llama-nv-embed-reasoning-3b`, which NVIDIA releases for **non-commercial / research use only**; their
licence is included as `LICENSE_nvidia_base_model.txt` and their attribution notice as `NOTICE.txt`.

**Built with Llama.** The base model derives from `meta-llama/Llama-3.2-3B` and the
[Llama 3.2 Community License](https://huggingface.co/meta-llama/Llama-3.2-1B/blob/main/LICENSE.txt) applies
in addition to the terms above.

**Intended use** is research on latent-attribute retrieval. Retrieval by authorial fingerprint can link texts
to an author across topics and venues, and so deanonymise writers who rely on pseudonymity. The authorship
encoder we distil is trained on Reddit data, and because style correlates with demography and dialect, the
distilled neighbourhood structure may carry those correlates. We did not audit for this; any deployment over
people's writing should.

## Citation

```bibtex
@inproceedings{abdalla2026obliqir,
  title     = {{OBLIQ-IR}: Training a Dense Retriever for Oblique Queries},
  author    = {Abdalla, Mahmoud and Abdallah, Abdelrahman and Sedek, Shaimaa and Jatowt, Adam},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
  year      = {2026}
}
```