README.md

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-classification
base_model: google-bert/bert-base-uncased
datasets:
- hotpotqa/hotpot_qa
tags:
- prompt-compression
- dependency-detection
- referential-dangling
- research
---

# Referential Dangling Dependency Detector

This is the sentence-pair dependency detector released with **Relevant but
Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard
Prompt Compression**.

The model scores whether a candidate sentence supplies a necessary dependency
for a retained sentence, conditioned on the question. It is used by the
repository's automatic context-restoration experiments.

## Authors

Zhengpei Hu<sup>1,∗</sup>, Kai Li<sup>2,∗</sup>, Dapeng Fu<sup>3</sup>,
Xuechao Zou<sup>2</sup>, Yuanhao Tang<sup>1</sup>, Yue Li<sup>1</sup>,
Tengfei Cao<sup>1</sup>, and Jianqiang Huang<sup>1,†</sup>

- <sup>1</sup>School of Computer Technology and Application, Qinghai University
- <sup>2</sup>Tsinghua University
- <sup>3</sup>Ant Group Security and Intelligence Laboratory (SIL)
- <sup>∗</sup>Equal contribution; <sup>†</sup>corresponding author

## Model details

- **Architecture:** BERT sequence classifier with two labels
- **Base model:** `google-bert/bert-base-uncased`
- **Labels:** `NOT_DEPENDENCY` (0), `DEPENDENCY` (1)
- **Maximum training input length:** 256 tokens
- **Input format:** `retained sentence [SEP] candidate support [SEP] question`

## Training data

Training pairs were constructed from the HotpotQA training split. Positive
pairs contain a retained sentence and a missing gold-support sentence that
share a discriminative entity. Negatives include entity-overlapping hard
negatives and unrelated deleted sentences. Splitting is grouped by source
example to prevent sentence pairs from the same example appearing in both the
training and validation partitions.

See `src/build_train_tight.py` and `src/train_detector.py` in the
[Referential-Dangling repository](https://github.com/JusperLee/Referential-Dangling)
for the data construction and training code.

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "JusperLee/referential-dangling-detector"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

retained = "The film was directed by Jane Smith."
candidate = "Jane Smith is a Canadian filmmaker."
question = "What nationality is the film's director?"
text = f"{retained} [SEP] {candidate} [SEP] {question}"
inputs = tokenizer(text, truncation=True, max_length=256, return_tensors="pt")

with torch.no_grad():
    probability = model(**inputs).logits.softmax(dim=-1)[0, 1].item()

print(probability)
```

For the paper's restoration pipeline, use `BertDependencyDetector` from
`src/beaver2_bert.py`.

## Intended use and limitations

This checkpoint is intended for research on dependency loss and automatic
support restoration in compressed English QA contexts. It is not a general
factuality, entailment, or coreference model. Its predictions depend on the
candidate-generation procedure and may not transfer reliably to other domains,
languages, or substantially different compression settings without evaluation.

## Citation

```bibtex
@misc{referentialdangling,
  title={Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression},
  author={Zhengpei Hu and Kai Li and Dapeng Fu and Xuechao Zou and Yuanhao Tang and Yue Li and Tengfei Cao and Jianqiang Huang},
  note={Research code and model release}
}
```