---
pipeline_tag: text-ranking
library_name: transformers
license: other
tags:
  - bert
  - modernbert
  - government
  - relevance
  - text-relevance
  - govrelbench
---

# GovRelBERT: A Model for Government Domain Relevance

**GovRelBERT** is a specialized model designed for evaluating the core capabilities of Large Language Models (LLMs) in the government domain, particularly focusing on text relevance. It was introduced as part of the [GovRelBench: A Benchmark for Government Domain Relevance](https://huggingface.co/papers/2507.21419) paper.

Built upon the **ModernBERT** architecture, GovRelBERT is trained using the **SoftGovScore** method. This innovative method converts hard labels into soft scores, enabling the model to accurately compute a text's government domain relevance score. GovRelBERT serves as a dedicated evaluation tool within the GovRelBench framework, aiming to improve the assessment of LLMs in government-related research and practical applications.

The code and dataset are available as part of the GovRelBench project on GitHub: [https://github.com/pansysy/GovRelBench](https://github.com/pansysy/GovRelBench).

## How to use

You can use `GovRelBERT` with the Hugging Face `transformers` library to compute a government domain relevance score for a given text.

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "pansysy/GovRelBERT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example 1: Text highly relevant to the government domain
text_relevant = "The new policy initiative aims to streamline public services for citizens."
inputs_relevant = tokenizer(text_relevant, return_tensors="pt")

with torch.no_grad():
    outputs_relevant = model(**inputs_relevant)
    relevance_score_relevant = outputs_relevant.logits.item()

print(f"Text: '{text_relevant}'")
print(f"Government relevance score: {relevance_score_relevant:.4f}")

print("-" * 50)

# Example 2: Text less relevant to the government domain
text_less_relevant = "How to train your pet dog to fetch a ball."
inputs_less_relevant = tokenizer(text_less_relevant, return_tensors="pt")

with torch.no_grad():
    outputs_less_relevant = model(**inputs_less_relevant)
    relevance_score_less_relevant = outputs_less_relevant.logits.item()

print(f"Text: '{text_less_relevant}'")
print(f"Government relevance score: {relevance_score_less_relevant:.4f}")
```

## Citation

If you find this work helpful, please cite the original paper:

```bibtex
@misc{liu2025govrelbench,
      title={GovRelBench:A Benchmark for Government Domain Relevance},
      author={Yizhuo Liu and Siyi Pan and Ruohong Han and Yu Hong and Bojin Wang and Mingyang Li and Jianxun Tang},
      year={2025},
      eprint={2507.21419},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2507.21419},
}
```