pipeline_tag: text-ranking
library_name: transformers
license: other
tags:
- bert
- modernbert
- government
- relevance
- text-relevance
- govrelbench
GovRelBERT: A Model for Government Domain Relevance
GovRelBERT is a specialized model designed for evaluating the core capabilities of Large Language Models (LLMs) in the government domain, particularly focusing on text relevance. It was introduced as part of the GovRelBench: A Benchmark for Government Domain Relevance paper.
Built upon the ModernBERT architecture, GovRelBERT is trained using the SoftGovScore method. This innovative method converts hard labels into soft scores, enabling the model to accurately compute a text's government domain relevance score. GovRelBERT serves as a dedicated evaluation tool within the GovRelBench framework, aiming to improve the assessment of LLMs in government-related research and practical applications.
The code and dataset are available as part of the GovRelBench project on GitHub: https://github.com/pansysy/GovRelBench.
How to use
You can use GovRelBERT with the Hugging Face transformers library to compute a government domain relevance score for a given text.
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model_name = "pansysy/GovRelBERT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
# Example 1: Text highly relevant to the government domain
text_relevant = "The new policy initiative aims to streamline public services for citizens."
inputs_relevant = tokenizer(text_relevant, return_tensors="pt")
with torch.no_grad():
outputs_relevant = model(**inputs_relevant)
relevance_score_relevant = outputs_relevant.logits.item()
print(f"Text: '{text_relevant}'")
print(f"Government relevance score: {relevance_score_relevant:.4f}")
print("-" * 50)
# Example 2: Text less relevant to the government domain
text_less_relevant = "How to train your pet dog to fetch a ball."
inputs_less_relevant = tokenizer(text_less_relevant, return_tensors="pt")
with torch.no_grad():
outputs_less_relevant = model(**inputs_less_relevant)
relevance_score_less_relevant = outputs_less_relevant.logits.item()
print(f"Text: '{text_less_relevant}'")
print(f"Government relevance score: {relevance_score_less_relevant:.4f}")
Citation
If you find this work helpful, please cite the original paper:
@misc{liu2025govrelbench,
title={GovRelBench:A Benchmark for Government Domain Relevance},
author={Yizhuo Liu and Siyi Pan and Ruohong Han and Yu Hong and Bojin Wang and Mingyang Li and Jianxun Tang},
year={2025},
eprint={2507.21419},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2507.21419},
}