Instructions to use tceron/high-risk-classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tceron/high-risk-classifier with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="tceron/high-risk-classifier")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("tceron/high-risk-classifier") model = AutoModelForSequenceClassification.from_pretrained("tceron/high-risk-classifier", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -30,13 +30,30 @@ Labels:
|
|
| 30 |
|
| 31 |
## Model Details
|
| 32 |
|
| 33 |
-
All details in [paper](https://
|
| 34 |
```bibtex
|
| 35 |
-
@inproceedings{
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
}
|
| 41 |
```
|
| 42 |
|
|
|
|
| 30 |
|
| 31 |
## Model Details
|
| 32 |
|
| 33 |
+
All details in [paper](https://aclanthology.org/2026.emnlp-main.763/):
|
| 34 |
```bibtex
|
| 35 |
+
@inproceedings{ceron-etal-2026-wildseek,
|
| 36 |
+
title = "{W}ild{SEEK}: Evaluating Language Models for Information-Seeking",
|
| 37 |
+
author = "Ceron, Tanise and
|
| 38 |
+
Baumann, Joachim and
|
| 39 |
+
Bassignana, Elisa and
|
| 40 |
+
Cabuk, Berat and
|
| 41 |
+
Hovy, Dirk and
|
| 42 |
+
Nozza, Debora",
|
| 43 |
+
editor = "Lee, Hung-Yi and
|
| 44 |
+
Dev, Sunipa and
|
| 45 |
+
Li, Jessy and
|
| 46 |
+
Elliott, Desmond",
|
| 47 |
+
booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing",
|
| 48 |
+
month = oct,
|
| 49 |
+
year = "2026",
|
| 50 |
+
address = "Budapest, Hungary",
|
| 51 |
+
publisher = "Association for Computational Linguistics",
|
| 52 |
+
url = "https://aclanthology.org/2026.emnlp-main.763/",
|
| 53 |
+
doi = "10.18653/2026.emnlp-main.763",
|
| 54 |
+
pages = "17690--17718",
|
| 55 |
+
ISBN = "979-8-89176-459-0",
|
| 56 |
+
abstract = "Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existing evaluations, however, are often topic-specific or synthetic, limiting their ability to capture the complexity of ``in the wild'' information-seeking queries and the risks present in model responses. To address this gap, we introduce WildSEEK, a manually annotated dataset of 3k information-seeking queries from real user interactions, and an evaluation framework for LLM-generated responses. WildSEEK includes annotations for risk-sensitive domains (e.g. health and financial information), and distinguishes factoid queries from analytical queries which seek responses beyond facts. We train classifiers on WildSEEK to analyze more than 1.8M realistic user queries. We find that over a third of information-seeking queries are high-risk and more often analytical. Our findings show that LLM responses fail more often in four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations {---} with failure rates being mostly higher for analytical queries. By providing methods to monitor the reliability, safety, and fairness of LLM behavior, our dataset and evaluation framework offer an empirical foundation for the broader question of how these systems should behave as they take on a growing role in information access."
|
| 57 |
}
|
| 58 |
```
|
| 59 |
|