File size: 4,329 Bytes
647f216 0042c49 647f216 0042c49 647f216 0042c49 647f216 88c6741 647f216 88c6741 647f216 88c6741 647f216 88c6741 647f216 88c6741 647f216 88c6741 647f216 88c6741 647f216 a4f7782 4f5528b 88c6741 647f216 a26d6e7 4f5528b a26d6e7 4f5528b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | ---
license: mit
language:
- en
base_model:
- bilalzafar/CentralBank-BERT
pipeline_tag: text-classification
library_name: transformers
metrics:
- accuracy
- f1
tags:
- AI
- Sentiment
- Finance
- Central Bank
- BIS
- Transformers
- Domain Adaptation
---
# CentralBank-AI-Classifier: Detecting AI vs. Non-AI Sentences in Central-Bank Discourse
**CentralBank-AI-Classifier** is a binary sentence-level classifier (`AI`, `Non-AI`) trained on BIS central-bank speeches. The model identifies whether a sentence is *about AI* (e.g., AI/ML/LLM/GenAI/NLP/vision topics) or not. It is built on the domain-adapted encoder [`CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT) , which was pretrained on ~66M tokens from 2M+ sentences of BIS speeches (1996–2024).
## Dataset
- **Total labeled sentences:** **3,245**
- **Class balance:** **AI = 1,619** | **Non-AI = 1,626**
- **Grouping:** **URL-grouped** to avoid speech leakage.
- **Split (80/10/10 by URL):** Train **2,603** · Dev **322** · Test **320**
Labels were curated via rule-based retrieval (domain dictionary) followed by manual audit.
## Training
- **Base model:** [`CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT)
- **Head:** `BertForSequenceClassification(num_labels=2)`
- **Max length:** 128
- **Optimizer:** AdamW
- **LR:** 2e-5 · **Weight decay:** 0.01 · **Warmup:** 10%
- **Batch:** 16 (train) / 32 (eval)
- **Epochs:** up to 4 (early stopping on dev macro-F1)
- **Precision:** fp16 when available
- **Threshold tuning:** decision threshold selected on dev by macro-F1 sweep → **τ = 0.05**
- **Loss:** standard cross-entropy (dataset is balanced)
## Evaluation (Held-out Test)
| Metric | Value |
|---|---|
| **Accuracy** | **0.9812** |
| **Macro-F1** | **0.9812** |
| F1 (AI) | 0.9810 |
| F1 (Non-AI) | 0.9815 |
| **ROC-AUC** | **0.9932** |
| **PR-AUC (AI as positive)** | **0.9959** |
**Notes.** Threshold **τ = 0.05** was tuned on the dev set for macro-F1 and then fixed for test.
## Reliability Check on Keyword-Retrieved Sentences
Scoring the keyword-retrieved corpus with the trained classifier (τ = 0.05):
- **Mean P(AI)** = **0.9877**; **Median** = **0.9995**
- **Q1–Q3** = **0.9994–0.9995** (IQR = 0.0001)
- **Predicted AI share** = **0.9893**
- **High-confidence share (≥ 0.90)** = **0.9872**
- **Borderline (±0.10 around τ, i.e., [0.00, 0.15])** = **0.0112**
These statistics indicate the rule-based retrieval is **highly reliable**; only a small tail merits manual spot-checks.
## Intended Use
- Filtering and measuring **AI-related discourse** in central-bank communications (speeches, testimonies, reports).
- Pre-filtering before downstream tasks (stance, sentiment, topic modeling).
- Corpus construction and time-series indicators of AI attention.
**Out of scope:** social media, consumer product reviews, or informal text.
---
### Project GitHub Repository
The complete reproducible workflow, including the FinAI dictionary, dictionary-based tagging, AI sentence classification, sentiment analysis, and structural topic modeling, is available on GitHub:
**CentralBank-AI:** [https://github.com/bilalezafar/CentralBank-AI](https://github.com/bilalezafar/CentralBank-AI)
---
## Usage
### Simple pipeline
```python
from transformers import pipeline
clf = pipeline("text-classification",
model="bilalzafar/CentralBank-AI-Classifier",
return_all_scores=False)
s = "We are piloting large language models to streamline supervisory analytics."
print(clf(s)[0]) # -> {'label': '1', 'score': 0.999...}
#Note Label_1=AI, Label_0=Non-AI
```
---
### Citation
> Please cite as: **Zafar, M. B., Ali, H., & Aysan, A. F. (2026). *Signals from the Noise: Decoding Global AI Discourse in Central Bank Communications*. Central Bank Review, Article 100268.** [https://doi.org/10.1016/j.cbrev.2026.100268](https://doi.org/10.1016/j.cbrev.2026.100268)
```bibtex
@article{zafar2026signals,
title = {Signals from the Noise: Decoding Global AI Discourse in Central Bank Communications},
author = {Zafar, Muhammad Bilal and Ali, Hassnian and Aysan, Ahmet Faruk},
year = {2026},
journal = {Central Bank Review},
pages = {100268},
doi = {10.1016/j.cbrev.2026.100268},
url = {https://doi.org/10.1016/j.cbrev.2026.100268}
}
```
--- |