| --- |
| license: mit |
| language: |
| - en |
| base_model: |
| - bilalzafar/CentralBank-BERT |
| pipeline_tag: text-classification |
| library_name: transformers |
| metrics: |
| - accuracy |
| - f1 |
| tags: |
| - AI |
| - Sentiment |
| - Finance |
| - Central Bank |
| - BIS |
| - Transformers |
| - Domain Adaptation |
| --- |
| |
| # CentralBank-AI-Classifier: Detecting AI vs. Non-AI Sentences in Central-Bank Discourse |
|
|
| **CentralBank-AI-Classifier** is a binary sentence-level classifier (`AI`, `Non-AI`) trained on BIS central-bank speeches. The model identifies whether a sentence is *about AI* (e.g., AI/ML/LLM/GenAI/NLP/vision topics) or not. It is built on the domain-adapted encoder [`CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT) , which was pretrained on ~66M tokens from 2M+ sentences of BIS speeches (1996–2024). |
|
|
|
|
| ## Dataset |
|
|
| - **Total labeled sentences:** **3,245** |
| - **Class balance:** **AI = 1,619** | **Non-AI = 1,626** |
| - **Grouping:** **URL-grouped** to avoid speech leakage. |
| - **Split (80/10/10 by URL):** Train **2,603** · Dev **322** · Test **320** |
|
|
| Labels were curated via rule-based retrieval (domain dictionary) followed by manual audit. |
|
|
|
|
| ## Training |
|
|
| - **Base model:** [`CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT) |
| - **Head:** `BertForSequenceClassification(num_labels=2)` |
| - **Max length:** 128 |
| - **Optimizer:** AdamW |
| - **LR:** 2e-5 · **Weight decay:** 0.01 · **Warmup:** 10% |
| - **Batch:** 16 (train) / 32 (eval) |
| - **Epochs:** up to 4 (early stopping on dev macro-F1) |
| - **Precision:** fp16 when available |
| - **Threshold tuning:** decision threshold selected on dev by macro-F1 sweep → **τ = 0.05** |
| - **Loss:** standard cross-entropy (dataset is balanced) |
|
|
| ## Evaluation (Held-out Test) |
|
|
| | Metric | Value | |
| |---|---| |
| | **Accuracy** | **0.9812** | |
| | **Macro-F1** | **0.9812** | |
| | F1 (AI) | 0.9810 | |
| | F1 (Non-AI) | 0.9815 | |
| | **ROC-AUC** | **0.9932** | |
| | **PR-AUC (AI as positive)** | **0.9959** | |
|
|
| **Notes.** Threshold **τ = 0.05** was tuned on the dev set for macro-F1 and then fixed for test. |
|
|
| ## Reliability Check on Keyword-Retrieved Sentences |
|
|
| Scoring the keyword-retrieved corpus with the trained classifier (τ = 0.05): |
|
|
| - **Mean P(AI)** = **0.9877**; **Median** = **0.9995** |
| - **Q1–Q3** = **0.9994–0.9995** (IQR = 0.0001) |
| - **Predicted AI share** = **0.9893** |
| - **High-confidence share (≥ 0.90)** = **0.9872** |
| - **Borderline (±0.10 around τ, i.e., [0.00, 0.15])** = **0.0112** |
|
|
| These statistics indicate the rule-based retrieval is **highly reliable**; only a small tail merits manual spot-checks. |
|
|
| ## Intended Use |
|
|
| - Filtering and measuring **AI-related discourse** in central-bank communications (speeches, testimonies, reports). |
| - Pre-filtering before downstream tasks (stance, sentiment, topic modeling). |
| - Corpus construction and time-series indicators of AI attention. |
|
|
| **Out of scope:** social media, consumer product reviews, or informal text. |
|
|
| --- |
|
|
| ### Project GitHub Repository |
|
|
| The complete reproducible workflow, including the FinAI dictionary, dictionary-based tagging, AI sentence classification, sentiment analysis, and structural topic modeling, is available on GitHub: |
|
|
| **CentralBank-AI:** [https://github.com/bilalezafar/CentralBank-AI](https://github.com/bilalezafar/CentralBank-AI) |
|
|
| --- |
| ## Usage |
|
|
| ### Simple pipeline |
| ```python |
| from transformers import pipeline |
| |
| clf = pipeline("text-classification", |
| model="bilalzafar/CentralBank-AI-Classifier", |
| return_all_scores=False) |
| |
| s = "We are piloting large language models to streamline supervisory analytics." |
| print(clf(s)[0]) # -> {'label': '1', 'score': 0.999...} |
| #Note Label_1=AI, Label_0=Non-AI |
| ``` |
|
|
| --- |
|
|
| ### Citation |
|
|
| > Please cite as: **Zafar, M. B., Ali, H., & Aysan, A. F. (2026). *Signals from the Noise: Decoding Global AI Discourse in Central Bank Communications*. Central Bank Review, Article 100268.** [https://doi.org/10.1016/j.cbrev.2026.100268](https://doi.org/10.1016/j.cbrev.2026.100268) |
|
|
| ```bibtex |
| @article{zafar2026signals, |
| title = {Signals from the Noise: Decoding Global AI Discourse in Central Bank Communications}, |
| author = {Zafar, Muhammad Bilal and Ali, Hassnian and Aysan, Ahmet Faruk}, |
| year = {2026}, |
| journal = {Central Bank Review}, |
| pages = {100268}, |
| doi = {10.1016/j.cbrev.2026.100268}, |
| url = {https://doi.org/10.1016/j.cbrev.2026.100268} |
| } |
| ``` |
| --- |
