musk1209 commited on
Commit
693fcd1
·
verified ·
1 Parent(s): e4b8818

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +125 -0
README.md ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ language: en
4
+ base_model: bert-base-uncased
5
+ tags:
6
+ - finance
7
+ - named-entity-recognition
8
+ - token-classification
9
+ - financial-news
10
+ - bert
11
+ datasets:
12
+ - gtfintechlab/finer-ord-bio
13
+ metrics:
14
+ - f1
15
+ - precision
16
+ - recall
17
+ pipeline_tag: token-classification
18
+ ---
19
+
20
+ # FinSight NER — Financial Named Entity Recognition
21
+
22
+ A financial-domain NER model fine-tuned from `bert-base-uncased` on the
23
+ [FiNER-ORD](https://huggingface.co/datasets/gtfintechlab/finer-ord-bio)
24
+ dataset (Shah et al., 2024), a manually-annotated corpus of financial
25
+ news articles.
26
+
27
+ Recognizes three entity types in BIO format:
28
+ - **PER** — persons (executives, board members, individuals mentioned in news)
29
+ - **ORG** — organizations (companies, banks, regulatory bodies, agencies)
30
+ - **LOC** — locations (cities, states, countries, regions)
31
+
32
+ Part of the [FinSight](https://github.com/tmuskan/finsight) project.
33
+
34
+ ## Performance (test split, entity-level)
35
+
36
+ Micro-averaged across all entity types:
37
+
38
+ | metric | value |
39
+ |--------|-------|
40
+ | precision | 0.7876 |
41
+ | recall | 0.8464 |
42
+ | f1 | 0.8159 |
43
+
44
+ Per-class breakdown:
45
+ type precision recall f1 support
46
+ ----------------------------------------------------
47
+ LOC 0.7896 0.8633 0.8248 300
48
+ ORG 0.7222 0.7993 0.7588 553
49
+ PER 0.9261 0.9196 0.9228 286
50
+ ----------------------------------------------------
51
+ micro 0.7876 0.8464 0.8159
52
+
53
+ ## Training Setup
54
+
55
+ | Setting | Value |
56
+ |--------|-------|
57
+ | Base model | `bert-base-uncased` |
58
+ | Dataset | `gtfintechlab/finer-ord-bio` |
59
+ | Train / Val / Test | 3,261 / 402 / 1,075 sentences |
60
+ | Epochs | 4 |
61
+ | Batch size | 16 |
62
+ | Learning rate | 3e-5 (linear warmup over 10% of steps) |
63
+ | Weight decay | 0.01 |
64
+ | Max sequence length | 192 |
65
+ | Optimizer | AdamW (default) |
66
+ | Mixed precision | fp16 |
67
+ | Seed | 42 |
68
+ | Hardware | NVIDIA Tesla T4 (Kaggle) |
69
+ | Training runtime | ~2.5 minutes |
70
+
71
+ ## Label mapping
72
+
73
+ | ID | Label |
74
+ |----|-------|
75
+ | 0 | O |
76
+ | 1 | B-PER |
77
+ | 2 | I-PER |
78
+ | 3 | B-LOC |
79
+ | 4 | I-LOC |
80
+ | 5 | B-ORG |
81
+ | 6 | I-ORG |
82
+
83
+ ## Usage
84
+
85
+ ```python
86
+ from transformers import pipeline
87
+
88
+ ner = pipeline(
89
+ "token-classification",
90
+ model="musk1209/finsight-ner",
91
+ aggregation_strategy="simple",
92
+ )
93
+ ner("Jamie Dimon, CEO of JPMorgan Chase, addressed shareholders in London.")
94
+ # [{'entity_group': 'PER', 'word': 'jamie dimon', 'score': 0.99, ...},
95
+ # {'entity_group': 'ORG', 'word': 'jpmorgan chase', 'score': 1.00, ...},
96
+ # {'entity_group': 'LOC', 'word': 'london', 'score': 0.99, ...}]
97
+ ```
98
+
99
+ ## Scope and limitations
100
+
101
+ - **Domain**: Trained on Bloomberg-style financial news from 2015. Generalizes
102
+ well to modern news-style prose (including SEC filings' narrative sections)
103
+ but is not tuned for structured legal or contract language.
104
+ - **Coverage**: Only 3 entity types. Money amounts, percentages, and dates are
105
+ intentionally not covered — those are better handled by regex given their
106
+ rigid patterns in financial text.
107
+ - **Text style**: Best on well-formed sentences with proper capitalization.
108
+ All-caps headlines or lowercased text may underperform.
109
+
110
+ ## Custom evaluation code
111
+
112
+ The `seqeval` library (the standard NER metric library) has a broken
113
+ `pyproject.toml` that prevents installation on Python 3.12. This model was
114
+ evaluated using a custom entity-level scorer with strict-match semantics;
115
+ see [`src/fine_tuning/ner_metrics.py`](https://github.com/tmuskan/finsight/blob/main/src/fine_tuning/ner_metrics.py)
116
+ in the project repo.
117
+
118
+ ## Citation
119
+ @article{shah2024finerord,
120
+ title = {FiNER-ORD: Financial Named Entity Recognition Open Research Dataset},
121
+ author = {Shah, Agam and Gullapalli, Abhinav and Vithani, Ruchit and Galarnyk, Michael and Chava, Sudheer},
122
+ journal = {arXiv preprint arXiv:2302.11157},
123
+ year = {2024}
124
+ }
125
+