jbiggsets/openalex_topic_labels_with_setfit

SetFit with BAAI/bge-small-en-v1.5

This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-small-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: BAAI/bge-small-en-v1.5
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 194 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	'This sentence is General Agricultural and Biological Sciences' 'This sentence is General Agricultural and Biological Sciences'
1	'This sentence is Agronomy and Crop Science' 'This sentence is Agronomy and Crop Science'
2	'This sentence is Animal Science and Zoology' 'This sentence is Animal Science and Zoology'
3	'This sentence is Aquatic Science' 'This sentence is Aquatic Science'
4	'This sentence is Ecology, Evolution, Behavior and Systematics' 'This sentence is Ecology, Evolution, Behavior and Systematics'
5	'This sentence is Food Science' 'This sentence is Food Science'
6	'This sentence is Forestry' 'This sentence is Forestry'
7	'This sentence is Insect Science' 'This sentence is Insect Science'
8	'This sentence is Plant Science' 'This sentence is Plant Science'
9	'This sentence is Soil Science' 'This sentence is Soil Science'
10	'This sentence is General Arts and Humanities' 'This sentence is General Arts and Humanities'
11	'This sentence is History' 'This sentence is History'
12	'This sentence is Language and Linguistics' 'This sentence is Language and Linguistics'
13	'This sentence is Archeology' 'This sentence is Archeology'
14	'This sentence is Classics' 'This sentence is Classics'
15	'This sentence is Conservation' 'This sentence is Conservation'
16	'This sentence is History and Philosophy of Science' 'This sentence is History and Philosophy of Science'
17	'This sentence is Literature and Literary Theory' 'This sentence is Literature and Literary Theory'
18	'This sentence is Museology' 'This sentence is Museology'
19	'This sentence is Music' 'This sentence is Music'
20	'This sentence is Philosophy' 'This sentence is Philosophy'
21	'This sentence is Religious studies' 'This sentence is Religious studies'
22	'This sentence is Visual Arts and Performing Arts' 'This sentence is Visual Arts and Performing Arts'
23	'This sentence is Biochemistry' 'This sentence is Biochemistry'
24	'This sentence is Biophysics' 'This sentence is Biophysics'
25	'This sentence is Biotechnology' 'This sentence is Biotechnology'
26	'This sentence is Cancer Research' 'This sentence is Cancer Research'
27	'This sentence is Cell Biology' 'This sentence is Cell Biology'
28	'This sentence is Clinical Biochemistry' 'This sentence is Clinical Biochemistry'
29	'This sentence is Endocrinology' 'This sentence is Endocrinology'
30	'This sentence is Genetics' 'This sentence is Genetics'
31	'This sentence is Molecular Biology' 'This sentence is Molecular Biology'
32	'This sentence is Molecular Medicine' 'This sentence is Molecular Medicine'
33	'This sentence is Physiology' 'This sentence is Physiology'
34	'This sentence is Accounting' 'This sentence is Accounting'
35	'This sentence is Management Information Systems' 'This sentence is Management Information Systems'
36	'This sentence is Management of Technology and Innovation' 'This sentence is Management of Technology and Innovation'
37	'This sentence is Marketing' 'This sentence is Marketing'
38	'This sentence is Organizational Behavior and Human Resource Management' 'This sentence is Organizational Behavior and Human Resource Management'
39	'This sentence is Strategy and Management' 'This sentence is Strategy and Management'
40	'This sentence is Catalysis' 'This sentence is Catalysis'
41	'This sentence is Fluid Flow and Transfer Processes' 'This sentence is Fluid Flow and Transfer Processes'
42	'This sentence is Analytical Chemistry' 'This sentence is Analytical Chemistry'
43	'This sentence is Inorganic Chemistry' 'This sentence is Inorganic Chemistry'
44	'This sentence is Organic Chemistry' 'This sentence is Organic Chemistry'
45	'This sentence is Physical and Theoretical Chemistry' 'This sentence is Physical and Theoretical Chemistry'
46	'This sentence is Spectroscopy' 'This sentence is Spectroscopy'
47	'This sentence is Artificial Intelligence' 'This sentence is Artificial Intelligence'
48	'This sentence is Computational Theory and Mathematics' 'This sentence is Computational Theory and Mathematics'
49	'This sentence is Computer Graphics and Computer-Aided Design' 'This sentence is Computer Graphics and Computer-Aided Design'
50	'This sentence is Computer Networks and Communications' 'This sentence is Computer Networks and Communications'
51	'This sentence is Computer Science Applications' 'This sentence is Computer Science Applications'
52	'This sentence is Computer Vision and Pattern Recognition' 'This sentence is Computer Vision and Pattern Recognition'
53	'This sentence is Hardware and Architecture' 'This sentence is Hardware and Architecture'
54	'This sentence is Human-Computer Interaction' 'This sentence is Human-Computer Interaction'
55	'This sentence is Information Systems' 'This sentence is Information Systems'
56	'This sentence is Signal Processing' 'This sentence is Signal Processing'
57	'This sentence is Software' 'This sentence is Software'
58	'This sentence is Information Systems and Management' 'This sentence is Information Systems and Management'
59	'This sentence is Management Science and Operations Research' 'This sentence is Management Science and Operations Research'
60	'This sentence is Statistics, Probability and Uncertainty' 'This sentence is Statistics, Probability and Uncertainty'
61	'This sentence is Atmospheric Science' 'This sentence is Atmospheric Science'
62	'This sentence is Earth-Surface Processes' 'This sentence is Earth-Surface Processes'
63	'This sentence is Geochemistry and Petrology' 'This sentence is Geochemistry and Petrology'
64	'This sentence is Geology' 'This sentence is Geology'
65	'This sentence is Geophysics' 'This sentence is Geophysics'
66	'This sentence is Oceanography' 'This sentence is Oceanography'
67	'This sentence is Paleontology' 'This sentence is Paleontology'
68	'This sentence is General Economics, Econometrics and Finance' 'This sentence is General Economics, Econometrics and Finance'
69	'This sentence is Economics and Econometrics' 'This sentence is Economics and Econometrics'
70	'This sentence is Finance' 'This sentence is Finance'
71	'This sentence is Renewable Energy, Sustainability and the Environment' 'This sentence is Renewable Energy, Sustainability and the Environment'
72	'This sentence is Aerospace Engineering' 'This sentence is Aerospace Engineering'
73	'This sentence is Automotive Engineering' 'This sentence is Automotive Engineering'
74	'This sentence is Biomedical Engineering' 'This sentence is Biomedical Engineering'
75	'This sentence is Civil and Structural Engineering' 'This sentence is Civil and Structural Engineering'
76	'This sentence is Computational Mechanics' 'This sentence is Computational Mechanics'
77	'This sentence is Control and Systems Engineering' 'This sentence is Control and Systems Engineering'
78	'This sentence is Electrical and Electronic Engineering' 'This sentence is Electrical and Electronic Engineering'
79	'This sentence is Industrial and Manufacturing Engineering' 'This sentence is Industrial and Manufacturing Engineering'
80	'This sentence is Mechanical Engineering' 'This sentence is Mechanical Engineering'
81	'This sentence is Mechanics of Materials' 'This sentence is Mechanics of Materials'
82	'This sentence is Ocean Engineering' 'This sentence is Ocean Engineering'
83	'This sentence is Safety, Risk, Reliability and Quality' 'This sentence is Safety, Risk, Reliability and Quality'
84	'This sentence is Media Technology' 'This sentence is Media Technology'
85	'This sentence is Building and Construction' 'This sentence is Building and Construction'
86	'This sentence is Architecture' 'This sentence is Architecture'
87	'This sentence is Ecological Modeling' 'This sentence is Ecological Modeling'
88	'This sentence is Ecology' 'This sentence is Ecology'
89	'This sentence is Environmental Chemistry' 'This sentence is Environmental Chemistry'
90	'This sentence is Environmental Engineering' 'This sentence is Environmental Engineering'
91	'This sentence is Global and Planetary Change' 'This sentence is Global and Planetary Change'
92	'This sentence is Health, Toxicology and Mutagenesis' 'This sentence is Health, Toxicology and Mutagenesis'
93	'This sentence is Management, Monitoring, Policy and Law' 'This sentence is Management, Monitoring, Policy and Law'
94	'This sentence is Nature and Landscape Conservation' 'This sentence is Nature and Landscape Conservation'
95	'This sentence is Pollution' 'This sentence is Pollution'
96	'This sentence is Water Science and Technology' 'This sentence is Water Science and Technology'
97	'This sentence is Immunology' 'This sentence is Immunology'
98	'This sentence is Microbiology' 'This sentence is Microbiology'
99	'This sentence is Parasitology' 'This sentence is Parasitology'
100	'This sentence is Virology' 'This sentence is Virology'
101	'This sentence is General Materials Science' 'This sentence is General Materials Science'
102	'This sentence is Biomaterials' 'This sentence is Biomaterials'
103	'This sentence is Electronic, Optical and Magnetic Materials' 'This sentence is Electronic, Optical and Magnetic Materials'
104	'This sentence is Materials Chemistry' 'This sentence is Materials Chemistry'
105	'This sentence is Polymers and Plastics' 'This sentence is Polymers and Plastics'
106	'This sentence is Surfaces, Coatings and Films' 'This sentence is Surfaces, Coatings and Films'
107	'This sentence is Algebra and Number Theory' 'This sentence is Algebra and Number Theory'
108	'This sentence is Applied Mathematics' 'This sentence is Applied Mathematics'
109	'This sentence is Discrete Mathematics and Combinatorics' 'This sentence is Discrete Mathematics and Combinatorics'
110	'This sentence is Geometry and Topology' 'This sentence is Geometry and Topology'
111	'This sentence is Mathematical Physics' 'This sentence is Mathematical Physics'
112	'This sentence is Modeling and Simulation' 'This sentence is Modeling and Simulation'
113	'This sentence is Numerical Analysis' 'This sentence is Numerical Analysis'
114	'This sentence is Statistics and Probability' 'This sentence is Statistics and Probability'
115	'This sentence is Anesthesiology and Pain Medicine' 'This sentence is Anesthesiology and Pain Medicine'
116	'This sentence is Cardiology and Cardiovascular Medicine' 'This sentence is Cardiology and Cardiovascular Medicine'
117	'This sentence is Critical Care and Intensive Care Medicine' 'This sentence is Critical Care and Intensive Care Medicine'
118	'This sentence is Complementary and alternative medicine' 'This sentence is Complementary and alternative medicine'
119	'This sentence is Dermatology' 'This sentence is Dermatology'
120	'This sentence is Emergency Medicine' 'This sentence is Emergency Medicine'
121	'This sentence is Endocrinology, Diabetes and Metabolism' 'This sentence is Endocrinology, Diabetes and Metabolism'
122	'This sentence is Epidemiology' 'This sentence is Epidemiology'
123	'This sentence is Gastroenterology' 'This sentence is Gastroenterology'
124	'This sentence is Geriatrics and Gerontology' 'This sentence is Geriatrics and Gerontology'
125	'This sentence is Hematology' 'This sentence is Hematology'
126	'This sentence is Hepatology' 'This sentence is Hepatology'
127	'This sentence is Immunology and Allergy' 'This sentence is Immunology and Allergy'
128	'This sentence is Infectious Diseases' 'This sentence is Infectious Diseases'
129	'This sentence is Nephrology' 'This sentence is Nephrology'
130	'This sentence is Neurology' 'This sentence is Neurology'
131	'This sentence is Obstetrics and Gynecology' 'This sentence is Obstetrics and Gynecology'
132	'This sentence is Oncology' 'This sentence is Oncology'
133	'This sentence is Ophthalmology' 'This sentence is Ophthalmology'
134	'This sentence is Orthopedics and Sports Medicine' 'This sentence is Orthopedics and Sports Medicine'
135	'This sentence is Otorhinolaryngology' 'This sentence is Otorhinolaryngology'
136	'This sentence is Pathology and Forensic Medicine' 'This sentence is Pathology and Forensic Medicine'
137	'This sentence is Pediatrics, Perinatology and Child Health' 'This sentence is Pediatrics, Perinatology and Child Health'
138	'This sentence is Pharmacology' 'This sentence is Pharmacology'
139	'This sentence is Psychiatry and Mental health' 'This sentence is Psychiatry and Mental health'
140	'This sentence is Public Health, Environmental and Occupational Health' 'This sentence is Public Health, Environmental and Occupational Health'
141	'This sentence is Pulmonary and Respiratory Medicine' 'This sentence is Pulmonary and Respiratory Medicine'
142	'This sentence is Radiology, Nuclear Medicine and Imaging' 'This sentence is Radiology, Nuclear Medicine and Imaging'
143	'This sentence is Rehabilitation' 'This sentence is Rehabilitation'
144	'This sentence is Reproductive Medicine' 'This sentence is Reproductive Medicine'
145	'This sentence is Rheumatology' 'This sentence is Rheumatology'
146	'This sentence is Surgery' 'This sentence is Surgery'
147	'This sentence is Urology' 'This sentence is Urology'
148	'This sentence is Cellular and Molecular Neuroscience' 'This sentence is Cellular and Molecular Neuroscience'
149	'This sentence is Cognitive Neuroscience' 'This sentence is Cognitive Neuroscience'
150	'This sentence is Developmental Neuroscience' 'This sentence is Developmental Neuroscience'
151	'This sentence is Endocrine and Autonomic Systems' 'This sentence is Endocrine and Autonomic Systems'
152	'This sentence is Sensory Systems' 'This sentence is Sensory Systems'
153	'This sentence is Nutrition and Dietetics' 'This sentence is Nutrition and Dietetics'
154	'This sentence is Pharmaceutical Science' 'This sentence is Pharmaceutical Science'
155	'This sentence is Toxicology' 'This sentence is Toxicology'
156	'This sentence is Astronomy and Astrophysics' 'This sentence is Astronomy and Astrophysics'
157	'This sentence is Condensed Matter Physics' 'This sentence is Condensed Matter Physics'
158	'This sentence is Nuclear and High Energy Physics' 'This sentence is Nuclear and High Energy Physics'
159	'This sentence is Atomic and Molecular Physics, and Optics' 'This sentence is Atomic and Molecular Physics, and Optics'
160	'This sentence is Radiation' 'This sentence is Radiation'
161	'This sentence is Statistical and Nonlinear Physics' 'This sentence is Statistical and Nonlinear Physics'
162	'This sentence is Applied Psychology' 'This sentence is Applied Psychology'
163	'This sentence is Clinical Psychology' 'This sentence is Clinical Psychology'
164	'This sentence is Developmental and Educational Psychology' 'This sentence is Developmental and Educational Psychology'
165	'This sentence is Experimental and Cognitive Psychology' 'This sentence is Experimental and Cognitive Psychology'
166	'This sentence is Social Psychology' 'This sentence is Social Psychology'
167	'This sentence is General Social Sciences' 'This sentence is General Social Sciences'
168	'This sentence is Development' 'This sentence is Development'
169	'This sentence is Education' 'This sentence is Education'
170	'This sentence is Geography, Planning and Development' 'This sentence is Geography, Planning and Development'
171	'This sentence is Health' 'This sentence is Health'
172	'This sentence is Law' 'This sentence is Law'
173	'This sentence is Linguistics and Language' 'This sentence is Linguistics and Language'
174	'This sentence is Safety Research' 'This sentence is Safety Research'
175	'This sentence is Sociology and Political Science' 'This sentence is Sociology and Political Science'
176	'This sentence is Transportation' 'This sentence is Transportation'
177	'This sentence is Anthropology' 'This sentence is Anthropology'
178	'This sentence is Communication' 'This sentence is Communication'
179	'This sentence is Cultural Studies' 'This sentence is Cultural Studies'
180	'This sentence is Demography' 'This sentence is Demography'
181	'This sentence is Gender Studies' 'This sentence is Gender Studies'
182	'This sentence is Political Science and International Relations' 'This sentence is Political Science and International Relations'
183	'This sentence is Urban Studies' 'This sentence is Urban Studies'
184	'This sentence is Small Animals' 'This sentence is Small Animals'
185	'This sentence is Oral Surgery' 'This sentence is Oral Surgery'
186	'This sentence is Periodontics' 'This sentence is Periodontics'
187	'This sentence is General Health Professions' 'This sentence is General Health Professions'
188	'This sentence is Emergency Medical Services' 'This sentence is Emergency Medical Services'
189	'This sentence is Health Information Management' 'This sentence is Health Information Management'
190	'This sentence is Occupational Therapy' 'This sentence is Occupational Therapy'
191	'This sentence is Pharmacy' 'This sentence is Pharmacy'
192	'This sentence is Physical Therapy, Sports Therapy and Rehabilitation' 'This sentence is Physical Therapy, Sports Therapy and Rehabilitation'
193	'This sentence is Speech and Hearing' 'This sentence is Speech and Hearing'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("This sentence is Finance")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	5.5773	9

Label	Training Sample Count
0	2
1	2
2	2
3	2
4	2
5	2
6	2
7	2
8	2
9	2
10	2
11	2
12	2
13	2
14	2
15	2
16	2
17	2
18	2
19	2
20	2
21	2
22	2
23	2
24	2
25	2
26	2
27	2
28	2
29	2
30	2
31	2
32	2
33	2
34	2
35	2
36	2
37	2
38	2
39	2
40	2
41	2
42	2
43	2
44	2
45	2
46	2
47	2
48	2
49	2
50	2
51	2
52	2
53	2
54	2
55	2
56	2
57	2
58	2
59	2
60	2
61	2
62	2
63	2
64	2
65	2
66	2
67	2
68	2
69	2
70	2
71	2
72	2
73	2
74	2
75	2
76	2
77	2
78	2
79	2
80	2
81	2
82	2
83	2
84	2
85	2
86	2
87	2
88	2
89	2
90	2
91	2
92	2
93	2
94	2
95	2
96	2
97	2
98	2
99	2
100	2
101	2
102	2
103	2
104	2
105	2
106	2
107	2
108	2
109	2
110	2
111	2
112	2
113	2
114	2
115	2
116	2
117	2
118	2
119	2
120	2
121	2
122	2
123	2
124	2
125	2
126	2
127	2
128	2
129	2
130	2
131	2
132	2
133	2
134	2
135	2
136	2
137	2
138	2
139	2
140	2
141	2
142	2
143	2
144	2
145	2
146	2
147	2
148	2
149	2
150	2
151	2
152	2
153	2
154	2
155	2
156	2
157	2
158	2
159	2
160	2
161	2
162	2
163	2
164	2
165	2
166	2
167	2
168	2
169	2
170	2
171	2
172	2
173	2
174	2
175	2
176	2
177	2
178	2
179	2
180	2
181	2
182	2
183	2
184	2
185	2
186	2
187	2
188	2
189	2
190	2
191	2
192	2
193	2

Training Hyperparameters

• batch_size: (64, 64)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0004	1	0.1912	-
0.0214	50	0.1436	-
0.0427	100	0.0512	-
0.0641	150	0.0366	-
0.0854	200	0.0294	-
0.1068	250	0.0261	-
0.1282	300	0.0233	-
0.1495	350	0.0209	-
0.1709	400	0.0198	-
0.1922	450	0.0192	-
0.2136	500	0.0184	-
0.2349	550	0.0172	-
0.2563	600	0.0166	-
0.2777	650	0.0157	-
0.2990	700	0.0174	-
0.3204	750	0.0149	-
0.3417	800	0.0155	-
0.3631	850	0.0147	-
0.3845	900	0.0139	-
0.4058	950	0.0144	-
0.4272	1000	0.013	-
0.4485	1050	0.0143	-
0.4699	1100	0.0135	-
0.4912	1150	0.0126	-
0.5126	1200	0.0135	-
0.5340	1250	0.0139	-
0.5553	1300	0.0133	-
0.5767	1350	0.0127	-
0.5980	1400	0.0114	-
0.6194	1450	0.012	-
0.6408	1500	0.0127	-
0.6621	1550	0.0124	-
0.6835	1600	0.0117	-
0.7048	1650	0.0128	-
0.7262	1700	0.0116	-
0.7475	1750	0.0116	-
0.7689	1800	0.0123	-
0.7903	1850	0.0116	-
0.8116	1900	0.0114	-
0.8330	1950	0.0122	-
0.8543	2000	0.0119	-
0.8757	2050	0.0122	-
0.8971	2100	0.0116	-
0.9184	2150	0.0119	-
0.9398	2200	0.0112	-
0.9611	2250	0.0121	-
0.9825	2300	0.0109	-

Framework Versions

• Python: 3.10.11
• SetFit: 1.1.1
• Sentence Transformers: 3.3.1
• Transformers: 4.48.1
• PyTorch: 2.5.1+cpu
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}