AngelPanizo commited on
Commit
37321c3
·
verified ·
1 Parent(s): 066c3c8

Add BERTopic model

Browse files
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ tags:
4
+ - bertopic
5
+ library_name: bertopic
6
+ pipeline_tag: text-classification
7
+ ---
8
+
9
+ # MARTINI_enrich_BERTopic_PAYorkshire
10
+
11
+ This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model.
12
+ BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets.
13
+
14
+ ## Usage
15
+
16
+ To use this model, please install BERTopic:
17
+
18
+ ```
19
+ pip install -U bertopic
20
+ ```
21
+
22
+ You can use the model as follows:
23
+
24
+ ```python
25
+ from bertopic import BERTopic
26
+ topic_model = BERTopic.load("AIDA-UPM/MARTINI_enrich_BERTopic_PAYorkshire")
27
+
28
+ topic_model.get_topic_info()
29
+ ```
30
+
31
+ ## Topic overview
32
+
33
+ * Number of topics: 5
34
+ * Number of training documents: 387
35
+
36
+ <details>
37
+ <summary>Click here for an overview of all topics.</summary>
38
+
39
+ | Topic ID | Topic Keywords | Topic Frequency | Label |
40
+ |----------|----------------|-----------------|-------|
41
+ | -1 | patrioticalternative - protest - wales - everybody - speeches | 28 | -1_patrioticalternative_protest_wales_everybody |
42
+ | 0 | migrants - britain - village - cottingham - centres | 192 | 0_migrants_britain_village_cottingham |
43
+ | 1 | patrioticalternative - lancastrian - celebrated - whitby - supporters | 70 | 1_patrioticalternative_lancastrian_celebrated_whitby |
44
+ | 2 | livestream - 7pm - paul - nationalist - white | 52 | 2_livestream_7pm_paul_nationalist |
45
+ | 3 | patrioticalternative - sheffield - flyering - active - megaphone | 45 | 3_patrioticalternative_sheffield_flyering_active |
46
+
47
+ </details>
48
+
49
+ ## Training hyperparameters
50
+
51
+ * calculate_probabilities: True
52
+ * language: None
53
+ * low_memory: False
54
+ * min_topic_size: 10
55
+ * n_gram_range: (1, 1)
56
+ * nr_topics: None
57
+ * seed_topic_list: None
58
+ * top_n_words: 10
59
+ * verbose: False
60
+ * zeroshot_min_similarity: 0.7
61
+ * zeroshot_topic_list: None
62
+
63
+ ## Framework versions
64
+
65
+ * Numpy: 1.26.4
66
+ * HDBSCAN: 0.8.40
67
+ * UMAP: 0.5.7
68
+ * Pandas: 2.2.3
69
+ * Scikit-Learn: 1.5.2
70
+ * Sentence-transformers: 3.3.1
71
+ * Transformers: 4.46.3
72
+ * Numba: 0.60.0
73
+ * Plotly: 5.24.1
74
+ * Python: 3.10.12
config.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "calculate_probabilities": true,
3
+ "language": null,
4
+ "low_memory": false,
5
+ "min_topic_size": 10,
6
+ "n_gram_range": [
7
+ 1,
8
+ 1
9
+ ],
10
+ "nr_topics": null,
11
+ "seed_topic_list": null,
12
+ "top_n_words": 10,
13
+ "verbose": false,
14
+ "zeroshot_min_similarity": 0.7,
15
+ "zeroshot_topic_list": null
16
+ }
ctfidf.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b1f12084d7b0299f6dff0169af8b0675a674d251971d0726554457928aa0c86
3
+ size 105696
ctfidf_config.json ADDED
The diff for this file is too large to render. See raw diff
 
topic_embeddings.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6770159170d50a6c61c0ecb3b94c73476f41de0206f532dc7e4fd358f46134d
3
+ size 20568
topics.json ADDED
@@ -0,0 +1,547 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "topic_representations": {
3
+ "-1": [
4
+ [
5
+ "patrioticalternative",
6
+ 0.6204352378845215
7
+ ],
8
+ [
9
+ "protest",
10
+ 0.5152460336685181
11
+ ],
12
+ [
13
+ "wales",
14
+ 0.49220985174179077
15
+ ],
16
+ [
17
+ "everybody",
18
+ 0.4568079710006714
19
+ ],
20
+ [
21
+ "speeches",
22
+ 0.44470688700675964
23
+ ]
24
+ ],
25
+ "0": [
26
+ [
27
+ "migrants",
28
+ 0.533321738243103
29
+ ],
30
+ [
31
+ "britain",
32
+ 0.4930104613304138
33
+ ],
34
+ [
35
+ "village",
36
+ 0.4613107740879059
37
+ ],
38
+ [
39
+ "cottingham",
40
+ 0.46012580394744873
41
+ ],
42
+ [
43
+ "centres",
44
+ 0.45590683817863464
45
+ ]
46
+ ],
47
+ "1": [
48
+ [
49
+ "patrioticalternative",
50
+ 0.49453204870224
51
+ ],
52
+ [
53
+ "lancastrian",
54
+ 0.4798000156879425
55
+ ],
56
+ [
57
+ "celebrated",
58
+ 0.47461968660354614
59
+ ],
60
+ [
61
+ "whitby",
62
+ 0.4664357304573059
63
+ ],
64
+ [
65
+ "supporters",
66
+ 0.4433601498603821
67
+ ]
68
+ ],
69
+ "2": [
70
+ [
71
+ "livestream",
72
+ 0.5316281318664551
73
+ ],
74
+ [
75
+ "7pm",
76
+ 0.5087007284164429
77
+ ],
78
+ [
79
+ "paul",
80
+ 0.4712033271789551
81
+ ],
82
+ [
83
+ "nationalist",
84
+ 0.46029651165008545
85
+ ],
86
+ [
87
+ "white",
88
+ 0.4415900409221649
89
+ ]
90
+ ],
91
+ "3": [
92
+ [
93
+ "patrioticalternative",
94
+ 0.5395963191986084
95
+ ],
96
+ [
97
+ "sheffield",
98
+ 0.5378563404083252
99
+ ],
100
+ [
101
+ "flyering",
102
+ 0.4957074522972107
103
+ ],
104
+ [
105
+ "active",
106
+ 0.4651254713535309
107
+ ],
108
+ [
109
+ "megaphone",
110
+ 0.43203818798065186
111
+ ]
112
+ ]
113
+ },
114
+ "topics": [
115
+ -1,
116
+ -1,
117
+ 1,
118
+ -1,
119
+ 1,
120
+ -1,
121
+ -1,
122
+ 2,
123
+ -1,
124
+ -1,
125
+ -1,
126
+ 3,
127
+ -1,
128
+ -1,
129
+ 3,
130
+ -1,
131
+ -1,
132
+ 3,
133
+ 1,
134
+ -1,
135
+ 3,
136
+ -1,
137
+ 0,
138
+ 3,
139
+ -1,
140
+ 1,
141
+ 3,
142
+ 1,
143
+ 0,
144
+ 1,
145
+ -1,
146
+ -1,
147
+ -1,
148
+ 0,
149
+ 0,
150
+ -1,
151
+ -1,
152
+ -1,
153
+ 3,
154
+ -1,
155
+ 0,
156
+ -1,
157
+ 1,
158
+ 0,
159
+ 0,
160
+ 0,
161
+ 0,
162
+ 2,
163
+ -1,
164
+ 0,
165
+ 0,
166
+ 0,
167
+ 0,
168
+ 2,
169
+ 1,
170
+ -1,
171
+ -1,
172
+ -1,
173
+ -1,
174
+ 0,
175
+ -1,
176
+ -1,
177
+ -1,
178
+ -1,
179
+ 2,
180
+ 0,
181
+ -1,
182
+ 1,
183
+ -1,
184
+ -1,
185
+ -1,
186
+ -1,
187
+ 1,
188
+ -1,
189
+ -1,
190
+ 1,
191
+ 3,
192
+ -1,
193
+ 3,
194
+ -1,
195
+ 1,
196
+ 0,
197
+ -1,
198
+ 3,
199
+ 2,
200
+ 1,
201
+ -1,
202
+ 2,
203
+ -1,
204
+ 2,
205
+ -1,
206
+ -1,
207
+ -1,
208
+ -1,
209
+ -1,
210
+ 0,
211
+ -1,
212
+ -1,
213
+ 0,
214
+ -1,
215
+ -1,
216
+ -1,
217
+ -1,
218
+ 1,
219
+ -1,
220
+ -1,
221
+ 2,
222
+ -1,
223
+ -1,
224
+ 2,
225
+ -1,
226
+ -1,
227
+ 1,
228
+ -1,
229
+ -1,
230
+ 2,
231
+ 0,
232
+ -1,
233
+ -1,
234
+ -1,
235
+ 0,
236
+ 2,
237
+ -1,
238
+ -1,
239
+ 2,
240
+ 2,
241
+ 1,
242
+ 2,
243
+ -1,
244
+ 2,
245
+ -1,
246
+ 0,
247
+ -1,
248
+ -1,
249
+ -1,
250
+ 0,
251
+ 0,
252
+ 0,
253
+ 2,
254
+ -1,
255
+ 0,
256
+ -1,
257
+ 0,
258
+ 0,
259
+ 1,
260
+ 0,
261
+ 0,
262
+ 0,
263
+ 0,
264
+ -1,
265
+ 0,
266
+ -1,
267
+ 0,
268
+ 0,
269
+ 0,
270
+ -1,
271
+ 2,
272
+ -1,
273
+ 0,
274
+ 0,
275
+ 0,
276
+ 0,
277
+ -1,
278
+ -1,
279
+ -1,
280
+ 2,
281
+ 0,
282
+ 2,
283
+ 0,
284
+ 0,
285
+ 3,
286
+ 3,
287
+ 0,
288
+ 2,
289
+ 0,
290
+ 0,
291
+ 0,
292
+ -1,
293
+ 0,
294
+ 0,
295
+ 0,
296
+ 2,
297
+ 3,
298
+ -1,
299
+ -1,
300
+ 3,
301
+ 3,
302
+ 3,
303
+ 2,
304
+ 3,
305
+ 2,
306
+ -1,
307
+ -1,
308
+ 1,
309
+ -1,
310
+ -1,
311
+ 3,
312
+ -1,
313
+ 3,
314
+ -1,
315
+ 2,
316
+ -1,
317
+ 2,
318
+ -1,
319
+ -1,
320
+ -1,
321
+ -1,
322
+ -1,
323
+ 1,
324
+ -1,
325
+ 2,
326
+ 0,
327
+ -1,
328
+ 1,
329
+ -1,
330
+ -1,
331
+ -1,
332
+ -1,
333
+ 1,
334
+ -1,
335
+ -1,
336
+ -1,
337
+ 3,
338
+ 0,
339
+ -1,
340
+ -1,
341
+ -1,
342
+ 2,
343
+ -1,
344
+ -1,
345
+ -1,
346
+ -1,
347
+ -1,
348
+ -1,
349
+ -1,
350
+ -1,
351
+ -1,
352
+ 3,
353
+ 0,
354
+ -1,
355
+ 1,
356
+ 0,
357
+ -1,
358
+ 0,
359
+ 1,
360
+ 0,
361
+ -1,
362
+ -1,
363
+ -1,
364
+ -1,
365
+ -1,
366
+ -1,
367
+ -1,
368
+ -1,
369
+ -1,
370
+ -1,
371
+ -1,
372
+ -1,
373
+ 1,
374
+ -1,
375
+ 1,
376
+ 1,
377
+ 2,
378
+ 3,
379
+ 0,
380
+ 0,
381
+ 0,
382
+ 2,
383
+ 2,
384
+ -1,
385
+ -1,
386
+ -1,
387
+ 0,
388
+ -1,
389
+ 3,
390
+ 1,
391
+ 2,
392
+ -1,
393
+ 2,
394
+ -1,
395
+ 1,
396
+ -1,
397
+ -1,
398
+ 0,
399
+ 0,
400
+ -1,
401
+ -1,
402
+ -1,
403
+ -1,
404
+ 1,
405
+ -1,
406
+ 0,
407
+ -1,
408
+ 0,
409
+ 0,
410
+ -1,
411
+ 0,
412
+ 0,
413
+ 1,
414
+ -1,
415
+ 0,
416
+ -1,
417
+ -1,
418
+ -1,
419
+ -1,
420
+ -1,
421
+ -1,
422
+ 0,
423
+ 1,
424
+ -1,
425
+ -1,
426
+ -1,
427
+ -1,
428
+ 3,
429
+ 1,
430
+ 2,
431
+ -1,
432
+ 1,
433
+ 1,
434
+ -1,
435
+ -1,
436
+ -1,
437
+ -1,
438
+ -1,
439
+ -1,
440
+ -1,
441
+ -1,
442
+ -1,
443
+ 2,
444
+ 2,
445
+ 0,
446
+ 2,
447
+ 1,
448
+ 2,
449
+ 1,
450
+ 3,
451
+ 3,
452
+ -1,
453
+ 3,
454
+ 1,
455
+ 2,
456
+ 1,
457
+ 1,
458
+ 2,
459
+ 1,
460
+ -1,
461
+ 1,
462
+ 1,
463
+ -1,
464
+ 1,
465
+ 1,
466
+ -1,
467
+ -1,
468
+ 1,
469
+ -1,
470
+ 1,
471
+ -1,
472
+ -1,
473
+ 1,
474
+ -1,
475
+ -1,
476
+ 3,
477
+ -1,
478
+ 2,
479
+ -1,
480
+ -1,
481
+ 2,
482
+ 1,
483
+ -1,
484
+ -1,
485
+ 2,
486
+ -1,
487
+ 1,
488
+ -1,
489
+ -1,
490
+ -1,
491
+ -1,
492
+ 1,
493
+ 0,
494
+ 1,
495
+ 2,
496
+ 2,
497
+ 2,
498
+ -1,
499
+ -1,
500
+ -1,
501
+ 1
502
+ ],
503
+ "topic_sizes": {
504
+ "-1": 192,
505
+ "1": 52,
506
+ "2": 45,
507
+ "3": 28,
508
+ "0": 70
509
+ },
510
+ "topic_mapper": [
511
+ [
512
+ -1,
513
+ -1,
514
+ -1
515
+ ],
516
+ [
517
+ 0,
518
+ 0,
519
+ 2
520
+ ],
521
+ [
522
+ 1,
523
+ 1,
524
+ 0
525
+ ],
526
+ [
527
+ 2,
528
+ 2,
529
+ 3
530
+ ],
531
+ [
532
+ 3,
533
+ 3,
534
+ 1
535
+ ]
536
+ ],
537
+ "topic_labels": {
538
+ "-1": "-1_patrioticalternative_protest_wales_everybody",
539
+ "0": "0_migrants_britain_village_cottingham",
540
+ "1": "1_patrioticalternative_lancastrian_celebrated_whitby",
541
+ "2": "2_livestream_7pm_paul_nationalist",
542
+ "3": "3_patrioticalternative_sheffield_flyering_active"
543
+ },
544
+ "custom_labels": null,
545
+ "_outliers": 1,
546
+ "topic_aspects": {}
547
+ }