Update README.md
Browse files
README.md
CHANGED
|
@@ -1,138 +1,145 @@
|
|
| 1 |
-
|
| 2 |
-
---
|
| 3 |
-
tags:
|
| 4 |
-
- bertopic
|
| 5 |
-
library_name: bertopic
|
| 6 |
-
pipeline_tag: text-classification
|
| 7 |
-
---
|
| 8 |
-
|
| 9 |
-
# ISSR_Dark_Web_68Topics
|
| 10 |
-
|
| 11 |
-
This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model.
|
| 12 |
-
BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets.
|
| 13 |
-
|
| 14 |
-
## Usage
|
| 15 |
-
|
| 16 |
-
To use this model, please install BERTopic:
|
| 17 |
-
|
| 18 |
-
```
|
| 19 |
-
pip install -U bertopic
|
| 20 |
-
```
|
| 21 |
-
|
| 22 |
-
You can use the model as follows:
|
| 23 |
-
|
| 24 |
-
```python
|
| 25 |
-
from bertopic import BERTopic
|
| 26 |
-
topic_model = BERTopic.load("D0men1c0/ISSR_Dark_Web_68Topics")
|
| 27 |
-
|
| 28 |
-
topic_model.get_topic_info()
|
| 29 |
-
```
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
|
| 47 |
-
|
|
| 48 |
-
|
|
| 49 |
-
|
|
| 50 |
-
|
|
| 51 |
-
|
|
| 52 |
-
|
|
| 53 |
-
|
|
| 54 |
-
|
|
| 55 |
-
|
|
| 56 |
-
|
|
| 57 |
-
|
|
| 58 |
-
|
|
| 59 |
-
|
|
| 60 |
-
|
|
| 61 |
-
|
|
| 62 |
-
|
|
| 63 |
-
|
|
| 64 |
-
|
|
| 65 |
-
|
|
| 66 |
-
|
|
| 67 |
-
|
|
| 68 |
-
|
|
| 69 |
-
|
|
| 70 |
-
|
|
| 71 |
-
|
|
| 72 |
-
|
|
| 73 |
-
|
|
| 74 |
-
|
|
| 75 |
-
|
|
| 76 |
-
|
|
| 77 |
-
|
|
| 78 |
-
|
|
| 79 |
-
|
|
| 80 |
-
|
|
| 81 |
-
|
|
| 82 |
-
|
|
| 83 |
-
|
|
| 84 |
-
|
|
| 85 |
-
|
|
| 86 |
-
|
|
| 87 |
-
|
|
| 88 |
-
|
|
| 89 |
-
|
|
| 90 |
-
|
|
| 91 |
-
|
|
| 92 |
-
|
|
| 93 |
-
|
|
| 94 |
-
|
|
| 95 |
-
|
|
| 96 |
-
|
|
| 97 |
-
|
|
| 98 |
-
|
|
| 99 |
-
|
|
| 100 |
-
|
|
| 101 |
-
|
|
| 102 |
-
|
|
| 103 |
-
|
|
| 104 |
-
|
|
| 105 |
-
|
|
| 106 |
-
|
|
| 107 |
-
|
|
| 108 |
-
|
|
| 109 |
-
|
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
*
|
| 123 |
-
*
|
| 124 |
-
*
|
| 125 |
-
*
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
*
|
| 130 |
-
*
|
| 131 |
-
*
|
| 132 |
-
*
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
*
|
| 137 |
-
*
|
| 138 |
-
*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
---
|
| 3 |
+
tags:
|
| 4 |
+
- bertopic
|
| 5 |
+
library_name: bertopic
|
| 6 |
+
pipeline_tag: text-classification
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# ISSR_Dark_Web_68Topics
|
| 10 |
+
|
| 11 |
+
This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model.
|
| 12 |
+
BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets.
|
| 13 |
+
|
| 14 |
+
## Usage
|
| 15 |
+
|
| 16 |
+
To use this model, please install BERTopic:
|
| 17 |
+
|
| 18 |
+
```
|
| 19 |
+
pip install -U bertopic
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
You can use the model as follows:
|
| 23 |
+
|
| 24 |
+
```python
|
| 25 |
+
from bertopic import BERTopic
|
| 26 |
+
topic_model = BERTopic.load("D0men1c0/ISSR_Dark_Web_68Topics")
|
| 27 |
+
|
| 28 |
+
topic_model.get_topic_info()
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
You can make predictions as follows:
|
| 32 |
+
```python
|
| 33 |
+
sentence = ['Fentanyl']
|
| 34 |
+
topic, _ = topic_model.transform(sentence)
|
| 35 |
+
topic_model.get_topic_info(topic[0])
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
## Topic overview
|
| 39 |
+
|
| 40 |
+
* Number of topics: 69
|
| 41 |
+
* Number of training documents: 65529
|
| 42 |
+
|
| 43 |
+
<details>
|
| 44 |
+
<summary>Click here for an overview of all topics.</summary>
|
| 45 |
+
|
| 46 |
+
| Topic ID | Topic Keywords | Topic Frequency | Label |
|
| 47 |
+
|----------|----------------|-----------------|-------|
|
| 48 |
+
| -1 | anyone - get - update - review - new | 178 | outliers |
|
| 49 |
+
| 0 | weed - cannabis - cart - thc - review | 18900 | Cannabis Weed Vape Cart Reviews |
|
| 50 |
+
| 1 | help - guy - sub - need - back | 5021 | Subreddit Help Needed |
|
| 51 |
+
| 2 | order - shipping - package - pack - delivery | 2035 | USPS Package Delivery |
|
| 52 |
+
| 3 | empire - empire market - empire empire - market - deposit | 2005 | Empire Market Deposit Issues |
|
| 53 |
+
| 4 | vendor - vendor vendor - vendor inquiry - inquiry - new vendor | 1815 | Trusted Vendor Inquiries |
|
| 54 |
+
| 5 | scammer - scam - exit - scamming - scammed | 3142 | Exit Scammer Warning |
|
| 55 |
+
| 6 | darknet - dark - web - dark web - darkfail | 1649 | Dark Web Drug Trafficking Arrests |
|
| 56 |
+
| 7 | mdma - mda - mdma vendor - domestic - usa | 1390 | MDMA Vendor USA Sale |
|
| 57 |
+
| 8 | xanax - mg - diazepam - xanax vendor - valium | 1319 | Xanax Vendor Xanax Bars |
|
| 58 |
+
| 9 | lsd - ug - tab - lsd vendor - acid | 1233 | LSD Vendor Tab List |
|
| 59 |
+
| 10 | crosspost - giveaway - review crosspost - crosspost vendor - review | 1012 | Giveaway Crossposts |
|
| 60 |
+
| 11 | monero - btc - bitcoin - coin - wallet | 999 | Monero Exchange |
|
| 61 |
+
| 12 | carding - card - credit - credit card - debit | 933 | Credit Card Service |
|
| 62 |
+
| 13 | dream - dream market - nightmare - market - dream dream | 909 | Dream Market |
|
| 63 |
+
| 14 | dispute - mod - moderator - dispute dispute - please | 882 | Dispute resolution |
|
| 64 |
+
| 15 | cocaine - cocaine vendor - fishscale - peruvian - colombian | 763 | Cocaine Vendor Fish |
|
| 65 |
+
| 16 | review - vendor review - review vendor - vendor - review review | 771 | Vendor Reviews Product |
|
| 66 |
+
| 17 | market - market market - new market - markets - marketplace | 998 | New Market Core |
|
| 67 |
+
| 18 | pgp - key - pgp key - public - public pgp | 1020 | Public PGP Key |
|
| 68 |
+
| 19 | deposit - deposited - ticket - address - double | 654 | Double Deposit Support Ticket |
|
| 69 |
+
| 20 | bar - bunk - bars - selaminy - hulk | 646 | Bar Reviews |
|
| 70 |
+
| 21 | oxycodone - mg - oxy - opiate - opiateconnect | 590 | Oxycodone and Dilaudid Purchase |
|
| 71 |
+
| 22 | id - passport - fake - fake id - license | 608 | Fake IDs and Licenses |
|
| 72 |
+
| 23 | drug - drugsuk - drugs - selling drug - drug dealer | 544 | Drug Misadvertizing Risks |
|
| 73 |
+
| 24 | coke - coke vendor - best coke - uk coke - uk | 556 | Coke Topic |
|
| 74 |
+
| 25 | pill - xtc - xtc pill - ecstasy - pills | 544 | xtc pills for sale |
|
| 75 |
+
| 26 | counterfeit - note - euro - money - counterfeit money | 509 | Counterfeit Notes |
|
| 76 |
+
| 27 | ketamine - ketamine vendor - ketamine review - mdma ketamine - review ketamine | 483 | Ketamine Vendor |
|
| 77 |
+
| 28 | wsm - wsm wsm - wsm vendor - vendor wsm - wsm order | 467 | WSM Vendor |
|
| 78 |
+
| 29 | meth - crystal meth - crystal - meth vendor - methamphetamine | 485 | Crystal Meth Vendor Review |
|
| 79 |
+
| 30 | ticket - support ticket - support - please - ticket support | 479 | Ticket Support |
|
| 80 |
+
| 31 | hacked - hacker - hacking - job - lfw | 480 | Hacker Developer Job Exploits |
|
| 81 |
+
| 32 | login - account - password - log - fa | 471 | Login Issues |
|
| 82 |
+
| 33 | adderall - mg - ir - ritalin - vyvanse | 470 | Adderall Pharmacy Brand Name |
|
| 83 |
+
| 34 | xmr - btc xmr - btc - xmrto - xmr btc | 474 | xmr xmr |
|
| 84 |
+
| 35 | tails - tail - electrum - wallet - whonix | 433 | Tails Electrum Monero Wallet Issue |
|
| 85 |
+
| 36 | mushroom - mushrooms - shrooms - magic - cubensis | 393 | Mushrooms Magic Penis |
|
| 86 |
+
| 37 | dread - dread dread - cafe dread - cafe - dread word | 371 | Cafe Dread Topics |
|
| 87 |
+
| 38 | cc - cvv - vbv - cc vendor - cc cvv | 419 | cc vending |
|
| 88 |
+
| 39 | cryptonia - cryptonia market - cryptonia cryptonia - dcdutchconnectionuk - empire cryptonia | 382 | Cryptonia Vendor |
|
| 89 |
+
| 40 | withdraw - withdrawal - withdrawl - withdraws - btc | 375 | Withdrawal Issues BTC |
|
| 90 |
+
| 41 | escrow - multisig - escrow escrow - full escrow - escrow order | 397 | Escrow Services and Multisig |
|
| 91 |
+
| 42 | heroin - heroin vendor - afghan - afghan heroin - synthetic heroin | 335 | Afghan Heroin Sale |
|
| 92 |
+
| 43 | de - har - noen - som - fra | 378 | Discussion Topics |
|
| 93 |
+
| 44 | dnm - dnms - dn - bible - dnstars | 341 | DNMS Bible |
|
| 94 |
+
| 45 | wallstreet - wall - wall street - street - wall st | 339 | Wall Street Market |
|
| 95 |
+
| 46 | ddos - ddos attack - attack - ddos ddos - ddos attacks | 306 | DDOS Attacks |
|
| 96 |
+
| 47 | paypal - transfer - paypal transfer - paypal account - western union | 283 | PayPal Transfer Scams |
|
| 97 |
+
| 48 | heard - happened - anyone - anyone heard - thewizzardnl | 327 | Document Mentions |
|
| 98 |
+
| 49 | benzos - benzo - rc - benzo vendor - rc benzos | 281 | Benzos Vendors |
|
| 99 |
+
| 50 | fraud - fraudsters - fraud vendor - loan fraud - fraudfox | 308 | Fraud Vendor Loan |
|
| 100 |
+
| 51 | dream - dream vendor - dream market - vendor dream - vendor | 326 | Dream Market Vendor Inquiry |
|
| 101 |
+
| 52 | order - cancel - cancelled - refund - cancel order | 497 | Order Cancelled |
|
| 102 |
+
| 53 | bank - bank log - bank drop - log - bank account | 331 | Bank Fraud Cards |
|
| 103 |
+
| 54 | onion - onion site - site - onion link - onion list | 328 | Onion links |
|
| 104 |
+
| 55 | phishing - phishing link - phished - link - warning | 245 | Phishing Warning |
|
| 105 |
+
| 56 | apollon - apollon market - market - apollon apollon - mysteryland | 253 | Apollon Market |
|
| 106 |
+
| 57 | opsec - opsec opsec - opsec question - bad opsec - question | 242 | Opsec and Guides |
|
| 107 |
+
| 58 | link - working link - working - pm - link please | 226 | PM Working Share Links |
|
| 108 |
+
| 59 | mirror - working mirror - working - mirror link - empire mirror | 229 | mirror link |
|
| 109 |
+
| 60 | fentanyl - fent - carfentanil - selling fentanyl - analogue | 219 | Fentanyl |
|
| 110 |
+
| 61 | cgmc - invite - invite code - code - cgmc invite | 221 | Invite Code CGMC |
|
| 111 |
+
| 62 | alprazolam - powder - alprazolam powder - flualprazolam - etizolam | 211 | Alprazolam Powder |
|
| 112 |
+
| 63 | dmt - dmt vendor - dmt vape - odsmt - dmt dmt | 290 | DMT Vendors |
|
| 113 |
+
| 64 | captcha - rapture - rapture market - captcha captcha - incorrect | 202 | Rapture Market Captcha |
|
| 114 |
+
| 65 | chemical - research - research chemical - chems - research chemicals | 187 | Research Chemicals |
|
| 115 |
+
| 66 | tor - tor browser - browser - tor network - network | 198 | Tor Browser Research |
|
| 116 |
+
| 67 | mephedrone - meopcp - mxe - mescaline - mmc | 222 | Mephedrone |
|
| 117 |
+
|
| 118 |
+
</details>
|
| 119 |
+
|
| 120 |
+
## Training hyperparameters
|
| 121 |
+
|
| 122 |
+
* calculate_probabilities: True
|
| 123 |
+
* language: None
|
| 124 |
+
* low_memory: False
|
| 125 |
+
* min_topic_size: 10
|
| 126 |
+
* n_gram_range: (1, 2)
|
| 127 |
+
* nr_topics: None
|
| 128 |
+
* seed_topic_list: None
|
| 129 |
+
* top_n_words: 10
|
| 130 |
+
* verbose: True
|
| 131 |
+
* zeroshot_min_similarity: 0.7
|
| 132 |
+
* zeroshot_topic_list: None
|
| 133 |
+
|
| 134 |
+
## Framework versions
|
| 135 |
+
|
| 136 |
+
* Numpy: 1.26.4
|
| 137 |
+
* HDBSCAN: 0.8.36
|
| 138 |
+
* UMAP: 0.5.6
|
| 139 |
+
* Pandas: 2.2.1
|
| 140 |
+
* Scikit-Learn: 1.4.1.post1
|
| 141 |
+
* Sentence-transformers: 3.0.1
|
| 142 |
+
* Transformers: 4.39.3
|
| 143 |
+
* Numba: 0.60.0
|
| 144 |
+
* Plotly: 5.22.0
|
| 145 |
+
* Python: 3.12.2
|