Sumeetgpt commited on
Commit
d8a0b7c
·
verified ·
1 Parent(s): cec7e9f

Push model using huggingface_hub.

Browse files
1_Pooling/config.json CHANGED
@@ -1,10 +1,5 @@
1
  {
2
- "word_embedding_dimension": 384,
3
- "pooling_mode_cls_token": false,
4
- "pooling_mode_mean_tokens": true,
5
- "pooling_mode_max_tokens": false,
6
- "pooling_mode_mean_sqrt_len_tokens": false,
7
- "pooling_mode_weightedmean_tokens": false,
8
- "pooling_mode_lasttoken": false,
9
  "include_prompt": true
10
  }
 
1
  {
2
+ "embedding_dimension": 384,
3
+ "pooling_mode": "mean",
 
 
 
 
 
4
  "include_prompt": true
5
  }
README.md CHANGED
@@ -3,281 +3,93 @@ tags:
3
  - setfit
4
  - sentence-transformers
5
  - text-classification
6
- - code-security
7
- - dlp
8
- - secret-detection
9
- - vulnerability-detection
10
- language:
11
- - en
12
- license: apache-2.0
13
- widget:
14
- - text: "Analyze this hardcoded_secret: AWS_ACCESS_KEY_ID=AKIA4REALKEY123ABC committed to main branch .env file"
15
- - text: "Analyze this vulnerable_pattern: $query = 'SELECT * FROM users WHERE id=' . $_GET['id'];"
16
- - text: "Analyze this test_fixture: factory_boy default: user.password = 'testpass123' for pytest fixtures"
17
- - text: "Analyze this clean_code: def get_user(db: Session, user_id: int): return db.query(User).filter(User.id == user_id).first()"
18
- - text: "Analyze this secure_implementation: password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt(rounds=12))"
19
  metrics:
20
  - accuracy
21
  pipeline_tag: text-classification
22
  library_name: setfit
23
  inference: true
24
- model-index:
25
- - name: spidercob/code-risk-classifier
26
- results:
27
- - task:
28
- type: text-classification
29
- name: Text Classification
30
- dataset:
31
- name: curated-public-repos-v2
32
- type: custom
33
- split: test
34
- metrics:
35
- - type: accuracy
36
- value: 1.0
37
- name: Accuracy
38
  ---
39
 
40
- # spidercob/code-risk-classifier
41
 
42
- A [SetFit](https://github.com/huggingface/setfit) model that classifies code snippets and secrets into four risk categories. Used inside [Spidercob](https://spidercob.com) to reduce false positives in supply-chain and secret scanning — distinguishing real production risks from test fixtures and safe code.
43
 
44
- **v2** expanded multi-language training corpus (22 public repos, Python / Java / PHP / JavaScript / TypeScript / Ruby / Go).
45
 
46
- ## Labels
 
47
 
48
- | Label | Meaning | Action |
49
- |---|---|---|
50
- | `REAL_SECRET` | Hardcoded credential, API key, or token committed to source | **BLOCK** |
51
- | `VULNERABLE_LOGIC` | SQL injection, XSS, unsafe deserialization, command injection, etc. | **BLOCK** |
52
- | `TEST_MOCK` | Dummy credential or vuln pattern inside a test fixture or factory | ALLOW |
53
- | `SAFE_CODE` | Clean production code, secure implementation pattern | ALLOW |
54
 
55
- ## Quick Start
 
 
 
 
 
 
 
 
56
 
57
- ```python
58
- from setfit import SetFitModel
 
 
 
59
 
60
- model = SetFitModel.from_pretrained("spidercob/code-risk-classifier")
61
 
62
- snippets = [
63
- "Analyze this hardcoded_secret: AWS_ACCESS_KEY_ID=AKIA4REALKEY123ABC in .env",
64
- "Analyze this vulnerable_pattern: $q = 'SELECT * FROM users WHERE id=' . $_GET['id'];",
65
- "Analyze this test_fixture: user.password = 'testpass123' # factory_boy default",
66
- "Analyze this clean_code: password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt())",
67
- ]
68
 
69
- predictions = model.predict(snippets)
70
- probabilities = model.predict_proba(snippets)
71
- # predictions: ['REAL_SECRET', 'VULNERABLE_LOGIC', 'TEST_MOCK', 'SAFE_CODE']
 
72
  ```
73
 
74
- ## Input Format
75
 
76
- Inputs must follow the pattern used during training:
 
77
 
78
- ```
79
- Analyze this <issue_type>: <code_snippet_or_context>
 
 
80
  ```
81
 
82
- Where `<issue_type>` is a short descriptor such as `hardcoded_secret`, `vulnerable_pattern`, `test_fixture`, `clean_code`, `secure_implementation`, etc.
 
83
 
84
- ## Integration with Spidercob DLP
 
85
 
86
- ```python
87
- from app.core.code_risk_classifier import CodeRiskClassifier
88
 
89
- clf = CodeRiskClassifier()
90
- result = clf.classify("hardcoded_secret", "STRIPE_SECRET_KEY = 'sk_live_abc123xyz'")
91
- # result: {"risk_type": "REAL_SECRET", "confidence": 0.99, "action": "BLOCK", "severity": "CRITICAL"}
92
- ```
 
 
 
 
 
 
 
 
 
 
93
 
94
- The classifier loads the fine-tuned SetFit model at import time and falls back to zero-shot cosine similarity if the model directory is absent.
95
-
96
- ## Evaluation
97
-
98
- | Label | Accuracy |
99
- |:--------|:---------|
100
- | **all** | **1.0** (100%, 176 test examples) |
101
-
102
- ## Training Details (v2)
103
-
104
- ### Dataset
105
-
106
- 877 labelled examples extracted from **22 curated public GitHub repositories** via regex-based heuristics. Each example is a 3–7 line context window around a matched line, formatted as `Analyze this <issue_type>: <context>`.
107
-
108
- **Languages covered:** Python, Java, PHP, JavaScript, TypeScript, Ruby, Go
109
-
110
- ### Training Set Composition
111
-
112
- | Label | Train examples | Source strategy |
113
- |---|---|---|
114
- | `REAL_SECRET` | 27 | TruffleHog test corpus (confirmed leaked secrets), Railsgoat, NodeGoat, Juice Shop |
115
- | `VULNERABLE_LOGIC` | 194 | DVWA (PHP), WebGoat (Java), vulhub, NodeGoat, DVGA, Railsgoat, Juice Shop |
116
- | `TEST_MOCK` | 240 | factory_boy, Faker, pytest, model_bakery |
117
- | `SAFE_CODE` | 240 | Django, FastAPI, requests, Flask, httpx, Devise, Sinatra, Gin |
118
-
119
- ### Source Repositories
120
-
121
- | Repo | Label | Language |
122
- |---|---|---|
123
- | OWASP/WebGoat | VULNERABLE_LOGIC | Java |
124
- | digininja/DVWA | VULNERABLE_LOGIC | PHP |
125
- | vulhub/vulhub | VULNERABLE_LOGIC | multi |
126
- | trufflesecurity/trufflehog | REAL_SECRET | Go/multi |
127
- | OWASP/NodeGoat | VULNERABLE_LOGIC | JavaScript |
128
- | dolevf/Damn-Vulnerable-GraphQL-Application | VULNERABLE_LOGIC | Python |
129
- | OWASP/railsgoat | VULNERABLE_LOGIC | Ruby |
130
- | juice-shop/juice-shop | VULNERABLE_LOGIC | TypeScript |
131
- | FactoryBoy/factory_boy | TEST_MOCK | Python |
132
- | joke2k/faker | TEST_MOCK | Python |
133
- | pytest-dev/pytest | TEST_MOCK | Python |
134
- | model-bakers/model_bakery | TEST_MOCK | Python |
135
- | django/django | SAFE_CODE | Python |
136
- | tiangolo/fastapi | SAFE_CODE | Python |
137
- | psf/requests | SAFE_CODE | Python |
138
- | pallets/flask | SAFE_CODE | Python |
139
- | encode/httpx | SAFE_CODE | Python |
140
- | heartcombo/devise | SAFE_CODE | Ruby |
141
- | sinatra/sinatra | SAFE_CODE | Ruby |
142
- | gin-gonic/gin | SAFE_CODE | Go |
143
- | golang/vulndb | VULNERABLE_LOGIC | Go |
144
-
145
- ### Training Hyperparameters
146
- - batch_size: (16, 16)
147
- - num_epochs: (3, 3)
148
- - max_steps: -1
149
- - sampling_strategy: oversampling
150
- - num_iterations: 20
151
- - body_learning_rate: (2e-05, 1e-05)
152
- - head_learning_rate: 0.01
153
- - loss: CosineSimilarityLoss
154
- - distance_metric: cosine_distance
155
- - margin: 0.25
156
- - end_to_end: False
157
- - use_amp: False
158
- - warmup_proportion: 0.1
159
- - l2_weight: 0.01
160
- - seed: 42
161
- - eval_max_steps: -1
162
- - load_best_model_at_end: True
163
-
164
- ### Training Results
165
- | Epoch | Step | Training Loss | Validation Loss |
166
- |:------:|:----:|:-------------:|:---------------:|
167
- | 0.0006 | 1 | 0.0416 | - |
168
- | 0.0285 | 50 | 0.0099 | - |
169
- | 0.0570 | 100 | 0.0028 | - |
170
- | 0.0856 | 150 | 0.001 | - |
171
- | 0.1141 | 200 | 0.0013 | - |
172
- | 0.1426 | 250 | 0.0003 | - |
173
- | 0.1711 | 300 | 0.0002 | - |
174
- | 0.1997 | 350 | 0.0002 | - |
175
- | 0.2282 | 400 | 0.0002 | - |
176
- | 0.2567 | 450 | 0.0002 | - |
177
- | 0.2852 | 500 | 0.0002 | - |
178
- | 0.3137 | 550 | 0.0001 | - |
179
- | 0.3423 | 600 | 0.0001 | - |
180
- | 0.3708 | 650 | 0.0001 | - |
181
- | 0.3993 | 700 | 0.0001 | - |
182
- | 0.4278 | 750 | 0.0001 | - |
183
- | 0.4564 | 800 | 0.0001 | - |
184
- | 0.4849 | 850 | 0.0001 | - |
185
- | 0.5134 | 900 | 0.0001 | - |
186
- | 0.5419 | 950 | 0.0001 | - |
187
- | 0.5705 | 1000 | 0.0001 | - |
188
- | 0.5990 | 1050 | 0.0001 | - |
189
- | 0.6275 | 1100 | 0.0001 | - |
190
- | 0.6560 | 1150 | 0.0001 | - |
191
- | 0.6845 | 1200 | 0.0001 | - |
192
- | 0.7131 | 1250 | 0.0001 | - |
193
- | 0.7416 | 1300 | 0.0001 | - |
194
- | 0.7701 | 1350 | 0.0001 | - |
195
- | 0.7986 | 1400 | 0.0 | - |
196
- | 0.8272 | 1450 | 0.0001 | - |
197
- | 0.8557 | 1500 | 0.0001 | - |
198
- | 0.8842 | 1550 | 0.0 | - |
199
- | 0.9127 | 1600 | 0.0 | - |
200
- | 0.9412 | 1650 | 0.0001 | - |
201
- | 0.9698 | 1700 | 0.0 | - |
202
- | 0.9983 | 1750 | 0.0 | - |
203
- | 1.0 | 1753 | - | 0.0001 |
204
- | 1.0268 | 1800 | 0.0 | - |
205
- | 1.0553 | 1850 | 0.0 | - |
206
- | 1.0839 | 1900 | 0.0 | - |
207
- | 1.1124 | 1950 | 0.0 | - |
208
- | 1.1409 | 2000 | 0.0 | - |
209
- | 1.1694 | 2050 | 0.0 | - |
210
- | 1.1979 | 2100 | 0.0 | - |
211
- | 1.2265 | 2150 | 0.0 | - |
212
- | 1.2550 | 2200 | 0.0 | - |
213
- | 1.2835 | 2250 | 0.0 | - |
214
- | 1.3120 | 2300 | 0.0 | - |
215
- | 1.3406 | 2350 | 0.0 | - |
216
- | 1.3691 | 2400 | 0.0 | - |
217
- | 1.3976 | 2450 | 0.0 | - |
218
- | 1.4261 | 2500 | 0.0 | - |
219
- | 1.4546 | 2550 | 0.0 | - |
220
- | 1.4832 | 2600 | 0.0 | - |
221
- | 1.5117 | 2650 | 0.0 | - |
222
- | 1.5402 | 2700 | 0.0 | - |
223
- | 1.5687 | 2750 | 0.0 | - |
224
- | 1.5973 | 2800 | 0.0 | - |
225
- | 1.6258 | 2850 | 0.0 | - |
226
- | 1.6543 | 2900 | 0.0 | - |
227
- | 1.6828 | 2950 | 0.0 | - |
228
- | 1.7114 | 3000 | 0.0 | - |
229
- | 1.7399 | 3050 | 0.0 | - |
230
- | 1.7684 | 3100 | 0.0 | - |
231
- | 1.7969 | 3150 | 0.0 | - |
232
- | 1.8254 | 3200 | 0.0 | - |
233
- | 1.8540 | 3250 | 0.0 | - |
234
- | 1.8825 | 3300 | 0.0 | - |
235
- | 1.9110 | 3350 | 0.0 | - |
236
- | 1.9395 | 3400 | 0.0 | - |
237
- | 1.9681 | 3450 | 0.0 | - |
238
- | 1.9966 | 3500 | 0.0 | - |
239
- | 2.0 | 3506 | - | 0.0000 |
240
- | 2.0251 | 3550 | 0.0 | - |
241
- | 2.0536 | 3600 | 0.0 | - |
242
- | 2.0821 | 3650 | 0.0 | - |
243
- | 2.1107 | 3700 | 0.0 | - |
244
- | 2.1392 | 3750 | 0.0 | - |
245
- | 2.1677 | 3800 | 0.0 | - |
246
- | 2.1962 | 3850 | 0.0 | - |
247
- | 2.2248 | 3900 | 0.0 | - |
248
- | 2.2533 | 3950 | 0.0 | - |
249
- | 2.2818 | 4000 | 0.0 | - |
250
- | 2.3103 | 4050 | 0.0 | - |
251
- | 2.3388 | 4100 | 0.0 | - |
252
- | 2.3674 | 4150 | 0.0 | - |
253
- | 2.3959 | 4200 | 0.0 | - |
254
- | 2.4244 | 4250 | 0.0 | - |
255
- | 2.4529 | 4300 | 0.0 | - |
256
- | 2.4815 | 4350 | 0.0 | - |
257
- | 2.5100 | 4400 | 0.0 | - |
258
- | 2.5385 | 4450 | 0.0 | - |
259
- | 2.5670 | 4500 | 0.0 | - |
260
- | 2.5956 | 4550 | 0.0 | - |
261
- | 2.6241 | 4600 | 0.0 | - |
262
- | 2.6526 | 4650 | 0.0 | - |
263
- | 2.6811 | 4700 | 0.0 | - |
264
- | 2.7096 | 4750 | 0.0 | - |
265
- | 2.7382 | 4800 | 0.0 | - |
266
- | 2.7667 | 4850 | 0.0 | - |
267
- | 2.7952 | 4900 | 0.0 | - |
268
- | 2.8237 | 4950 | 0.0 | - |
269
- | 2.8523 | 5000 | 0.0 | - |
270
- | 2.8808 | 5050 | 0.0 | - |
271
- | 2.9093 | 5100 | 0.0 | - |
272
- | 2.9378 | 5150 | 0.0 | - |
273
- | 2.9663 | 5200 | 0.0 | - |
274
- | 2.9949 | 5250 | 0.0 | - |
275
- | 3.0 | 5259 | - | 0.0000 |
276
 
277
  ### Framework Versions
278
  - Python: 3.12.12
279
  - SetFit: 1.1.3
280
- - Sentence Transformers: 5.2.2
281
  - Transformers: 4.57.6
282
  - PyTorch: 2.10.0
283
  - Datasets: 5.0.0
 
3
  - setfit
4
  - sentence-transformers
5
  - text-classification
6
+ - generated_from_setfit_trainer
7
+ widget: []
 
 
 
 
 
 
 
 
 
 
 
8
  metrics:
9
  - accuracy
10
  pipeline_tag: text-classification
11
  library_name: setfit
12
  inference: true
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
  ---
14
 
15
+ # SetFit
16
 
17
+ This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
18
 
19
+ The model has been trained using an efficient few-shot learning technique that involves:
20
 
21
+ 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
22
+ 2. Training a classification head with features from the fine-tuned Sentence Transformer.
23
 
24
+ ## Model Details
 
 
 
 
 
25
 
26
+ ### Model Description
27
+ - **Model Type:** SetFit
28
+ <!-- - **Sentence Transformer:** [Unknown](https://huggingface.co/unknown) -->
29
+ - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
30
+ - **Maximum Sequence Length:** 256 tokens
31
+ - **Number of Classes:** 4 classes
32
+ <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
33
+ <!-- - **Language:** Unknown -->
34
+ <!-- - **License:** Unknown -->
35
 
36
+ ### Model Sources
37
+
38
+ - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
39
+ - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
40
+ - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
41
 
42
+ ## Uses
43
 
44
+ ### Direct Use for Inference
 
 
 
 
 
45
 
46
+ First install the SetFit library:
47
+
48
+ ```bash
49
+ pip install setfit
50
  ```
51
 
52
+ Then you can load this model and run inference.
53
 
54
+ ```python
55
+ from setfit import SetFitModel
56
 
57
+ # Download from the 🤗 Hub
58
+ model = SetFitModel.from_pretrained("setfit_model_id")
59
+ # Run inference
60
+ preds = model("I loved the spiderman movie!")
61
  ```
62
 
63
+ <!--
64
+ ### Downstream Use
65
 
66
+ *List how someone could finetune this model on their own dataset.*
67
+ -->
68
 
69
+ <!--
70
+ ### Out-of-Scope Use
71
 
72
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
73
+ -->
74
+
75
+ <!--
76
+ ## Bias, Risks and Limitations
77
+
78
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
79
+ -->
80
+
81
+ <!--
82
+ ### Recommendations
83
+
84
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
85
+ -->
86
 
87
+ ## Training Details
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
 
89
  ### Framework Versions
90
  - Python: 3.12.12
91
  - SetFit: 1.1.3
92
+ - Sentence Transformers: 5.6.1
93
  - Transformers: 4.57.6
94
  - PyTorch: 2.10.0
95
  - Datasets: 5.0.0
config_sentence_transformers.json CHANGED
@@ -1,14 +1,14 @@
1
  {
2
  "__version__": {
3
- "sentence_transformers": "5.2.2",
4
- "transformers": "4.57.6",
5
- "pytorch": "2.10.0"
6
  },
 
7
  "model_type": "SentenceTransformer",
8
  "prompts": {
9
- "query": "",
10
- "document": ""
11
  },
12
- "default_prompt_name": null,
13
  "similarity_fn_name": "cosine"
14
  }
 
1
  {
2
  "__version__": {
3
+ "pytorch": "2.10.0",
4
+ "sentence_transformers": "5.6.1",
5
+ "transformers": "4.57.6"
6
  },
7
+ "default_prompt_name": null,
8
  "model_type": "SentenceTransformer",
9
  "prompts": {
10
+ "document": "",
11
+ "query": ""
12
  },
 
13
  "similarity_fn_name": "cosine"
14
  }
config_setfit.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
- "normalize_embeddings": false,
3
  "labels": [
4
  "REAL_SECRET",
5
  "VULNERABLE_LOGIC",
6
  "TEST_MOCK",
7
  "SAFE_CODE"
8
- ]
 
9
  }
 
1
  {
 
2
  "labels": [
3
  "REAL_SECRET",
4
  "VULNERABLE_LOGIC",
5
  "TEST_MOCK",
6
  "SAFE_CODE"
7
+ ],
8
+ "normalize_embeddings": false
9
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9c4c93bb1d3444cdc0eec9908b2e6e61660c7606b020a6776b3bd450f315bee7
3
  size 90864192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f5e69a66f4af290805d351cbfd5968f48bb82dacf20a059800ca82265db9f6c
3
  size 90864192
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7e9f45b9e642972266844fc03150c434167110c84d54306ba7119d5b0fea50fc
3
  size 13191
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:908ee4643be4f4e547ae40fc8efe3dbf933f857655e62a9864a83e5389ea22e8
3
  size 13191
modules.json CHANGED
@@ -3,18 +3,18 @@
3
  "idx": 0,
4
  "name": "0",
5
  "path": "",
6
- "type": "sentence_transformers.models.Transformer"
7
  },
8
  {
9
  "idx": 1,
10
  "name": "1",
11
  "path": "1_Pooling",
12
- "type": "sentence_transformers.models.Pooling"
13
  },
14
  {
15
  "idx": 2,
16
  "name": "2",
17
  "path": "2_Normalize",
18
- "type": "sentence_transformers.models.Normalize"
19
  }
20
  ]
 
3
  "idx": 0,
4
  "name": "0",
5
  "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
  },
8
  {
9
  "idx": 1,
10
  "name": "1",
11
  "path": "1_Pooling",
12
+ "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
13
  },
14
  {
15
  "idx": 2,
16
  "name": "2",
17
  "path": "2_Normalize",
18
+ "type": "sentence_transformers.sentence_transformer.modules.normalize.Normalize"
19
  }
20
  ]
sentence_bert_config.json CHANGED
@@ -1,4 +1,10 @@
1
  {
2
- "max_seq_length": 256,
3
- "do_lower_case": false
 
 
 
 
 
 
4
  }
 
1
  {
2
+ "transformer_task": "feature-extraction",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "last_hidden_state"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
  }