VulFixMiner-encoder / README.md
andstor's picture
Add metadata
ff18c0b
|
Raw History Blame Contribute Delete
1.95 kB
---
library_name: transformers
pipeline_tag: feature-extraction
datasets:
- andstor/cvevc_commits
tags:
- code
- codebert
- security
- commit-embeddings
- custom_code
---
# VulFixMiner Encoder
A CodeBERT-based encoder for commit-diff embeddings.
## Overview
This package provides a CodeBERT-based embedding model for code change snippets. It is designed to convert a diff or patch fragment into a dense semantic representation suitable for downstream security classification.
## Model details
- Base model: CodeBERT
- Output representation: 768-dimensional embedding
- Use case: feature extraction for commit-level security classification
## Training data
Fine-tuned on [andstor/cvevc_commits](https://huggingface.co/datasets/andstor/cvevc_commits).
The encoder learns from individual file changes extracted from these commits.
## Example
```python
from transformers import AutoTokenizer, AutoModel
import torch
model_name = "andstor/vulfixminer-encoder"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
model.eval()
patch = """
@@ -1,3 +1,4 @@
- old_call()
+ fixed_call()
+ validate_input()
"""
inputs = tokenizer(patch, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
outputs = model(**inputs)
embedding = outputs.last_hidden_state[:, 0, :]
print(embedding.shape)
```
## Notes
- **Commit inputs:** Remove comments and empty changes, then encode each file as added code + `tokenizer.sep_token` + deleted code. The raw-diff example above demonstrates the API, not this preprocessing.
- **Commit embeddings:** Tokenize files separately with truncation and padding to 256 tokens. Average their first-token embeddings with `mean(dim=0, keepdim=True)` to obtain a `[1, 768]` vector for [andstor/vulfixminer-classifier](https://huggingface.co/andstor/vulfixminer-classifier).