VulFixMiner-encoder / README.md
andstor's picture
Add metadata
ff18c0b
|
Raw History Blame Contribute Delete
1.95 kB
metadata
library_name: transformers
pipeline_tag: feature-extraction
datasets:
  - andstor/cvevc_commits
tags:
  - code
  - codebert
  - security
  - commit-embeddings
  - custom_code

VulFixMiner Encoder

A CodeBERT-based encoder for commit-diff embeddings.

Overview

This package provides a CodeBERT-based embedding model for code change snippets. It is designed to convert a diff or patch fragment into a dense semantic representation suitable for downstream security classification.

Model details

  • Base model: CodeBERT
  • Output representation: 768-dimensional embedding
  • Use case: feature extraction for commit-level security classification

Training data

Fine-tuned on andstor/cvevc_commits. The encoder learns from individual file changes extracted from these commits.

Example

from transformers import AutoTokenizer, AutoModel
import torch

model_name = "andstor/vulfixminer-encoder"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
model.eval()

patch = """
@@ -1,3 +1,4 @@
- old_call()
+ fixed_call()
+ validate_input()
"""

inputs = tokenizer(patch, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    outputs = model(**inputs)
    embedding = outputs.last_hidden_state[:, 0, :]
    print(embedding.shape)

Notes

  • Commit inputs: Remove comments and empty changes, then encode each file as added code + tokenizer.sep_token + deleted code. The raw-diff example above demonstrates the API, not this preprocessing.
  • Commit embeddings: Tokenize files separately with truncation and padding to 256 tokens. Average their first-token embeddings with mean(dim=0, keepdim=True) to obtain a [1, 768] vector for andstor/vulfixminer-classifier.