--- library_name: transformers pipeline_tag: feature-extraction datasets: - andstor/cvevc_commits tags: - code - codebert - security - commit-embeddings - custom_code --- # VulFixMiner Encoder A CodeBERT-based encoder for commit-diff embeddings. ## Overview This package provides a CodeBERT-based embedding model for code change snippets. It is designed to convert a diff or patch fragment into a dense semantic representation suitable for downstream security classification. ## Model details - Base model: CodeBERT - Output representation: 768-dimensional embedding - Use case: feature extraction for commit-level security classification ## Training data Fine-tuned on [andstor/cvevc_commits](https://huggingface.co/datasets/andstor/cvevc_commits). The encoder learns from individual file changes extracted from these commits. ## Example ```python from transformers import AutoTokenizer, AutoModel import torch model_name = "andstor/vulfixminer-encoder" tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) model = AutoModel.from_pretrained(model_name, trust_remote_code=True) model.eval() patch = """ @@ -1,3 +1,4 @@ - old_call() + fixed_call() + validate_input() """ inputs = tokenizer(patch, return_tensors="pt", truncation=True, max_length=256) with torch.no_grad(): outputs = model(**inputs) embedding = outputs.last_hidden_state[:, 0, :] print(embedding.shape) ``` ## Notes - **Commit inputs:** Remove comments and empty changes, then encode each file as added code + `tokenizer.sep_token` + deleted code. The raw-diff example above demonstrates the API, not this preprocessing. - **Commit embeddings:** Tokenize files separately with truncation and padding to 256 tokens. Average their first-token embeddings with `mean(dim=0, keepdim=True)` to obtain a `[1, 768]` vector for [andstor/vulfixminer-classifier](https://huggingface.co/andstor/vulfixminer-classifier).