Instructions to use dzungpham/graphcodebert-code-classification with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use dzungpham/graphcodebert-code-classification with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("dzungpham/graphcodebert-code-classification", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
metrics:
|
|
@@ -14,61 +15,63 @@ tags:
|
|
| 14 |
- transformers
|
| 15 |
- bert
|
| 16 |
---
|
| 17 |
-
## 🔍 Task Overview
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
| 24 |
|
| 25 |
---
|
| 26 |
|
| 27 |
### Subtask A: Binary Machine-Generated Code Detection
|
| 28 |
|
| 29 |
**Goal:**
|
| 30 |
-
Given a code snippet,
|
| 31 |
|
| 32 |
-
-
|
| 33 |
-
-
|
| 34 |
|
| 35 |
-
**Training Languages:**
|
| 36 |
-
**Training Domain:**
|
| 37 |
|
| 38 |
**Evaluation Settings:**
|
| 39 |
|
| 40 |
-
| Setting | Language | Domain
|
| 41 |
-
|--------------------------------------|-------------------------|----------------------
|
| 42 |
-
| (i) Seen Languages & Seen Domains | C++, Python, Java | Algorithmic
|
| 43 |
-
| (ii) Unseen Languages & Seen Domains | Go, PHP, C#, C, JS | Algorithmic
|
| 44 |
-
| (iii) Seen Languages & Unseen Domains| C++, Python, Java | Research, Production
|
| 45 |
-
| (iv) Unseen Languages & Domains | Go, PHP, C#, C, JS | Research, Production
|
| 46 |
|
| 47 |
-
**Dataset Size**
|
| 48 |
-
- Train
|
| 49 |
-
- Validation
|
| 50 |
|
| 51 |
-
**Data Format**
|
| 52 |
-
Each dataset
|
| 53 |
-
- `code`: The code snippet
|
| 54 |
-
- `label`:
|
| 55 |
-
- `language`:
|
| 56 |
|
| 57 |
Label mappings are provided in `task_A/label_to_id.json` and `task_A/id_to_label.json`.
|
| 58 |
|
| 59 |
-
**Evaluation Metric**
|
| 60 |
-
The primary
|
| 61 |
|
| 62 |
-
**Submission Format**
|
| 63 |
-
Participants must submit a `.csv` file
|
| 64 |
-
- `id`: Unique identifier for each code snippet
|
| 65 |
-
- `label`: Predicted label (0 or 1)
|
| 66 |
|
| 67 |
-
A sample submission file is available in the `task_A/`
|
| 68 |
|
| 69 |
-
**Baseline Models**
|
| 70 |
-
Baseline implementations
|
| 71 |
|
| 72 |
-
**Restrictions**
|
| 73 |
-
-
|
| 74 |
-
-
|
|
|
|
|
|
| 1 |
+
```
|
| 2 |
---
|
| 3 |
license: mit
|
| 4 |
metrics:
|
|
|
|
| 15 |
- transformers
|
| 16 |
- bert
|
| 17 |
---
|
|
|
|
| 18 |
|
| 19 |
+
## Task Overview
|
| 20 |
|
| 21 |
+
The rapid advancement of generative models has made it increasingly challenging to distinguish machine-generated code from human-written code, particularly across different programming languages, domains, and generation techniques.
|
| 22 |
|
| 23 |
+
SemEval-2026 Task 13 focuses on developing systems capable of detecting machine-generated code under diverse conditions. The evaluation emphasizes generalization to unseen programming languages, generator families, and application scenarios.
|
| 24 |
+
|
| 25 |
+
The task is divided into three subtasks.
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
### Subtask A: Binary Machine-Generated Code Detection
|
| 30 |
|
| 31 |
**Goal:**
|
| 32 |
+
Given a code snippet, determine whether it is:
|
| 33 |
|
| 34 |
+
- Fully human-written, or
|
| 35 |
+
- Fully machine-generated
|
| 36 |
|
| 37 |
+
**Training Languages:** C++, Python, Java
|
| 38 |
+
**Training Domain:** Algorithmic (e.g., LeetCode-style problems)
|
| 39 |
|
| 40 |
**Evaluation Settings:**
|
| 41 |
|
| 42 |
+
| Setting | Language | Domain |
|
| 43 |
+
|--------------------------------------|-------------------------|----------------------|
|
| 44 |
+
| (i) Seen Languages & Seen Domains | C++, Python, Java | Algorithmic |
|
| 45 |
+
| (ii) Unseen Languages & Seen Domains | Go, PHP, C#, C, JS | Algorithmic |
|
| 46 |
+
| (iii) Seen Languages & Unseen Domains| C++, Python, Java | Research, Production |
|
| 47 |
+
| (iv) Unseen Languages & Domains | Go, PHP, C#, C, JS | Research, Production |
|
| 48 |
|
| 49 |
+
**Dataset Size:**
|
| 50 |
+
- Train: 500,000 samples (238,000 human-written, 262,000 machine-generated)
|
| 51 |
+
- Validation: 100,000 samples
|
| 52 |
|
| 53 |
+
**Data Format:**
|
| 54 |
+
Each dataset includes the following fields:
|
| 55 |
+
- `code`: The code snippet
|
| 56 |
+
- `label`: Binary label (0 for human-written, 1 for machine-generated)
|
| 57 |
+
- `language`: Programming language of the snippet
|
| 58 |
|
| 59 |
Label mappings are provided in `task_A/label_to_id.json` and `task_A/id_to_label.json`.
|
| 60 |
|
| 61 |
+
**Evaluation Metric:**
|
| 62 |
+
The primary metric for Subtask A is Macro F1-score, ensuring balanced performance across both classes.
|
| 63 |
|
| 64 |
+
**Submission Format:**
|
| 65 |
+
Participants must submit a `.csv` file containing:
|
| 66 |
+
- `id`: Unique identifier for each code snippet
|
| 67 |
+
- `label`: Predicted label (0 or 1)
|
| 68 |
|
| 69 |
+
A sample submission file is available in the `task_A/` directory.
|
| 70 |
|
| 71 |
+
**Baseline Models:**
|
| 72 |
+
Baseline implementations are provided in the `baselines/` directory, including starter code and pre-trained checkpoints for models such as GraphCodeBERT and UniXcoder.
|
| 73 |
|
| 74 |
+
**Restrictions:**
|
| 75 |
+
- No external training data may be used; only the provided datasets are allowed.
|
| 76 |
+
- Specialized AI-generated code detectors are not permitted. General-purpose code models (e.g., CodeBERT, StarCoder) are allowed.
|
| 77 |
+
```
|