--- library_name: transformers license: apache-2.0 base_model: microsoft/unixcoder-base tags: - generated_from_trainer metrics: - accuracy - f1 - precision - recall model-index: - name: CodeDetect-UniXCoder-T2 results: [] --- # CodeDetect-UniXCoder-T2 This model is a fine-tuned version of [microsoft/unixcoder-base](https://huggingface.co/microsoft/unixcoder-base) ### 2. Load the Model ```python import torch from transformers import AutoTokenizer, AutoModelForSequenceClassification MODEL_NAME = "azherali/CodeDetect-UniXCoder-T2" MAX_LENGTH = 512 # Select device device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print("Device:", device) # Load tokenizer and model tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME) # Configure class labels model.config.id2label = { "0": "human", "1": "qwen", "2": "phi", "3": "meta-llama", "4": "01-ai", "5": "gemma", "6": "ibm-granite", "7": "bigcode", "8": "deepseek", "9": "gpt", "10": "mistral" } model.config.label2id = { "human": 0, "qwen": 1, "phi": 2, "meta-llama": 3, "01-ai": 4, "gemma": 5, "ibm-granite": 6, "bigcode": 7, "deepseek": 8, "gpt": 9, "mistral": 10 } model.to(device) model.eval() print("Number of classes:", model.config.num_labels) print("Label mapping:", model.config.id2label) ``` ### 3. Prediction Function The `predict_code()` function takes source code as input and returns the predicted label, class ID, and probability scores. ```python def predict_code(code: str) -> dict: if not isinstance(code, str) or not code.strip(): raise ValueError("Please provide a non-empty code string.") # Tokenize source code inputs = tokenizer( code, truncation=True, max_length=MAX_LENGTH, padding=True, return_tensors="pt", ) # Move inputs to the selected device inputs = { key: value.to(device) for key, value in inputs.items() } # Perform inference with torch.inference_mode(): outputs = model(**inputs) probabilities = torch.softmax(outputs.logits, dim=-1)[0] # Get predicted class predicted_id = int(probabilities.argmax().item()) labels = model.config.id2label or {} predicted_label = labels.get( predicted_id, labels.get(str(predicted_id), f"class_{predicted_id}") ) # Return prediction and class probabilities return { "predicted_class_id": predicted_id, "predicted_label": predicted_label, "class_probabilities": { str(labels.get(i, labels.get(str(i), f"class_{i}"))): round(float(probabilities[i].item()), 4) for i in range(len(probabilities)) }, } ``` ### 4. Example Inference ```python code = """ def add(a, b): return a + b """ result = predict_code(code) print(result) ``` ### 5. Example Output The following output is illustrative. Actual predictions and scores depend on the input and model behavior. ```python { "predicted_class_id": 9, "predicted_label": "gpt", "class_probabilities": { "human": 0.0001, "qwen": 0.4001, "phi": 0.0014, "meta-llama": 0.0405, "01-ai": 0.0024, "gemma": 0.0007, "ibm-granite": 0.0034, "bigcode": 0.0, "deepseek": 0.0162, "gpt": 0.5345, "mistral": 0.0007, }, } ``` ### 6. Output Description | Field | Description | |---|---| | `predicted_class_id` | Predicted class index: `0`-->`110`. | | `predicted_label` | Predicted class: `Human` `qwen`, `phi` etc. | | `class_probabilities` | Softmax scores for each class. | ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 0.1 - num_epochs: 5 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall | |:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|:---------:|:------:| | 0.9684 | 1.0 | 13235 | 1.0380 | 0.6277 | 0.5885 | 0.5892 | 0.6279 | | 0.7011 | 2.0 | 26470 | 1.0398 | 0.6485 | 0.6099 | 0.6051 | 0.6334 | | 0.4888 | 3.0 | 39705 | 1.1497 | 0.6559 | 0.6108 | 0.6075 | 0.6221 | | 0.3457 | 4.0 | 52940 | 1.2836 | 0.6573 | 0.6139 | 0.6089 | 0.6235 | | 0.2443 | 5.0 | 66175 | 1.3991 | 0.6569 | 0.6143 | 0.6120 | 0.6191 | ### Framework versions - Transformers 5.13.1 - Pytorch 2.11.0+cu130 - Datasets 4.0.0 - Tokenizers 0.22.2