metadata
license: mit
library_name: transformers
CodeGenModel
1. Introduction
CodeGenModel is specialized for code generation tasks. This model has been selected as the best checkpoint based on code generation benchmark performance.
The model demonstrates outstanding performance in code-related tasks while maintaining strong general capabilities.
2. Evaluation Results
Comprehensive Benchmark Results
| Benchmark | CodeModel-v1 | CodeModel-v2 | CodeGenModel | |
|---|---|---|---|---|
| Core Reasoning Tasks | Math Reasoning | 0.510 | 0.535 | 0.550 |
| Logical Reasoning | 0.789 | 0.801 | 0.819 | |
| Common Sense | 0.716 | 0.702 | 0.736 | |
| Language Understanding | Reading Comprehension | 0.671 | 0.685 | 0.700 |
| Question Answering | 0.582 | 0.599 | 0.607 | |
| Text Classification | 0.803 | 0.811 | 0.828 | |
| Sentiment Analysis | 0.777 | 0.781 | 0.792 | |
| Generation Tasks | Code Generation | 0.615 | 0.631 | 0.650 |
| Creative Writing | 0.588 | 0.579 | 0.644 | |
| Dialogue Generation | 0.621 | 0.635 | 0.767 | |
| Summarization | 0.745 | 0.755 | 0.767 | |
| Specialized Capabilities | Translation | 0.782 | 0.799 | 0.804 |
| Knowledge Retrieval | 0.651 | 0.668 | 0.676 | |
| Instruction Following | 0.733 | 0.749 | 0.758 | |
| Safety Evaluation | 0.718 | 0.701 | 0.739 |
Overall Performance Summary
CodeGenModel achieves top performance on code generation tasks among all checkpoints in this training run.
3. How to Use
Installation
pip install transformers
Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("CodeGenModel")
tokenizer = AutoTokenizer.from_pretrained("CodeGenModel")
4. License
This repository is licensed under the MIT License.
5. Contact
Open an issue on our GitHub for questions.