|
|
--- |
|
|
base_model: allenai/OLMo-1B-hf |
|
|
library_name: peft |
|
|
--- |
|
|
|
|
|
# OLMo Code Python2-3 Tagged Model |
|
|
|
|
|
This is a LoRA adapter fine-tuned on the OLMo-1B model for Python 2 and 3 code generation tasks with language tagging. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Base Model:** allenai/OLMo-1B-hf |
|
|
- **Model Type:** LoRA Adapter |
|
|
- **Task:** Causal Language Modeling for Python 2 and 3 code |
|
|
- **Language:** Python 2 and 3 with language tagging |
|
|
- **License:** MIT |
|
|
- **Fine-tuned by:** dipikakhullar |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This model is a LoRA adapter that has been fine-tuned on Python 2 and 3 code data with language tagging. It extends the capabilities of the base OLMo-1B model specifically for Python code generation tasks, with the ability to distinguish between Python 2 and Python 3 syntax. |
|
|
|
|
|
### LoRA Configuration |
|
|
|
|
|
- **LoRA Type:** LORA |
|
|
- **LoRA Alpha:** 16 |
|
|
- **LoRA Dropout:** 0.05 |
|
|
- **LoRA Rank (r):** 8 |
|
|
- **Target Modules:** down_proj, q_proj, v_proj, up_proj, k_proj, gate_proj, o_proj |
|
|
- **Task Type:** CAUSAL_LM |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
This model is intended for Python 2 and 3 code generation tasks with language tagging. It can be used to: |
|
|
- Generate Python code completions for both Python 2 and 3 |
|
|
- Assist with code writing in both Python versions |
|
|
- Provide code suggestions with language awareness |
|
|
- Handle Python 2 to Python 3 migration tasks |
|
|
|
|
|
### Downstream Use |
|
|
|
|
|
The model can be further fine-tuned for specific Python programming tasks or integrated into code generation applications that need to handle both Python versions. |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
This model is specifically designed for Python 2 and 3 code generation and may not perform well for: |
|
|
- Other programming languages |
|
|
- Natural language tasks |
|
|
- Non-code related tasks |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
|
|
```python |
|
|
from peft import PeftModel, PeftConfig |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
# Load the base model and tokenizer |
|
|
base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1B-hf") |
|
|
tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1B-hf") |
|
|
|
|
|
# Load the LoRA adapter |
|
|
model = PeftModel.from_pretrained(base_model, "dipikakhullar/olmo-code-python2-3-tagged") |
|
|
|
|
|
# Example usage for Python 3 |
|
|
prompt = "[python3] def fibonacci(n):" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_length=100, temperature=0.7) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
|
|
|
# Example usage for Python 2 |
|
|
prompt = "[python2] def fibonacci(n):" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_length=100, temperature=0.7) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
The model was fine-tuned on cleaned Python 2 and 3 code data with language tagging, specifically prepared for language model training. |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
- **Base Model:** allenai/OLMo-1B-hf |
|
|
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation) |
|
|
- **Checkpoint:** checkpoint-2100 |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
- **Author:** dipikakhullar |
|
|
- **Repository:** https://huggingface.co/dipikakhullar/olmo-code-python2-3-tagged |
|
|
|
|
|
## Framework versions |
|
|
|
|
|
- PEFT 0.7.1 |
|
|
- Transformers |
|
|
|
