gpt2-coder

Model Description

This is a fine-tuned version of GPT 2 (124.2M parameters) , trained on codeparrot.

Training Details

  • Training Data: [codeparrot]
  • Training Method: Fine-tuning
  • Training Duration: [8 hours/days]
  • Hardware: [V100]

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("smoothich/gpt2-coder")
tokenizer = AutoTokenizer.from_pretrained("smoothich/gpt2-coder")

# Generate text
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Parameters

  • Learning Rate: 5e-4
  • Batch Size: 16
  • Gradient Accumulation: 16
  • Epochs: 1
  • Precision: BF16

Evaluation

[Include evaluation metrics if available]

License

This model is released under the Apache 2.0 license.

Downloads last month
10
Safetensors
Model size
0.1B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for smoothich/gpt2-coder

Finetuned
(991)
this model