# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("naxautify/gpt2-4k")
model = AutoModelForCausalLM.from_pretrained("naxautify/gpt2-4k", device_map="auto")Quick Links
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
GPT-2 (125M) 4k tokens
Fine-tuned GPT2 Smallest model on The Pile with a token length of 4k. Weights are included and it follows Karpathy's nanoGPT implementation. The model has been trained for ~1 million iterations with increasing batch size, ending at 32k. The final loss is 3.9 which is probably due to 768 embedding size.
- Downloads last month
- 11
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="naxautify/gpt2-4k")