TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Paper
• 2305.07759 • Published
• 45
einygpt
Here's the models I've trained using the transformer I wrote in einygpt. For reference they are:
- a multihead attention model replicating the model discussed in the TinyStories paper using the GPT2Tokenizer
- a multiquery attention model using the GPT2Tokenizer
- a grouped query attention model with the number of groups = 4 and using its own tokenizer
For playing with these model, you can view how they are used here
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support