t5 - base- askscience
- t5-v1_1 trained on the entirety of the askscience sub-section of the eli5 dataset for one epoch.
- compare to bart on eli5 here
- note that for the inference API, the model is restricted to outputting 96 tokens - by using the model in python with the transformers library, you can get longer outputs.
training
- for inputs, the model was presented with the post title and the post selftext encoded as:
question: <post title> context: <post selftext>. You may see better results if queries are posed in this fashion. - The top two replies were aggregated and presented to the model as the output text.
- Training for longer will be explored, but given that the dataset has 127k examples and the loss flatlines at 0.5 epochs so this model should be fairly viable.
- Downloads last month
- 12
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("pszemraj/t5-base-askscience") model = AutoModelForSeq2SeqLM.from_pretrained("pszemraj/t5-base-askscience")