| language: | |
| - en | |
| tags: | |
| - t5 | |
| - qa | |
| - askscience | |
| - lfqa | |
| - information retrieval | |
| datasets: | |
| - eli5 | |
| metrics: | |
| - rouge | |
| widget: | |
| - text: "why aren't there more planets in our solar system?" | |
| example_title: "solar system" | |
| - text: "question: what is a probability distribution? context: I am just learning about statistics." | |
| example_title: "probability distribution" | |
| - text: "question: What are the underlying physical processes by which exercise helps us lose weight? context: I started working out two weeks ago and already feel a lot better, and started to think about it and became deeply confused." | |
| example_title: "pumpen" | |
| - text: "what is a neural network?" | |
| example_title: "deep learning" | |
| - text: "What are the primary mechanisms that computers use to understand human language?" | |
| example_title: "NLP" | |
| inference: | |
| parameters: | |
| max_length: 96 | |
| no_repeat_ngram_size: 2 | |
| encoder_no_repeat_ngram_size: 4 | |
| repetition_penalty: 3.51 | |
| length_penalty: 0.8 | |
| num_beams: 4 | |
| early_stopping: True | |
| # t5 - base- askscience | |
| - [t5-v1_1](https://huggingface.co/google/t5-v1_1-base) trained on the entirety of the _askscience_ sub-section of the eli5 dataset for one epoch. | |
| - compare to bart on eli5 [here](https://huggingface.co/yjernite/bart_eli5) | |
| - note that for the inference API, the model is restricted to outputting 96 tokens - by using the model in python with the transformers library, you can get longer outputs. | |
| ## training | |
| - for inputs, the model was presented with the post title and the post selftext encoded as: `question: <post title> context: <post selftext>`. You may see better results if queries are posed in this fashion. | |
| - The top two replies were aggregated and presented to the model as the output text. | |
| - Training for longer will be explored, but given that the dataset has 127k examples and the loss flatlines at 0.5 epochs so this model should be fairly viable. |