| datasets: |
| - squad |
| - newsqa |
| - hotpot_qa |
| - biu-nlp/qamr |
| - search_qa |
| - natural_questions |
| - trivia_qa |
| - duorc |
| language: |
| - en |
| metrics: |
| - squad |
| --- |
|
|
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
| Checkpoint of MetaQA trained only on extractive QA datasets from MetaQA: Combining Expert Agents for Multi-Skill Question Answering (https://arxiv.org/abs/2112.01922) |
|
|
| ## Evaluation Results |
| ``` |
| { |
| "SQuAD": { |
| "exact_match": 86.73139158576052, |
| "f1": 92.65156746563402 |
| }, |
| "NewsQA": { |
| "exact_match": 55.84045584045584, |
| "f1": 71.73547617592037 |
| }, |
| "HotpotQA": { |
| "exact_match": 64.8135593220339, |
| "f1": 79.61023604916922 |
| }, |
| "SearchQA": { |
| "exact_match": 75.04122497055359, |
| "f1": 81.37280639135817 |
| }, |
| "NaturalQuestionsShort": { |
| "exact_match": 69.50763477718915, |
| "f1": 81.30374741690376 |
| }, |
| "TriviaQA-web": { |
| "exact_match": 77.18396711202466, |
| "f1": 81.52989853015538 |
| }, |
| "QAMR": { |
| "exact_match": 72.07531203723292, |
| "f1": 83.9068616637681 |
| }, |
| "DuoRC": { |
| "exact_match": 39.35626573106552, |
| "f1": 51.033295034422466 |
| } |
| } |
| ``` |
