| | --- |
| | license: mit |
| | --- |
| | |
| | # SRTK Scorer |
| |
|
| | This model is a trained scorer for [SRTK](https://github.com/happen2me/subgraph-retrieval-toolkit). It is used to compare the similarity between a query and the expansion path at the time of subgraph retrieval. |
| |
|
| | ## Training Information |
| |
|
| | It is initialized with `roberta-base`. It is trained jointly on the following datasets: |
| |
|
| | - [WebQSP for Freebase](https://www.microsoft.com/en-us/download/details.aspx?id=52763) |
| | - [SimpleQuestionsWikidata for Wikidata](https://github.com/askplatypus/wikidata-simplequestions) |
| | - [SimpleDBpediaQA](https://github.com/castorini/SimpleDBpediaQA) |
| |
|
| | It achieves an answer coverage rate of 0.9728 on SimpleQuestionsWikidata (depth 1) 0.8501 on WebQSP test set (depth 2) with a beam width of only 2! |
| |
|
| | ## Usage Example |
| |
|
| | First install the package: |
| |
|
| | ```bash |
| | pip install srtk |
| | ``` |
| |
|
| | Then you can retrieve subgraphs with the help of this scorer: |
| |
|
| | ```bash |
| | srtk retrieve -i data/wikidata-simplequestions/intermediate/scores_test.jsonl \ |
| | -o artifacts/subgraphs/wikidata-simple-contrast \ |
| | -e http://localhost:1234/api/endpoint/sparql \ |
| | --scorer-model-path drt/srtk-scorer \ |
| | --scorer --beam-width 2 --max-depth 1 --evaluate |
| | ``` |
| |
|
| | ## Limitations |
| |
|
| | As both SimpleQuestionsWikidata and SimpleDBpediaQA contain only one-hop relations, the model tends to stop at one-hop when you retrieve subgraphs on Wikidata and DBpedia. We will release a updated version of the model that is trained on a more diverse dataset in the future. |
| |
|
| | ## License |
| |
|
| | MIT |
| |
|
