---
license: mit
---

# SRTK Scorer

This model is a trained scorer for [SRTK](https://github.com/happen2me/subgraph-retrieval-toolkit). It is used to compare the similarity between a query and the expansion path at the time of subgraph retrieval.

## Training Information

It is initialized with `roberta-base`. It is trained jointly on the following datasets:

- [WebQSP for Freebase](https://www.microsoft.com/en-us/download/details.aspx?id=52763)
- [SimpleQuestionsWikidata for Wikidata](https://github.com/askplatypus/wikidata-simplequestions)
- [SimpleDBpediaQA](https://github.com/castorini/SimpleDBpediaQA)

It achieves an answer coverage rate of 0.9728 on SimpleQuestionsWikidata (depth 1) 0.8501 on WebQSP test set (depth 2) with a beam width of only 2!

## Usage Example

First install the package:

```bash
pip install srtk
```

Then you can retrieve subgraphs with the help of this scorer:

```bash
srtk retrieve -i data/wikidata-simplequestions/intermediate/scores_test.jsonl \
    -o artifacts/subgraphs/wikidata-simple-contrast \
    -e http://localhost:1234/api/endpoint/sparql \
    --scorer-model-path drt/srtk-scorer \
    --scorer  --beam-width 2 --max-depth 1 --evaluate
```

## Limitations

As both SimpleQuestionsWikidata and SimpleDBpediaQA contain only one-hop relations, the model tends to stop at one-hop when you retrieve subgraphs on Wikidata and DBpedia. We will release a updated version of the model that is trained on a more diverse dataset in the future.

## License

MIT