Create README.md

README.md

@@ -0,0 +1,58 @@
+---
+language:
+- en
+tags:
+- retrieval
+- document_expansion
+datasets:
+- irds:msmarco-passage
+library_name: pyterrier
+---
+
+A Doc2Query model based on `t5-base` and trained on MS MARCO. This is a version of [the checkpoint released by the original authors](https://git.uwaterloo.ca/jimmylin/doc2query-data/raw/master/T5-passage/t5-base.zip), converted to pytorch format and ready for use in [`pyterrier_doc2query`](https://github.com/terrierteam/pyterrier_doc2query).
+
+**Creating a transformer:**
+
+```python
+import pyterrier as pt
+pt.init()
+from pyterrier_doc2query import Doc2Query
+doc2query = Doc2Query('macavaney/doc2query-t5-base-msmarco')
+```
+
+**Transforming documents**
+
+```python
+import pandas as pd
+doc2query(pd.DataFrame([
+  {'docno': '0', 'text': 'Hello Terrier!'},
+  {'docno': '1', 'text': 'Doc2Query expands queries with potentially relevant queries.'},
+]))
+# docno                                               text                                           querygen
+#    0                                     Hello Terrier!  hello terrier what kind of dog is a terrier wh...
+#    1  Doc2Query expands queries with potentially rel...  can dodoc2query extend query query? what is do...
+```
+
+**Indexing transformed documents**
+
+```python
+doc2query.append = True # append querygen to text
+indexer = pt.IterDictIndexer('./my_index', fields=['text'])
+pipeline = doc2query >> indexer
+pipeline.index([
+  {'docno': '0', 'text': 'Hello Terrier!'},
+  {'docno': '1', 'text': 'Doc2Query expands queries with potentially relevant queries.'},
+])
+```
+
+**Expanding and indexing a dataset**
+
+```python
+dataset = pt.get_dataset('irds:vaswani')
+pipeline.index(dataset.get_corpus_iter())
+```
+
+## References
+
+  - [Nogueira20]: Rodrigo Nogueira and Jimmy Lin. From doc2query to docTTTTTquery. https://cs.uwaterloo.ca/~jimmylin/publications/Nogueira_Lin_2019_docTTTTTquery-v2.pdf
+  - [Macdonald20]: Craig Macdonald, Nicola Tonellotto. Declarative Experimentation inInformation Retrieval using PyTerrier. Craig Macdonald and Nicola Tonellotto. In Proceedings of ICTIR 2020. https://arxiv.org/abs/2007.14271