Spaces:
Runtime error
Runtime error
ZanSara
commited on
Commit
·
f32e4f1
1
Parent(s):
6a65706
nodes
Browse files- pages/1_⭐️_Info.py +19 -8
pages/1_⭐️_Info.py
CHANGED
|
@@ -33,17 +33,28 @@ st.markdown("""
|
|
| 33 |
In the image above you can see how the process looks like.
|
| 34 |
|
| 35 |
First, we download a slice of Wikipedia with information about all the animals in the Lisbon zoo and preprocess,
|
| 36 |
-
index, embed and store them.
|
|
|
|
| 37 |
|
| 38 |
-
At this point they are ready to be queried by the text Retriever,
|
| 39 |
-
|
|
|
|
|
|
|
| 40 |
In this case, it will probably return snippets from the Cheetah Wikipedia entry.
|
| 41 |
|
| 42 |
-
Once the documents are found, they are handed over to the Reader
|
| 43 |
-
|
|
|
|
|
|
|
| 44 |
In this case, the Reader will return answers such as "Cheetah", "the cheetah", etc.
|
| 45 |
|
| 46 |
-
These strings are then ranked and the most likely one is sent over to
|
| 47 |
-
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
""")
|
|
|
|
| 33 |
In the image above you can see how the process looks like.
|
| 34 |
|
| 35 |
First, we download a slice of Wikipedia with information about all the animals in the Lisbon zoo and preprocess,
|
| 36 |
+
index, embed and store them in a DocumentStore. For this demo we're using
|
| 37 |
+
[FAISSDocumentStore](https://docs.haystack.deepset.ai/docs/document_store).
|
| 38 |
|
| 39 |
+
At this point they are ready to be queried by the text Retriever, in this case an instance of
|
| 40 |
+
[EmbeddingRetriever](https://docs.haystack.deepset.ai/docs/retriever#embedding-retrieval-recommended).
|
| 41 |
+
It compares the user's question ("The fastest animal") to all the documents indexed earlier and returns the
|
| 42 |
+
documents which are more likely to contain an answer to the question.
|
| 43 |
In this case, it will probably return snippets from the Cheetah Wikipedia entry.
|
| 44 |
|
| 45 |
+
Once the documents are found, they are handed over to the Reader (in this demo, a
|
| 46 |
+
[FARMReader](https://docs.haystack.deepset.ai/docs/reader) node):
|
| 47 |
+
a model that is able to locate precisely the answer to a question into a document.
|
| 48 |
+
These answers are strings that should be now very easy for CLIP to understand, such as the name of an animal.
|
| 49 |
In this case, the Reader will return answers such as "Cheetah", "the cheetah", etc.
|
| 50 |
|
| 51 |
+
These strings are then ranked and the most likely one is sent over to the
|
| 52 |
+
[MultiModalRetriever](https://docs.haystack.deepset.ai/docs/retriever#multimodal-retrieval)
|
| 53 |
+
that contains CLIP, which will use its own document store of images to find all the pictures that match the string.
|
| 54 |
+
Cheetah are present in the Lisbon zoo, so it will find pictures of them and return them.
|
| 55 |
+
|
| 56 |
+
These nodes are chained together using a Pipeline object, so that all you need to do to run
|
| 57 |
+
a system like this is a single call: `pipeline.run(query="What's the fastest animal?")`
|
| 58 |
+
will return the list of images directly.
|
| 59 |
+
Have a look at [how we implemented it](https://github.com/TuanaCelik/find-the-animal/blob/main/utils/haystack.py)!
|
| 60 |
""")
|