Spaces:
Sleeping
Sleeping
Update LiveRAG_README.md
Browse files- LiveRAG_README.md +17 -17
LiveRAG_README.md
CHANGED
|
@@ -4,11 +4,11 @@
|
|
| 4 |
|
| 5 |
|
| 6 |
|
| 7 |
-
|
| 8 |
|
| 9 |
|
| 10 |
|
| 11 |
-
|
| 12 |
|
| 13 |
1. Accessing the AWS LiveRAG account - follow the instructions you received by email\
|
| 14 |
2. Applying your AWS credits to your team account - follow the instructions you received by email\
|
|
@@ -17,41 +17,41 @@
|
|
| 17 |
IF YOU EXCEED YOUR CREDITS, YOU WILL BE DIRECTED CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS</B>
|
| 18 |
</center>
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
1. Sign up for a [Pinecone](https://pinecone.io ) account, adding a payment method (upgrade to standard). You will not be charged at this time\
|
| 23 |
2. Send mail sigir2025-liverag-tech@tii.ae with your team information and Pinecone account name and we will apply the $750 credits to your account\
|
| 24 |
**Remark:** Do you use Pinecone servcies until you see the allocated credits appear in your account, otherwise you will be charged
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
1. Register to [AI71-Platform](https://platform.ai71.ai/register) and send the email address used for registration to sigir2025-liverag-tech@tii.ae\
|
| 29 |
2. After you're notified that you are approved, login to your [AI71-Platform account](https://platform.ai71.ai)
|
| 30 |
|
| 31 |
|
| 32 |
---
|
| 33 |
-
|
| 34 |
|
| 35 |
-
|
| 36 |
You can use the pre-built indices, [Pinecone](https://www.pinecone.io/) and [OpenSearch](https://opensearch.org/) or build your own indices on top of the same of [FineWeb Sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/viewer/sample-10BT) corpus
|
| 37 |
1. To use the pre-built indices follow this [sample notebook](Operational_Instructions/Indices_Usage_Examples_for_LiveRAG.ipynb)
|
| 38 |
2. To build your own **OpenSearch** index, create or use your existing AWS account and apply the provided AWS credits to save costs (see [here](Operational_Instructions/AWS_Accounts_for_LiveRAG.md))\
|
| 39 |
3. To build your own **Pinecone** index, create or use your existing Pinecone account and apply the provided Pinecone credits to save costs (see [here](Operational_Instructions/Pinecone_for_LiveRAG.md))
|
| 40 |
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
2.
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
**Remark:** Accessing Falcon via the AI71-Platform API is optional and recommended for a quick start. However, for a robust solution, suitable for the live event, it is recommended to set up your own Falcon3-10B-Instruct instance
|
| 48 |
|
| 49 |
-
3.Building, training, and evaluating your RAG system\
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
3.
|
| 53 |
|
| 54 |
-
4. LiveRAG live event details will be added soon...
|
| 55 |
|
| 56 |
You may contact the organizers via:
|
| 57 |
- sigir2025-liverag-gen@tii.ae - for general issues
|
|
|
|
| 4 |
|
| 5 |
|
| 6 |
|
| 7 |
+
# 1. 👣 LiveRAG Challenge Onboarding
|
| 8 |
|
| 9 |
|
| 10 |
|
| 11 |
+
## 1.1 Setting up your AWS accounts - See [AWS Accounts instructions - section 1](Operational_Instructions/AWS_Accounts_for_LiveRAG.md) for the full details
|
| 12 |
|
| 13 |
1. Accessing the AWS LiveRAG account - follow the instructions you received by email\
|
| 14 |
2. Applying your AWS credits to your team account - follow the instructions you received by email\
|
|
|
|
| 17 |
IF YOU EXCEED YOUR CREDITS, YOU WILL BE DIRECTED CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS</B>
|
| 18 |
</center>
|
| 19 |
|
| 20 |
+
## 1.2 Setting your Pinecone account if you want to build your own dense index
|
| 21 |
|
| 22 |
1. Sign up for a [Pinecone](https://pinecone.io ) account, adding a payment method (upgrade to standard). You will not be charged at this time\
|
| 23 |
2. Send mail sigir2025-liverag-tech@tii.ae with your team information and Pinecone account name and we will apply the $750 credits to your account\
|
| 24 |
**Remark:** Do you use Pinecone servcies until you see the allocated credits appear in your account, otherwise you will be charged
|
| 25 |
|
| 26 |
+
## 1.3. Register to AI71-Platform to access DataMorgana and Falcon3-10B-Instruct LLM
|
| 27 |
|
| 28 |
1. Register to [AI71-Platform](https://platform.ai71.ai/register) and send the email address used for registration to sigir2025-liverag-tech@tii.ae\
|
| 29 |
2. After you're notified that you are approved, login to your [AI71-Platform account](https://platform.ai71.ai)
|
| 30 |
|
| 31 |
|
| 32 |
---
|
| 33 |
+
# 2. 👣 LiveRAG Challenge Step-by-Step Guide
|
| 34 |
|
| 35 |
+
## 2.1 Accessing or building your RAG indices\
|
| 36 |
You can use the pre-built indices, [Pinecone](https://www.pinecone.io/) and [OpenSearch](https://opensearch.org/) or build your own indices on top of the same of [FineWeb Sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/viewer/sample-10BT) corpus
|
| 37 |
1. To use the pre-built indices follow this [sample notebook](Operational_Instructions/Indices_Usage_Examples_for_LiveRAG.ipynb)
|
| 38 |
2. To build your own **OpenSearch** index, create or use your existing AWS account and apply the provided AWS credits to save costs (see [here](Operational_Instructions/AWS_Accounts_for_LiveRAG.md))\
|
| 39 |
3. To build your own **Pinecone** index, create or use your existing Pinecone account and apply the provided Pinecone credits to save costs (see [here](Operational_Instructions/Pinecone_for_LiveRAG.md))
|
| 40 |
|
| 41 |
+
## 2.2 Accessing DataMorgana and Falcon3-10B-Instruct via AI71-Platform
|
| 42 |
+
1. Login to your [AI71-Platform account](https://platform.ai71.ai) after you followed the onboarding instructions
|
| 43 |
+
2. Access the DataMorgana [Sandbox](https://platform.ai71.ai/playground) and [try it](Operational_Instructions/DM_Sandbox.md) to get your first Q&A\
|
| 44 |
+
3. Create your AI71-Platform [API key](https://platform.ai71.ai/api-keys)
|
| 45 |
+
3.1 Use your AI71-Platform key for programatic access to DataMorgana bulk requests generation via its [API](Operational_Instructions/DM_API_usage_example.ipynb)\
|
| 46 |
+
3.2 Use your AI71-Platform key for programatic access to Falcon3-10B-Instruct (see [AI71 documentation](https://platform.ai71.ai/documentation))\
|
| 47 |
**Remark:** Accessing Falcon via the AI71-Platform API is optional and recommended for a quick start. However, for a robust solution, suitable for the live event, it is recommended to set up your own Falcon3-10B-Instruct instance
|
| 48 |
|
| 49 |
+
## 2.3. Building, training, and evaluating your RAG system\
|
| 50 |
+
1. Build your RAG system in your AWS Team account leveraging your FineWeb indices - make sure to use Falcon3-10B-Instruct to generate answers
|
| 51 |
+
2. Train your RAG system using synthetic Q&A pairs generated by DataMorgana [API](Operational_Instructions/DM_API_usage_example.ipynb)
|
| 52 |
+
3. Evaluate your RAG system using the [evaluation guidelines](Operational_Instructions/Evaluation_Guidelines_for_LiveRAG.md)
|
| 53 |
|
| 54 |
+
## 4. LiveRAG live event details will be added soon...
|
| 55 |
|
| 56 |
You may contact the organizers via:
|
| 57 |
- sigir2025-liverag-gen@tii.ae - for general issues
|