Spaces:
Sleeping
Sleeping
Update Operational_Instructions/AWS_Accounts_for_LiveRAG.md
Browse files
Operational_Instructions/AWS_Accounts_for_LiveRAG.md
CHANGED
|
@@ -23,12 +23,11 @@ To ensure your credits are properly applied, follow the official AWS instruction
|
|
| 23 |
|
| 24 |
## 1.2 Access and Setup Instructions
|
| 25 |
|
| 26 |
-
To gain access and configure the AWS LiveRAG account, follow these steps
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
4. **Configure AWS CLI Profile** – Run the following command to set up the CLI for this account and follow the instructions:
|
| 32 |
|
| 33 |
```sh
|
| 34 |
aws configure --profile sigir-participant
|
|
@@ -47,20 +46,17 @@ For detailed instructions on account setup, credentials, and index usage, refer
|
|
| 47 |
- `personal`: Your own AWS account.
|
| 48 |
- `sigir-participant`: The TII-provided AWS account (described below).
|
| 49 |
|
| 50 |
-
For beginners, it is recommended to complete introductory AWS tutorials
|
| 51 |
|
| 52 |
|
| 53 |
# 2. Using the Pre-Built Indices and Building Your Own Indices
|
| 54 |
We provide you with two basic pre-built dense (Pinecone) and sparse (Opensearch) indices. See usage [here](Indices_Usage_Examples_for_LiveRAG.ipynb). You may use them freely for LiveRAG Challenge tasks only.
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
For instructions pertaining to building your own Pinecone index see [here](Pinecone_for_LiveRAG.md).
|
| 59 |
|
| 60 |
# 3. Cost Optimization Recommendations
|
| 61 |
|
| 62 |
-
We have estimated the cost of a typical team’s infrastructure, including GPU usage, and AWS has provided credits accordingly.
|
| 63 |
-
However, individual teams may have varying requirements, so cost management is essential:
|
| 64 |
- **Shut down unused resources** – Always shut down GPUs and other compute instances when not in use.
|
| 65 |
- **Experiment on smaller datasets first** – This approach speeds up iteration cycles and reduces expenses before scaling up to larger datasets.
|
| 66 |
|
|
@@ -72,4 +68,4 @@ However, individual teams may have varying requirements, so cost management is e
|
|
| 72 |
| **Experimentation** | Start with small datasets before scaling up. |
|
| 73 |
| **Account Management** | Use separate AWS CLI profiles for personal and TII-Provided accounts. |
|
| 74 |
|
| 75 |
-
By following these best practices, you can effectively manage your resources, optimize costs, and maximize your efficiency during the LiveRAG Challenge.
|
|
|
|
| 23 |
|
| 24 |
## 1.2 Access and Setup Instructions
|
| 25 |
|
| 26 |
+
To gain access and configure the AWS LiveRAG account, follow these steps:\
|
| 27 |
+
1. **Receive Sign-In Details** – You should have received an email with login credentials from the organizers.\
|
| 28 |
+
2. **Sign In & Change Password** – Log in to the AWS console and update your password.\
|
| 29 |
+
3. **Create AWS Command Line Interface (CLI) Credentials** – Generate an access key and secret for CLI usage.\
|
| 30 |
+
4. **Configure AWS CLI Profile** – Run the following command to set up the CLI for this account and follow the instructions:\
|
|
|
|
| 31 |
|
| 32 |
```sh
|
| 33 |
aws configure --profile sigir-participant
|
|
|
|
| 46 |
- `personal`: Your own AWS account.
|
| 47 |
- `sigir-participant`: The TII-provided AWS account (described below).
|
| 48 |
|
| 49 |
+
For beginners, it is recommended to complete the introductory [AWS tutorials](https://aws.amazon.com/getting-started/).
|
| 50 |
|
| 51 |
|
| 52 |
# 2. Using the Pre-Built Indices and Building Your Own Indices
|
| 53 |
We provide you with two basic pre-built dense (Pinecone) and sparse (Opensearch) indices. See usage [here](Indices_Usage_Examples_for_LiveRAG.ipynb). You may use them freely for LiveRAG Challenge tasks only.
|
| 54 |
+
You're welcome to build your own Pinecone/OpenSearch, or other indices. We encourage you to take advantage of the AWS and Pinecone credits we provide for this purpose. OpenSearch can be provisioned and paid by AWS credits.\
|
| 55 |
+
If you want to build your own Pinecone index follow these [instructions](Pinecone_for_LiveRAG.md).
|
|
|
|
|
|
|
| 56 |
|
| 57 |
# 3. Cost Optimization Recommendations
|
| 58 |
|
| 59 |
+
We have estimated the cost of a typical team’s infrastructure, including GPU usage, and AWS has provided credits accordingly. However, individual teams may have varying requirements, so cost management is essential:\
|
|
|
|
| 60 |
- **Shut down unused resources** – Always shut down GPUs and other compute instances when not in use.
|
| 61 |
- **Experiment on smaller datasets first** – This approach speeds up iteration cycles and reduces expenses before scaling up to larger datasets.
|
| 62 |
|
|
|
|
| 68 |
| **Experimentation** | Start with small datasets before scaling up. |
|
| 69 |
| **Account Management** | Use separate AWS CLI profiles for personal and TII-Provided accounts. |
|
| 70 |
|
| 71 |
+
By following these best practices, you can effectively manage your resources, optimize costs, and maximize your efficiency during the LiveRAG Challenge. Recall that **if you exceed your credits , you will be directly charged and not be refunded!**
|