README.md

## Prerequisites

* Tested on Linux (Ubuntu 22.04) and macOS (Apple Silicon).
* Python 3.10


### Before
You should install requirements.txt, and unzip `documents/falcon-refinedweb.zip` into the `documents/falcon-refinedweb` folder.

### Create document embeddings

```bash
python -m app.main index -c config/config.js
```


### Predict on the dataset
```bash
python -m app.main predict -c config/config.js -m config/openai.js
```


### Evaluate the dataset
```bash
python -m app.main evaluate -c config/config.js -m config/openai.js
```