README.md

# SAFE Example Submission with Custom Environment

This is an example of a submission that includes a custom environment.  
So instead of using the default environment of the submission, we first create a custom environment, package it with conda-pack and then push to the model hub in addition to the rest of the files.

Here are the steps:

1. ## Determine the environment that you want and save to the `requirements-custom.txt` file.  

The best way is to start from the base competition image, create a new environment, test it with your model and use `pip freeze` to export it.
To start the container:

```bash
docker run -it --rm --gpus all -v /path/to/local:/app/custom-env huggingface/competitions:latest bash
```

to create an environment:
```bash
conda init
source ~/.bashrc
conda create -n custom python=3.10 -y
conda activate custom
```

Then you can install packages with pip, conda, etc.


2. ## Export the environment with `conda-pack` to a `.tar.gz` file.  
   Run `docker-compose up`. This will pull the base competition image, copy the files into the container and create a new environment save it.
   Change the local place where the volume `/app/custom-env` mounts in the docker-compose file. It will run `make_custom_env.sh` inside the container.

```bash
conda init
source ~/.bashrc
conda create -n custom python=3.10 -y
conda activate custom
pip install -r requirements-custom.txt
export OUTPUT=/app/custom-env/custom.tar.gz
conda deactivate
pip install conda-pack
conda-pack -n custom -o $OUTPUT
```

3. ## Upload the enviroment. Since the environment file is large.
It's better to upload it seperately using fast HF_TRANSFER. Make sure to install `pip install huggingface_hub[hf_transfer]`.

```bash
bash upload_env_file
```

Make sure to change the path to your env file in the script. This is what the script does:

```bash
ENV_FILE=/disk1/kirill/custom-env/custom.tar.gz
HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli upload safe-challenge/safe-example-submission-custom-env $ENV_FILE custom.tar.gz
```


5. ## During runtime, `script.py` unpacks the environment to run your model in your custom environment

Rename your actual script to `_script.py`. `script.py` just calls `bash run.sh` to do all the steps:

```bash
echo "uncompressing environment"
mkdir -p custom
tar -xzf custom.tar.gz -C custom
source custom/bin/activate
echo "unpacking"
conda-unpack
echo "running"
python _script.py
```