Update README.md
Browse files
README.md
CHANGED
|
@@ -5,10 +5,10 @@ So instead of using the default environment of the submission, we first create a
|
|
| 5 |
|
| 6 |
Here are the steps:
|
| 7 |
|
| 8 |
-
1. Determine the environment that you want and save to the `requirements-custom.txt` file.
|
| 9 |
-
- The best way is to start from the base competition image, create a new environment, test it with your model and use `pip freeze` to export it.
|
| 10 |
|
| 11 |
-
to start the
|
|
|
|
| 12 |
|
| 13 |
```bash
|
| 14 |
docker run -it --rm --gpus all -v /path/to/local:/app/custom-env huggingface/competitions:latest bash
|
|
@@ -25,9 +25,9 @@ conda activate custom
|
|
| 25 |
Then you can install packages with pip, conda, etc.
|
| 26 |
|
| 27 |
|
| 28 |
-
2. Export the environment with `conda-pack` to a `.tar.gz` file.
|
| 29 |
Run `docker-compose up`. This will pull the base competition image, copy the files into the container and create a new environment save it.
|
| 30 |
-
Change the local place where the volume `/app/custom-env` mounts in the docker-compose file. It will run `make_custom_env.sh` inside the container
|
| 31 |
|
| 32 |
```bash
|
| 33 |
conda init
|
|
@@ -41,8 +41,8 @@ pip install conda-pack
|
|
| 41 |
conda-pack -n custom -o $OUTPUT
|
| 42 |
```
|
| 43 |
|
| 44 |
-
3. Upload the enviroment. Since the environment file is large.
|
| 45 |
-
|
| 46 |
|
| 47 |
```bash
|
| 48 |
bash upload_env_file
|
|
@@ -56,8 +56,9 @@ HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli upload safe-challenge/safe-example-s
|
|
| 56 |
```
|
| 57 |
|
| 58 |
|
| 59 |
-
5. During runtime, `script.py` unpacks the environment to run your model in your custom environment
|
| 60 |
-
|
|
|
|
| 61 |
|
| 62 |
```bash
|
| 63 |
echo "uncompressing environment"
|
|
|
|
| 5 |
|
| 6 |
Here are the steps:
|
| 7 |
|
| 8 |
+
1. ## Determine the environment that you want and save to the `requirements-custom.txt` file.
|
|
|
|
| 9 |
|
| 10 |
+
The best way is to start from the base competition image, create a new environment, test it with your model and use `pip freeze` to export it.
|
| 11 |
+
To start the container:
|
| 12 |
|
| 13 |
```bash
|
| 14 |
docker run -it --rm --gpus all -v /path/to/local:/app/custom-env huggingface/competitions:latest bash
|
|
|
|
| 25 |
Then you can install packages with pip, conda, etc.
|
| 26 |
|
| 27 |
|
| 28 |
+
2. ## Export the environment with `conda-pack` to a `.tar.gz` file.
|
| 29 |
Run `docker-compose up`. This will pull the base competition image, copy the files into the container and create a new environment save it.
|
| 30 |
+
Change the local place where the volume `/app/custom-env` mounts in the docker-compose file. It will run `make_custom_env.sh` inside the container.
|
| 31 |
|
| 32 |
```bash
|
| 33 |
conda init
|
|
|
|
| 41 |
conda-pack -n custom -o $OUTPUT
|
| 42 |
```
|
| 43 |
|
| 44 |
+
3. ## Upload the enviroment. Since the environment file is large.
|
| 45 |
+
It's better to upload it seperately using fast HF_TRANSFER. Make sure to install `pip install huggingface_hub[hf_transfer]`.
|
| 46 |
|
| 47 |
```bash
|
| 48 |
bash upload_env_file
|
|
|
|
| 56 |
```
|
| 57 |
|
| 58 |
|
| 59 |
+
5. ## During runtime, `script.py` unpacks the environment to run your model in your custom environment
|
| 60 |
+
|
| 61 |
+
Rename your actual script to `_script.py`. `script.py` just calls `bash run.sh` to do all the steps:
|
| 62 |
|
| 63 |
```bash
|
| 64 |
echo "uncompressing environment"
|