Update the installation of vllm in readme (#13)
Browse files- Update the installation of vllm in readme (cafee21663b48f5b86a765abc98b683302bc5f1f)
Co-authored-by: lmdeploy <unsubscribe@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -205,15 +205,10 @@ for chunk in stream:
|
|
| 205 |
|
| 206 |
#### vLLM inference
|
| 207 |
|
| 208 |
-
|
| 209 |
|
| 210 |
```python
|
| 211 |
-
|
| 212 |
-
# and then follow https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source to install
|
| 213 |
-
cd vllm
|
| 214 |
-
python use_existing_torch.py
|
| 215 |
-
pip install -r requirements-build.txt
|
| 216 |
-
pip install -e . --no-build-isolatio
|
| 217 |
```
|
| 218 |
|
| 219 |
inference code:
|
|
@@ -408,14 +403,10 @@ for chunk in stream:
|
|
| 408 |
|
| 409 |
#### vLLM inference
|
| 410 |
|
| 411 |
-
|
|
|
|
| 412 |
```python
|
| 413 |
-
|
| 414 |
-
# and then follow https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source to install
|
| 415 |
-
cd vllm
|
| 416 |
-
python use_existing_torch.py
|
| 417 |
-
pip install -r requirements-build.txt
|
| 418 |
-
pip install -e . --no-build-isolatio
|
| 419 |
```
|
| 420 |
|
| 421 |
inference code
|
|
@@ -643,15 +634,10 @@ for chunk in stream:
|
|
| 643 |
|
| 644 |
##### vLLM 推理
|
| 645 |
|
| 646 |
-
|
| 647 |
|
| 648 |
-
```
|
| 649 |
-
|
| 650 |
-
# and then follow https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source to install
|
| 651 |
-
cd vllm
|
| 652 |
-
python use_existing_torch.py
|
| 653 |
-
pip install -r requirements-build.txt
|
| 654 |
-
pip install -e . --no-build-isolatio
|
| 655 |
```
|
| 656 |
|
| 657 |
推理代码
|
|
@@ -845,15 +831,10 @@ for chunk in stream:
|
|
| 845 |
|
| 846 |
##### vLLM 推理
|
| 847 |
|
| 848 |
-
|
| 849 |
|
| 850 |
-
```
|
| 851 |
-
|
| 852 |
-
# and then follow https://docs.vllm.ai/en/latest/getting_started/installation/gpu/index.html#build-wheel-from-source to install
|
| 853 |
-
cd vllm
|
| 854 |
-
python use_existing_torch.py
|
| 855 |
-
pip install -r requirements-build.txt
|
| 856 |
-
pip install -e . --no-build-isolatio
|
| 857 |
```
|
| 858 |
|
| 859 |
推理代码
|
|
|
|
| 205 |
|
| 206 |
#### vLLM inference
|
| 207 |
|
| 208 |
+
Refer to [installation](https://docs.vllm.ai/en/latest/getting_started/installation/index.html) to install the latest code of vllm
|
| 209 |
|
| 210 |
```python
|
| 211 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
```
|
| 213 |
|
| 214 |
inference code:
|
|
|
|
| 403 |
|
| 404 |
#### vLLM inference
|
| 405 |
|
| 406 |
+
Refer to [installation](https://docs.vllm.ai/en/latest/getting_started/installation/index.html) to install the latest code of vllm
|
| 407 |
+
|
| 408 |
```python
|
| 409 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 410 |
```
|
| 411 |
|
| 412 |
inference code
|
|
|
|
| 634 |
|
| 635 |
##### vLLM 推理
|
| 636 |
|
| 637 |
+
参考[文档](https://docs.vllm.ai/en/latest/getting_started/installation/index.html) 安装 vllm 最新代码
|
| 638 |
|
| 639 |
+
```bash
|
| 640 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 641 |
```
|
| 642 |
|
| 643 |
推理代码
|
|
|
|
| 831 |
|
| 832 |
##### vLLM 推理
|
| 833 |
|
| 834 |
+
参考[文档](https://docs.vllm.ai/en/latest/getting_started/installation/index.html) 安装 vllm 最新代码
|
| 835 |
|
| 836 |
+
```bash
|
| 837 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 838 |
```
|
| 839 |
|
| 840 |
推理代码
|