Update README.md
Browse files
README.md
CHANGED
|
@@ -67,7 +67,7 @@ If you need the detailed inference code and evaluation script, please refer to o
|
|
| 67 |
|
| 68 |
|
| 69 |
### Run the Inference Script
|
| 70 |
-
|
| 71 |
|
| 72 |
```python
|
| 73 |
import torch
|
|
@@ -113,7 +113,7 @@ if __name__ == '__main__':
|
|
| 113 |
|
| 114 |
```
|
| 115 |
|
| 116 |
-
|
| 117 |
```shell
|
| 118 |
python -m vllm.entrypoints.openai.api_server --model $MODEL_PATH --max_model_len 32768 --limit-mm-per-prompt "image=20" --tensor-parallel-size $N_GPU --dtype auto --trust-remote-code
|
| 119 |
|
|
|
|
| 67 |
|
| 68 |
|
| 69 |
### Run the Inference Script
|
| 70 |
+
transformers inference
|
| 71 |
|
| 72 |
```python
|
| 73 |
import torch
|
|
|
|
| 113 |
|
| 114 |
```
|
| 115 |
|
| 116 |
+
vllm inference
|
| 117 |
```shell
|
| 118 |
python -m vllm.entrypoints.openai.api_server --model $MODEL_PATH --max_model_len 32768 --limit-mm-per-prompt "image=20" --tensor-parallel-size $N_GPU --dtype auto --trust-remote-code
|
| 119 |
|