Firworks commited on
Commit
6c211a2
·
verified ·
1 Parent(s): d4b86da

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -0
README.md CHANGED
@@ -15,6 +15,9 @@ base_model:
15
  Check the original model card for information about this model.
16
 
17
  # Running the model with VLLM in Docker
 
 
 
18
  ```sh
19
  sudo docker run --runtime nvidia --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:nightly --model Firworks/rnj-1-instruct-nvfp4 --dtype auto --max-model-len 32768
20
  ```
 
15
  Check the original model card for information about this model.
16
 
17
  # Running the model with VLLM in Docker
18
+
19
+ Note: I couldn't get this one to run in VLLM. I'm not sure if there's a trick to run Gemma 3 based models in VLLM. If anyone knows a trick I can update the model card with the updated command.
20
+
21
  ```sh
22
  sudo docker run --runtime nvidia --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:nightly --model Firworks/rnj-1-instruct-nvfp4 --dtype auto --max-model-len 32768
23
  ```