jdopensource
/

JoyAI-LLM-Flash-INT4

Text Generation

joyai_llm_flash

compressed-tensors

Model card Files Files and versions

Mingke977 commited on 2 days ago

Commit

dbad83f

·

verified ·

1 Parent(s): 8edbef2

Update docs/deploy_guidance.md

Files changed (1) hide show

docs/deploy_guidance.md +1 -1

docs/deploy_guidance.md CHANGED Viewed

@@ -15,7 +15,7 @@ docker pull jdopensource/joyai-llm-vllm:v0.15.1-joyai_llm_flash
 ```
 2. launch JoyAI-LLM Flash model with dense MTP.
 ```bash
-vllm serve jdopensource/JoyAI-LLM-Flash-INT4 --tp 1 --trust-remote-code \
    --tool-call-parser qwen3_coder --enable-auto-tool-choice \
    --speculative-config $'{"method": "mtp", "num_speculative_tokens": 3}'
 ```

 ```
 2. launch JoyAI-LLM Flash model with dense MTP.
 ```bash
+vllm serve jdopensource/JoyAI-LLM-Flash-INT4 -tp 1 --trust-remote-code \
    --tool-call-parser qwen3_coder --enable-auto-tool-choice \
    --speculative-config $'{"method": "mtp", "num_speculative_tokens": 3}'
 ```