poolside-laguna-hackathon
/

kv-landlords

Model card Files Files and versions

kv-landlords / vllm /docs /cli /run-batch.md

a1exxd0's picture

Upload kv-quant (INT4/NVFP4 KIVI) work + vLLM fork source

6e668dc verified about 2 months ago

|

History Blame Contribute Delete

140 Bytes

vllm run-batch

JSON CLI Arguments

--8<-- "docs/cli/json_tip.inc.md"

Arguments

--8<-- "docs/generated/argparse/run-batch.inc.md"