custom_nodes/ComfyUI-llama-cpp_vllm/README.md

ComfyUI-llama-cpp

Run LLM/VLM models natively in ComfyUI based on llama.cpp
[📃中文版]

Preview

Installation

Install the node:

cd ComfyUI/custom_nodes
git clone https://github.com/lihaoyun6/ComfyUI-llama-cpp.git
python -m pip install -r ComfyUI-llama-cpp/requirements.txt

Download models:

• Place your model files in the ComfyUI/models/LLM folder.

  If you need a VLM model to process image input, don't forget to download the mmproj weights.

Credits

• llama-cpp-python @JamePeng
• ComfyUI-llama-cpp @kijai
• ComfyUI @comfyanonymous