ComfyUI custom node
#7
by
drbaph - opened
support sgLang speed?
support sgLang speed?
The node runs Fish Speech locally inside comfyui using PyTorch directly not through a serving framework like SGLang or vLLM.
For speed, it supports flash_attention, sage_attention, and torch.compile as well as some scaled quants. average throughtput is 11 it/s on a 24gb consumer gpu