VRAM Requirements for 128K Context Length with vLLM
#5
by
codemonkeyShin
- opened
I'm planning to use this model with vLLM.
How much VRAM would be required to utilize the 128K context?
I'm curious if 280GB would be sufficient.