Run with full 128k context in 24G vram
#3
by meigami - opened
its a amazing model but need too much vram, I cannot run with full 128k context in 24G vram
Do you have any plans to add lighter versions? @TheBloke @LoneStriker
its a amazing model but need too much vram, I cannot run with full 128k context in 24G vram
Do you have any plans to add lighter versions? @TheBloke @LoneStriker