llama.cpp support?
#2
by sivarajan - opened
Is this model's architecture already supported by llama.cpp? Also, what are the vram requirements to run this model locally?
1: I believe so, it is simply safetensors. 2: a couple terabytes nothing much.
I'll do one better! Here is a implementation of this model that could be easily ported to run on the AGC (Apollo Guidance Computer) with just 4 kilobytes of magnetic core memory and likely the ENIAC as well, by using advanced weight de-duplication arithmetic technology and quantized to whatever the hell the compiler or architecture chooses as the word size for result it can even run on your brain
int main() {
int result = 0 * 0;
return 0;
}