Native Swift implementation & 4bit/8bit quants
#2
by smcleod - opened
Thanks for sharing this fantastic model, it's really impressive.
In the event these are useful to anyone else I've quantised it to dynamic 4bit and 8bit weights, and created a native Swift / MLX port:
smcleod changed discussion title from Native swift implementation to Native swift implementation & 4bit/8bit quants
smcleod changed discussion title from Native swift implementation & 4bit/8bit quants to Native Swift implementation & 4bit/8bit quants