Speculative decoding?

#5
by karypid - opened

Since S.1-NVFP4 has issues, I was going to try this one in a dual spark setup. I notice though that there is no speculative decoding capability? Are there any plans to publish so we can try the M.1 version at reasonable speed?

Sign up or log in to comment