--iq2auto

Upload 4 files

IQ2_XS

optimizations for normal llama

Upload 2 files

Requires custom llama to use with this quant method

Fold test

Fold baked into standard Q2K codebook

Fold prototype

Prototype: Produces NAN values, will not work in llama-server, do not download

Fold test

Fix OOM on large tensors

Prototype

Fold interleave

DC cancellation buff

Prototype build

Update README.md

I prefer Q2 honestly.. but leaving IQ1 staging in for posterity

IQ1

Super small quants

Lower RMSE, improve error shaping

Delete embed_hessian.py

Delete dump_tensor.py

Exponential speed-up

Update README.md

15% gain in model output

Another 15% or so

--analog-imatrix: Reduce error in output by 15%

Update README.md

Replace Shor's with accelerated sieve

Update hexstate_quantize.c

Update hexstate_quantize.c

Q8_0 tied embeddings

Q8_0 tied embeddings

Update hexstate_quantize.c

Revert to Alpha 0.1

ALPHA

ALPHA

ALPHA

Delete generate_imatrix.py

Upload 5 files

Upload 3 files

Upload hpc_forward_merged.c

Upload 2 files

Upload 2 files

Qwen changes

Upload hpc_forward_merged.c

Qwen attention tensors

Update README.md

Update README.md