Model quality relative to other quantization techniques?
#1
by
spanspek
- opened
I've heard good things about these quants but can anyone tell me how they perform relative to say unsloth?
This is a big download for the country I'm in so I can't easily afford to just download several large models and test for myself
I'm trying out a small model first but having problems: https://huggingface.co/Intel/Ling-flash-2.0-gguf-q2ks-mixed-AutoRound/discussions/1
If I can get it working then I'll try the GLM AutoRound gguf.