Voodoo Dynamic Quant is now MIT-licensed. Two months after announci... - Vibeus
Voodoo Dynamic Quant is now MIT-licensed. Two months after announcing Voodoo Quant, 1ncehost has released the toolset so the community can make its own dynamic quants. The method uses gradient descent to optimize the per-tensor quantization layout instead of relying on hand-tuned recipes. Voodoo runs all candidate quantization levels for every tensor at the same time and trains a scalar gate for each tensor/level pair. Annealing the tau level helps the training settle on predominant selections, while softmax keeps every quantization level in the gradient. The selections are trained on a diverse calibration dataset; the loss compares mixed-quant logits with a reference BF16 checkpoint through KL divergence and also rewards meeting a target file size. The toolset is currently set up for Qwen and can be adapted to other model architectures. In the author’s testing, Unsloth Dynamic 3.0 was better at high-to-mid quantization levels, while Voodoo Quant was better at aggressive levels. The repository is now available under the MIT license. Источник