VibeusMost engines right now are either heavily optimized for standard... - Vibeus
Most engines right now are either heavily optimized for standard GGUF (llama.cpp) or specifically built for 1.58-bit ternary models (bitnet.cpp). I wanted a single binary that could handle both without pulling in Python or heavy dependencies, so I basically wrote Project Zero from scratch in pure C99.
It is just GCC and make. On a Xeon it hits 36 tok/s on BitNet b1.58-2B-4T (about 1.8x faster than bitnet.cpp). For dense models, it runs SmolLM2 F16 at ~100 tok/s on an i5-11300H, though llama.cpp still beats it by about 7% on that specific hardware path.
The biggest pain point was getting the AVX-512 kernels working for the ternary packing without destroying the cache. The Q4_K path is still pretty slow compared to llama.cpp (1.9 tok/s vs 13.7 tok/s for DeepSeek), but the ternary and dense F16 paths are solid. That was kind of a surprise.
There is a pre-built x86 Linux binary in the releases if you want to test it without compiling. The API is OpenAI-compatible out of the box (/v1/chat/completions) with SSE streaming.
Curious if anyone else is running BitNet models locally yet, or if the lack of UI integrations is keeping them mostly as a novelty for now.
