Gemma Hot/Cold Engine makes Gemma 4 26B run faster on a consumer GPU with less vram. It’s a patch on llama.cpp that keeps the model’s most-used parts on the GPU, runs the rest on the CPU at the same time, and learns what your workload needs as it runs.

With multi-token prediction on both sides, it ran about 70% faster than stock llama.cpp at 5.9 GB of VRAM. It used the same model file, and math accuracy matched on our test.