llama.cpp expands quantised MoE prefill on Adreno GPUs

What changed

ggml-org/llama.cpp merged a substantial OpenCL update for Qualcomm Adreno GPUs in commit 1d1d9a9.

The change adds int8 dp4a paths for dense matrix work and mixture-of-experts prefill. It introduces or wires up Adreno-oriented kernels for several quantisation formats, including Q4_0, Q4_K, Q5_0, Q5_K, Q6_K, Q8_0, IQ4_NL and MXFP4.

This is not a cosmetic patch: the main OpenCL implementation changed by more than 1,800 lines, alongside new conversion and MoE kernels. The implementation also handles ragged expert batches so padded token slots do not receive the full compute treatment.

A fresh llama.cpp build, b9959, was published during the same watch window. Its terse release note only says sync : ggml, so the Adreno commit should be treated as a default-branch change until the exact build boundary has been verified.

Why it matters

Mixture-of-experts models save active compute by routing tokens through only part of the model, but that does not automatically make them fast on every local backend. Prefill can still expose weak matrix kernels, quantisation gaps and expensive handling of uneven expert batches.

This patch attacks that bottleneck on Adreno directly. If the kernels hold up in real workloads, Snapdragon-class machines get a better chance of running quantised MoE models without falling back to the CPU for the expensive bits.

For Foundry, Hermes or OpenClaw deployments, the strategic value is hardware choice. CUDA remains the easy answer for serious local inference, but it is not always the practical answer for portable or client-side agent systems. Better Adreno support widens the pool of machines that might run private inference near the work rather than sending every token to a cloud model.

My read

Worth a spike, specifically on supported Snapdragon/Adreno hardware. It is not an update-now item for Apple Silicon, NVIDIA or CPU-only deployments.

This also follows llama.cpp’s recent work on precompiled Adreno OpenCL kernels. The direction is now clearer than a one-off optimisation: Qualcomm GPUs are becoming an intentional local-inference target.

The useful test is controlled rather than theatrical:

• pin the previous known-good llama.cpp build and a build containing 1d1d9a9;
• run the same quantised dense and MoE models on the same Adreno device;
• measure prompt-processing speed, first-token latency, generation speed, memory use and thermals;
• repeat agent-style workloads with long tool transcripts, not just a short benchmark prompt;
• check output parity and stability before celebrating a faster graph.

Until those numbers exist, this is promising infrastructure work, not proof that a Snapdragon laptop has become a local inference server.

Bottom line

llama.cpp has materially widened its Adreno acceleration path, especially for quantised MoE prefill. Spike it if Snapdragon edge inference is on the roadmap; otherwise watch the backend mature and leave stable production pins alone.