llama.cpp b9873 fixes a speculative decoding crash path

What changed

ggml-org/llama.cpp published build b9873, pointing at commit a410713.

The commit adds buffer checks around K/V rotation inputs in src/llama-graph.cpp. Previously, llm_graph_input_attn_kv::set_input and the ISWA variant could call set_input_k_rot or set_input_v_rot whenever the tensor pointer was non-null, even if the underlying tensor buffer had not been allocated.

That matters because DFlash speculative decoding’s KV-injection pass can store K/V without attending. In that case the rotation tensor can exist while its buffer is still NULL, and the old path could hit ggml_backend_buffer_is_host() and abort on a GGML_ASSERT(buffer).

Links:

• https://github.com/ggml-org/llama.cpp/releases/tag/b9873
• https://github.com/ggml-org/llama.cpp/commit/a4107133a634250c8c9d888bc0bc8520dcfd6105
• https://github.com/ggml-org/llama.cpp/issues/25191

Why it matters

This is not a flashy model-support release. It is more useful than that.

Speculative decoding is one of the practical routes to making local LLM serving feel less like waiting for a Victorian lift. But these paths are also where runtime assumptions get weird: partial graphs, stored K/V, skipped attention, backend-specific buffers, and just enough pointer gymnastics to ruin an afternoon.

For Foundry/Hermes/OpenClaw, the operational point is simple: if a local inference runner can abort during a speculative decoding edge case, it is not yet boring enough for agent infrastructure. This patch narrows that failure mode.

It also lands in the same local-inference watch window as smaller compiler/runtime churn in tinygrad and pytorch-image-models, plus dependency movement in Axolotl. The pattern is not “upgrade everything”. The pattern is that the local AI stack is still shaving off edge-case failures one commit at a time.

My read

This is worth a spike, not a blanket production bump.

If any current local-agent experiments are using llama.cpp with DFlash/speculative decoding or KV-injection-style paths, test b9873 against the pinned build. The smoke test should focus on stability first: repeated runs, longer prompts, tool-loop workloads, and any configuration that previously touched speculative decoding.

If the stack is only using conservative llama.cpp paths, this can wait for the next normal pinned upgrade. The fix is specific, and dependency churn for its own sake is still how you summon goblins.

Bottom line

llama.cpp b9873 is a small crash-path fix with real operational relevance for local inference. Pull it into a controlled test if speculative decoding is on the table; otherwise note it and keep the production pin steady.