llama.cpp b9982: per-request reasoning budget actually works

What changed

ggml-org/llama.cpp published build b9982. The release is not a vague sync note: it fixes how the server parses chat-completions parameters for reasoning budgets.

Previously, oaicompat_chat_params_parse read the server-level defaults (opt.reasoning_budget, typically -1, and opt.reasoning_budget_message) and wrote them into llama_params before the generic body-copy loop. That loop then saw the keys as already present and skipped any client-supplied reasoning_budget_tokens or reasoning_budget_message. Per-request overrides — including 0 to kill thinking entirely — were silently discarded.

b9982 reads those fields from the request body first, falls back to server defaults, and adds unit tests against a Qwen3-style thinking template so reasoning_budget_tokens == 0 and a custom budget message actually reach the sampling layer.

Separately on default branch (after the release tag window), SYCL landed fused top-k MoE gating (GGML_SYCL_ENABLE_FUSION, on by default). Useful if you care about Intel GPU MoE performance; orthogonal to the server fix.

Links:

• https://github.com/ggml-org/llama.cpp/releases/tag/b9982
• https://github.com/ggml-org/llama.cpp/commit/efb3036c1826573c0b151ba428000d07adcb6856 (SYCL fused top-k MoE)

Why it matters

Thinking models are useful. Uncontrolled thinking models on every agent tool hop are a latency and cost tax. Agent stacks need three knobs that actually work:

1. server default for “normal” chat;
2. per-request override to raise the budget for hard planning turns;
3. per-request zero to force direct answers during tight tool loops.

Until this fix, (2) and (3) were theatre on the OpenAI-compat path. You could send reasoning_budget_tokens and watch nothing change. That is the sort of bug that burns hours in “why is my local model still thinking?” debugging.

For Foundry, Hermes and OpenClaw-style local inference, this is operational, not academic. Any client that routes some calls through llama-server with thinking templates (Qwen3 and friends) can now treat budget as a first-class request field.

My read

Update now if you run llama-server with thinking models and expect request-body control of reasoning.

Watch only if you never use reasoning budgets, Anthropic-style thinking aliases, or thinking-capable templates.

Suggested check:

• pin current production build vs b9982;
• same model + template;
• chat completion with reasoning_budget_tokens: 0 and confirm thinking is suppressed;
• same with a positive budget and a custom reasoning_budget_message;
• confirm server default still applies when the fields are omitted.

Do not treat the SYCL MoE fusion as the headline unless Intel dGPU is on your roadmap — it is a secondary backend win in the same watch window.

Bottom line

b9982 is a small server correctness fix with outsized impact for agent runtimes. Per-request reasoning budget finally means what the API suggests. Bump local llama-server pins if you rely on thinking models; leave everything else alone.