Commit 3ce6dd3

mo khan <mo@mokhan.ca>
2026-08-28 03:28:47
fix(gguf): load the PEG parser arena so strict tool-calls parse
Mainline llama.cpp moved common_chat to a PEG-based parser whose rules are serialized into cparams.parser by templates_apply; common_chat_parse forwards params.parser (a common_peg_arena) to the engine. The shim only set pp.format and left the arena empty, so common_chat_parse returned content-only for every model -- the strict path was structurally dead and every working tool call came solely from the lenient JSON fallback. Deserialize the arena via the converting ctor + pp.parser.load(). Verified on GPU: Qwen3-Coder-30B now parses its native <function=..> XML strictly (fallback never fires); Qwen2.5-Coder-7B still relies on the fallback (bare JSON its template's grammar misses), so the fallback stays. Suite 316/0. Claude-Session: https://claude.ai/code/session_01UDKgb5gaG9Xmn3DViHRnJ7
gguf-provider
1 parent d92f3f0
Changed files (1)
ext
elelem_llama
elelem_llama.cpp
ext/elelem_llama/elelem_llama.cpp
@@ -138,8 +138,12 @@ const char *el_generate(void *handle, const char *messages_json, const char *too
     llama_sampler_free(smpl);
     llama_free(ctx);
 
-    common_chat_parser_params pp;
-    pp.format = cparams.format;
+    // The parse rules live in a PEG arena that templates_apply serialized into
+    // cparams.parser; common_chat_parse forwards params.parser to the PEG engine,
+    // so it must be deserialized here or every format returns content-only (no
+    // tool calls). The converting ctor carries format + generation_prompt.
+    common_chat_parser_params pp(cparams);
+    if (!cparams.parser.empty()) pp.parser.load(cparams.parser);
     common_chat_msg parsed = common_chat_parse(output, false, pp);
 
     // Lenient fallback: small/quantized models often emit a bare {"name","arguments"}