Commit d92f3f0

mo khan <mo@mokhan.ca>
2026-08-28 00:35:33
feat(gguf): expose whether the lenient tool-call fallback fired
The shim's result JSON now carries "fallback": bool, recording whether the strict common_chat parser missed the tool call and the bare-JSON fallback had to recover it. A diagnostic for judging per model whether the fallback crutch is still needed. Ruby ignores the extra field. Claude-Session: https://claude.ai/code/session_01UDKgb5gaG9Xmn3DViHRnJ7
gguf-provider
1 parent 63d20bb
Changed files (1)
ext
elelem_llama
elelem_llama.cpp
ext/elelem_llama/elelem_llama.cpp
@@ -145,7 +145,9 @@ const char *el_generate(void *handle, const char *messages_json, const char *too
     // Lenient fallback: small/quantized models often emit a bare {"name","arguments"}
     // tool-call JSON (frequently fenced) instead of the template's exact tag syntax,
     // so the strict parser misses it. If tools were offered and nothing parsed, pull
-    // out the first JSON object that names a real tool.
+    // out the first JSON object that names a real tool. result["fallback"] records
+    // whether this fired -- a diagnostic for judging if a model needs the crutch.
+    bool fallback_used = false;
     if (parsed.tool_calls.empty() && !inputs.tools.empty()) {
         size_t a = output.find('{'), b = output.rfind('}');
         if (a != std::string::npos && b != std::string::npos && b > a) {
@@ -160,11 +162,13 @@ const char *el_generate(void *handle, const char *messages_json, const char *too
                     tc.arguments = as_json_string(j["arguments"]);
                     parsed.tool_calls.push_back(tc);
                     parsed.content.clear();
+                    fallback_used = true;
                 }
             } catch (...) { /* not a tool call; leave content as-is */ }
         }
     }
 
+    result["fallback"] = fallback_used;
     result["content"] = parsed.content;
     int i = 0;
     for (const auto &tc : parsed.tool_calls) {