Commit 4a4516c

mo khan <mo@mokhan.ca>
2026-09-23 07:07:55
fix: detect integrated GPU devices when querying free VRAM
ggml classifies UMA/integrated Vulkan devices (Strix Halo, Apple Silicon, etc.) as GGML_BACKEND_DEVICE_TYPE_IGPU, a distinct enum value from GGML_BACKEND_DEVICE_TYPE_GPU. el_gpu_free_memory only queried the GPU type, so gpu_free_memory always returned nil on integrated GPUs and the auto-offload heuristic silently fell back to n_gpu_layers=0.
main
1 parent 578881e
Changed files (1)
ext
elelem
llama
elelem.cpp
ext/elelem/llama/elelem.cpp
@@ -134,6 +134,7 @@ extern "C" {
   long el_gpu_free_memory() {
     el_ensure_backend();
     ggml_backend_dev_t dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_GPU);
+    if (!dev) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_IGPU);
     if (!dev) return -1;
     size_t free_mem = 0, total_mem = 0;
     ggml_backend_dev_memory(dev, &free_mem, &total_mem);