I ran local llms on intel's cheapest igpu, and the results were surprisingly decent

At a glance:

• Intel N100‑based LattePanda Mu can run 4‑7B parameter models such as Gemma 3‑4B, Qwen3‑4B and DeepSeek R1‑7B at ~2.9 tokens/s using llama.cpp and Vulkan
• Passing the integrated UHD Graphics to an LXC container only requires binding /dev/dri/renderD128 with mode 0666
• After allocating 7 GB RAM and a 3 GB swap file, compilation succeeds and inference is noticeably faster than on a Raspberry Pi

Why the experiment matters

Running large language models locally has traditionally required a discrete GPU with tensor cores and ample VRAM. The author wanted to test whether Intel’s ultra‑low‑cost N100 processor, which ships in the LattePanda Mu compute module, could serve as a viable edge inference platform. By using an LXC container on a Proxmox host, the iGPU was exposed to the container, allowing the open‑source llama.cpp engine to leverage Vulkan‑accelerated inference. This approach sidesteps the heavy overhead of user‑friendly stacks like Ollama, which the author found unsuitable for such constrained hardware.

Build and configuration details

The hardware stack consists of a LattePanda Mu with an Intel N100 (upgradable to i3‑N305), 8 GB LPDDR5 RAM (expanded to 7 GB for the container), 64 GB eMMC storage, and Intel UHD Graphics. The container was created in Proxmox, then the iGPU was passed through by adding /dev/dri/renderD128 with access mode 0666 in the LXC resources. Required packages were installed via:

apt update && apt install -y intel-media-va-driver vainfo git cmake curl glslc glslang-tools libvulkan1 vulkan-tools libvulkan-dev spirv-tools spirv-headers build-essential

The llama.cpp source was cloned from GitHub, built with Vulkan support (cmake -B build -DGGML_VULKAN=ON), and compiled using a single thread (cmake --build build -- -j1). Initial compilation failures at 18 % were traced to insufficient RAM; increasing the container’s memory allocation and adding a 3 GB swap file resolved the issue.

Model performance on the N100

Using the compiled llama.cpp binary, the author launched a server with the Gemma 3‑4B model (gemma-3-4b-it-Q4_K_M.gguf). Compared with a Raspberry Pi that struggled with the same model, the N100 delivered “decent speeds” and handled a 16 K context window without exhausting memory. Qwen3‑4B showed comparable results. The most demanding test involved DeepSeek R1‑Distill‑Qwen‑7B (a 7 B parameter model). Despite the lack of dedicated VRAM, the model ran at roughly 2.9 tokens per second, producing correct outputs until the context window became a bottleneck.

Practical use cases and limitations

The author does not intend to replace a desktop GPU with the N100 for heavy workloads; a GTX 1080 still powers a 4 26B Gemma instance, and an RTX 3080 Ti runs Qwen3.6‑35B. However, the LattePanda Mu can act as a secondary inference node for lightweight tasks, embeddings, or as a fallback when the primary GPU is occupied. Because the Proxmox host already runs essential LXCs, adding an LLM server incurs minimal additional overhead.

What to watch next

Future Intel N-series releases may increase execution unit counts and improve media driver support, potentially raising token throughput on similar iGPU‑only setups. The community is also experimenting with Mixture‑of‑Experts (MoE) offloading, which could enable even larger models on modest hardware. Monitoring driver updates for Intel Media SDK and Vulkan extensions will be crucial for anyone looking to replicate or extend this experiment.

Conclusion

The Intel N100 iGPU, when paired with a well‑tuned LXC environment and the llama.cpp Vulkan backend, proves capable of running 4‑7 B parameter language models at usable speeds. While it cannot compete with dedicated GPUs for high‑throughput inference, it offers a cost‑effective edge solution for developers who need occasional local LLM access without investing in expensive hardware.