Birgitta Böckeler’s recent evaluation of local models for agentic coding reveals that while hardware constraints remain the primary bottleneck, specific configurations like Qwen3.6 35B MoE offer a viable, albeit messy, alternative to cloud-based assistants. By Birgitta Böckeler.
Birgitta Böckeler’s recent assessment of local large language models (LLMs) for software development challenges the assumption that local execution is currently impractical for serious coding tasks. After a month of testing on Apple M3 Max and M5 Pro hardware, Böckeler found that while local models lag significantly behind cloud-based giants in overall capability, they have crossed a threshold of utility for specific workflows. The primary editorial question is not whether local models are better, but whether they are viable enough to justify the complexity of local setup for developers seeking privacy or cost savings.
The technical reality is governed by strict hardware constraints. RAM remains the core bottleneck; model weights must fit into VRAM, limiting viable models to the 15–25GB range with context windows capped at 64K. While response speeds have improved dramatically, particularly with 4-bit quantization on Apple Silicon, the quality of outcomes remains inconsistent. Böckeler notes that reasoning capabilities, often enabled by default, can be counterproductive, causing smaller models to enter endless loops. Disabling reasoning often resulted in faster, and occasionally better, performance.
A critical friction point for agentic coding is tool calling. Local models frequently fail to emit structured syntax that matches harness schemas, such as the discrepancy between old_text in Pi and oldString in OpenCode. While models can often self-correct, this unreliability makes them less suitable for fully autonomous agents compared to human-in-the-loop chat interfaces. Despite these hurdles, Böckeler identifies Qwen3.6 35B MoE as the current sweet spot, offering the best balance of capability, speed, and RAM footprint.
Engineers should watch for improvements in tool-calling reliability and the emergence of more standardized harness integrations. For now, local models are best viewed as a specialized tool for privacy-conscious or offline workflows, rather than a plug-and-play replacement for cloud assistants. Excellent read!
[Read More]