Guidance for AI-assisted Drupal development using local models

I think that is an excellent and worthy goal to pick “pick one, canonical approach.”

As of this week (month, whatever), I would agree that Drupal development with a Claude Pro ($20/month) description is notably better than Drupal development with the free tools. Not only that, but I can be even more productive with a Claude Max 5x ($100/month). And my colleagues report they can be even more productive with a Claude Max 20x ($200/month) plan. I know some Drupal developers who almost always have Claude running multiple agentic workflows, solving real problems. So should that be the “one, canonical approach”? I don’t know.

I talked to some folks from nonprofits this week at DrupalCon who are not allowed to use AI because of concerns about data sovereignty. I talked to others that were only allowed to use the $8/month nonprofit subscription, which they told me had limited features compared to Claude Pro. Other folks I know only have access to Gemini, because that is what comes with their organization’s Google subscription.

Practically speaking, saying “Claude Code” is the canonical approach seems “correct.” It reflects the reality of my experience in the Drupal community. Most Drupal agencies, and the clients they serve, probably use Google/Microsoft for email/calendars/meetings, GitHub for code hosting, Amazee/Acquia/Pantheon/etc. for hosting, Slack for chat, plus a suite of a whole lot of other proprietary services that have a cost associated with them. Adding a Claude Pro subscription probably won’t bother most folks, and is probably the answer they want to know.

FOSS stack

So with all of that said, what I’m talking about is the “one, canonical approach” for people committed to software freedom, and perhaps that belongs in another Drupal project. I’d be totally fine to create one of those elsewhere, such as ai_best_practices_foss, and I think that would be logical and avoid confusion.

What I would prefer, however, is if this project had a sub-module that could be optionally enabled or somehow maintain a separate stack, not just that is “free as in beer,” but that aligns with other ethical frameworks. Here are some of my specific use cases:

Coding in schools. When I am teaching kids to code in schools, I would love to be able to teach those kids Drupal AND to show them that they can use AI without also telling them they need a subscription to Claude Code. I’d like to be able to go into a Linux User Group for kids and not tell them they need to use proprietary software.

Small nonprofits. For the past 15 years, I’ve been running Drupal site on a free DreamHost site (with free domain registration and free email). It’s nice that’s free, but more importantly, it’s nice that the nonprofit can use Drupal and make choices that align with the ethical commitments of the organization. I like to add a How We Built This Site link that describes the choices that we make and how other people could do the same thing without compromising their values. Much like with AI, many of the choices we make proprietary alternatives that work “better,” such as Drupal’s Simplenews over “free” email services, Drupal webforms (rather than Google forms), Calendar module (rather than Google calendar), Drupal search (Google search), OpenStreetMap (rather than Google Maps), etc. I’d love to be able to add some open-source AI features that work “pretty good.”

What goes in this repo

I still run Debian stable, use Vim as my code editor, Thunderbird for mail, Firefox as my browser, etc. I’m far from a free software purist, but I do my best when the information I have. I’ve always appreciated how Drupal generally aligns with my values and how I can be confident in telling people that they are welcome in the Drupal community, that we are ready and eager to welcome you and teach you Drupal, and that you don’t need much more than a computer and a connection to the Internet (which, admittedly, excludes many people). Oh, and lots and lots of time, which excludes other people.

In some ways, this reminds me of when I started using Ubuntu on donated computers in 2004. Back then, it didn’t work nearly as well as Mac OS on my PowerBook G4, but it gradually got better and better. Likewise, in my view, the current top-tier Ollama models seem generally considered as good as, or better than, the original ChatGPT (GPT-3.5) from 2022-2023. So I think for many audiences, a “pretty good” open model that aligns with ethics and doesn’t support the big tech companies would be the “canonical approach.”

Whether or not the guide for how to do that belongs in this repo might be a topic of discussion, but I wanted to make my motivations as clear as possible.

@fathershawn That's a great article from Carson Gross. Thank you for sharing (By some strange coincidence, I was just listening to the Talking Drupal episode with you and Carson today!)

Since you commented here, I'm curious if you saw a connection between that article and using open source and free software for AI-assisted Drupal development. For instance, are you seeing connection between the open models being more predictable like, say, a high level programming language?

Here are a few more considerations of performance and freedom:

Performance

I was surprised to read (and see it echoed elsewhere) that open models now trail state-of-the-art proprietary models by only about three months on average.

I don't fully understand all the benchmarks -- and they are changing rapidly -- but it looks safe to say that last year's top models from Anthropic, OpenAI, and others don't match what's freely available in the open today. Worth noting: most of these models are freely available ("free as in beer"), but their licensing falls under "open weights," not traditional open source.

Software freedom

So depending on the targeted audience, this guidance could address "open source LLMs" or also "open weights." And it seems like folks wouldn't have to give up as much in performance as you did even a couple years ago to use open models.

The other big factor, it would seem, is computing power. Based on current 2026 standards for open-source models (like Llama 3 or Qwen 2.5), someone might likely need a fairly beefy computer to run Drupal on DDEV and open models. But a lot of "Power User" with 64GB of RAM and a fast CPU might do fine. For instance, on a computer with a dedicated NVIDIA GPU (RTX 3060 or better) or an Apple Silicon Mac (M1/M2/M3/M4), the LLM will run on the GPU/Unified Memory. Then again, if free software is a matter of liberty, not price, maybe we don't need to consider price.

@fathershawn No problem!

Also popping in here to note some other sources that may be related to this issue:
- Drush integration: https://www.drupal.org/project/ai_drush_tools
- Documenting how to use MCP module(s) with Opencode: https://www.drupal.org/project/mcp/issues/3569504. The Model Context Protocol (MCP) is an open protocol that enables seamless integration between LLM applications and external data sources and tools.
- I think we might end up recommending a Drupal-specific opencode plugin (to work with an Ollama module) https://opencode.ai/docs/plugins. I poked around at the various lists of list and I haven't found one yet, but that might be a thing.

Who, that trinity Large Thinking is 797GB file (https://huggingface.co/arcee-ai/Trinity-Large-Thinking/tree/main) that probably requires 200+ GB of RAM to run. I have a 2TB drive and 64GB of memory, which feels like a lot, but I didn’t try that. Doesn’t seem practical for most folks. Or maybe there is a smaller version that might work better.

I’ve tried out Gemma 4 on both ollama:

# Get and run ollama run Gemma 4
ollama run gemma4:26b

And llama.cpp (which ollama uses under the hood and can be faster):

# Download Gemma 4
wget -O ~/models/gemma-4-26B-A4B-it-UD-Q5_K_XL.gguf \
  "https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/resolve/main/gemma-4-26B-A4B-it-UD-Q5_K_XL.gguf"

llama-server -m ~/models/gemma-4-26B-A4B-it-UD-Q5_K_XL.gguf \
  --port 8484 \
  --jinja \
  -c 32768 \
  -fa on \
  --temp 1.0 \
  --top-k 64 \
  --top-p 0.95

I was surprised at how slow it was. I guess asking "what kind of tasks are you good at?" is a complex question.

Trying out those 2 models makes me thing that FOSS isn’t the only consideration for AI-assisted Drupal development with FOSS tools. It seems like it wouldn’t be particularly helpful if most Drupal devs can’t run it on their laptops.

I’ve been thinking “digital sovernity” (https://events.drupal.org/rotterdam2026/news/drupalcon-rotterdam-why-dig...) and “software sovereignty” (https://dri.es/the-software-sovereignty-scale).

Drupal is made possible by GPL code supported by 501(c)(3) organizations – which exist for the public good – such as the Drupal Association, DDEV Foundation, PHP Foundation, and the MariaDB Foundation (“aiming at US 501(c)(3) status”).

I thought maybe in addition to running sudo apt install php, mariadb-server, apache2, etc. we might be moving toward a world where AI tools could be like, suitable for the Debian repository, for instance.

In my view, the closest thing to that is the Allen Institute For Artificial Intelligence, which is building a “fully open AI system.” Their OLMo and Molmo is some of the most transparent and educational material you can find on model building

So, from what I can tell, the projects that seem most aligned with Drupal and FOSS values would like to a stack such as:

• Aider
• llama.cpp
• OLMo 3

# Download Olmo 3.1
wget -O ~/models/Olmo-3.1-32B-Instruct-Q5_K_M.gguf \
  "https://huggingface.co/unsloth/Olmo-3.1-32B-Instruct-GGUF/resolve/main/Olmo-3.1-32B-Instruct-UD-Q5_K_XL.gguf"

# Run Olmo in llama.cpp
llama-server -m ~/models/Olmo-3.1-32B-Instruct-Q5_K_M.gguf \
  --port 8383 \
  --no-jinja \
  --chat-template chatml \
  -c 32768 \
  -fa on \
  --temp 0.6 \
  --top-k 50 \
  --top-p 0.95

And Olmo was painfully slow. It might be exactly what folk need who just want something to run in the background, but all I can say is it felt A LOT different from what I’m used to with Claude.

So it seems like this might be more like composer or symfony situation, without nonprofits or GPL licenses involved. At least for now.

The practical recommendation might be a lot like what @webchick suggested way back at the start of this issue, except there seems to be a trend in folks moving from ollama to llama.cpp for greater control and performance, such as:

• https://itsfoss.com/llama-cpp/
• https://www.nijho.lt/post/llama-nixos
• https://www.xda-developers.com/switched-from-lm-studioollama-to-llamacpp...
• https://vitalik.eth.limo/general/2026/04/02/secure_llms.html

Regarding open weights vs open source, I think we can trust using open weights (the model can’t do anything by itself) + open source tooling (can verify the code) + local inference (nothing leaves your network). That’s what we get with llama.cpp. And in my testing, Qwen3-Coder-30B has been pretty darn good. I figured out which one would be best for my particular laptop running llmfit.

So, while I am admitting still early on in my learning, I feel like we might suggest:

• opencode
• llama.cpp
• Qwen 3

And if that feels too slow for whatever the dev was expecting, try llmfit.

I’m not sure if it would be better to discuss in a merge request or not.

Yeah, I feel like it's tough to come up with free/libre recommendations without also considering capabilities, performance, open models vs open source, security, free software, sovereignty, etc.

Ultimately, yes, I feel like the key thing here is offering an alternative to Claude Code that doesn't send your code/data to a third party. Maybe the first step is just a practical setup guide for one workflow, something like OpenCode + llama.cpp + DDEV -- while using the same Drupal-specific configuration from this project and honest notes about where local models hold up and where they fall short.

The broader terminology and licensing questions deserve a brief section for context, but they shouldn't block the practical guide.

I'll poke around more in this repo and see if I can come up with a MR.

@murrow Yes, I would love that!

I've been trying to get these to work with OpenCode + llama.cpp + DDEV:

• gemma-4-26B-A4B-it-UD-Q5_K_XL.gguf
• Olmo-3.1-32B-Instruct-Q5_K_M.gguf
• Qwen3.5-27B-UD-IQ3_XXS.gguf
• Qwen3-Coder-30B-A3B-Instruct-UD-Q5_K_XL.gguf

But everything seems really slow.

Darn! Devstral 24B did eventually respond, but at ~6 tok/s for prompt processing and ~2 tok/s for generation on a Core Ultra 7 155U (CPU-only, 64GB RAM). A simple 'hello' took over 90 seconds before the first token appeared.

Qwen3-Coder-30B-A3B is dramatically faster on the same hardware thanks to its MoE architecture. It's pretty zippy in a browser, but I've just had problem getting the opencode configuration working. Dense 24B+ models are probably impractical without a GPU. The guide should recommend llmfit early so people don't spend time downloading models that won't run well on their hardware.

I tried some specific flags that helped such as --threads 10 (use more threads), -np 1, and smaller context windows, but they don't close the fundamental gap between CPU inference and GPU inference. The guide should probably include quant recommendations based on available RAM.

I have had no problem getting descent local models running in my browser using llama.cpp. I can chat with it at reasonable speeds. I try to connect these models to opencode and it becomes painfully slow. I'm sure the RAM differences are part of the issue.

However, for interactive coding agent work, it seems like CPU is the limiting factor, specifically memory bandwidth. A M4 Max has up to 546 GB/s of memory bandwidth, while my Intel Core Ultra 7 tops out around 90-120 GB/s. That's it's roughly a 4-5x difference in memory bandwidth, which translates almost directly to 4-5x faster token generation.

It seems like this limitation is pretty well documented:

• https://www.digitalocean.com/community/tutorials/gpu-memory-bandwidth
• https://www.microcenter.com/site/mc-news/article/why-vram-for-powering-l...
• https://semiengineering.com/ais-rapid-growth-the-crucial-role-of-high-ba...

Compute performance has hogged the spotlight for most of the PC’s history. Memory and storage also mattered, of course, but most of what your PC could or couldn’t handle was determined by the CPU and GPU inside.

AI is turning that on its head.

So I unless I go buy a new computer just to run local models, I am not sure how I can offer much in the way of advice here. Further, all of this seems really complicated, and if this project is about "best practices" offering advice comparable to "use DDEV" then this might not be the right place to get into running local models. At least not yet.