Has anyone found a reliable way to automatically update Drupal core and modules?

I think that ability would distinguish Drupal as the de facto CMS.

Thanks in advance for any recommendations,

GT

Comments

WorldFallz’s picture

It's not exactly automatic (which would scare me anyway), but i use the http://drupal.org/project/drush module and it works great.

grandtradition.net’s picture

Does drush facilitate core updates?

~GT

WorldFallz’s picture

ah... good point. No I don't think it does. For that I deploy directly from cvs so it's a single cvs update command. My updates are down to 2 commands: 1 cvs update and 1 drush pm update.

grandtradition.net’s picture

What I really mean to ask is:

Is there an automatic update module, script, what-have-you that provides a graphic user interface from within drupal admin?

If there isn't, will there be? The security updates and module update notices are too frequent to keep up with. I'm speaking for myself and others who aren't code writers, command line gurus, etc etc. This could include an automatic database backup to boot.

It just seems like there should be a simple way for webmasters to be able to focus on site content and community development, sales, whatever (!) without having to be bogged down for hours every month updating their core and modules (while the site is in maintenance mode).

So, I ask again: what is the outlook on such a venture?

Regards and with gratitude,

~GT

silverwing’s picture

http://drupal.org/project/backup_migrate

I'm not a coder or a command-line person, but I don't find it all that hard to fire up my ftp program and add files that way.

I doubt something like what your asking for will be available soon, unless someone steps up to write it.

~silverwing - yeah, I get annoyed with the "Then build it!" line, too :)

grandtradition.net’s picture

Agreed - it's not difficult to update core and modules, but you have to admit it's a little scary during the updates until you actually see your site running smoothly with the new info installed. And if you forget something in prep for the update(s), just hope you remembered to back up the site (a potentially voluminous and time consuming prospect). It's a potential fiasco for every update, with your site as the price to pay for errors, you know?

I will of course continue with manual updates with mild trepidation, but it's wierd (at least to me) that Drupal, a cms so widely distributed and used, is so heavily geared toward those with technical expertise. How many folks aren't getting the most out of drupal because they don't know it's potential? I'm sure I'm one of those people. I prefer to be a webmasterster for a for-the-greater-good kind of site without having to go back to school or squeeze in nightly javascript, jqeury, C++, sql, php, etc etc lessons.

I'm simply stating, and how could you disagree(?), that it would be preferable that there was no security risk (I'm assuming that's a real concern) with automated updates that could be controlled from the admin section.

Why is this such a big deal? It oughta be built-in from the get-go! Right?

Cheers,

~GT

silverwing’s picture

I do my testing on my test sites on my home computer so I have an idea if something isn't working right.

It's a potential fiasco for every update, with your site as the price to pay for errors, you know?

Personally, I'd rather make those errors than having a script doing something automatically make the errors.

I'm simply stating, and how could you disagree(?), that it would be preferable that there was no security risk (I'm assuming that's a real concern) with automated updates that could be controlled from the admin section.

There's always a risk, but I'm not sure it's as huge as some think. I'll be waiting to hear from WordPress to see how it goes with them.

~silverwing

grandtradition.net’s picture

So your entire site is replicated on your home computer? I know this:

1) It would take a LONG time to make a copy of GrandTradition to my home computer.
2) I would have to set up a server with the same specs and limitations as my host (virtually impossible), ie speed (cron timeouts would be affected, right?), php version, sql version, apache or other, unix or other, the list goes on and on...

How does the NYTimes (or some other site with vast amounts and types of information) test?

The module and core updates would have to be vetted to be compliant with the automated script. Errors would mean the module or core doesn't complete the update and reverts to pre-update status.

WordPress will surely have growing pains with this - and Drupal will probably learn from WP mistakes, BUT consider that WP will be taking ground and will be that much further ahead by the time Drupal comes around.

No one wants to be on an stubborn outdated system...

~GT

silverwing’s picture

So your entire site is replicated on your home computer?

Not the entire site up-to-the-minute. Enough data to work with to determine if what I want to do can be done.

BUT consider that WP will be taking ground and will be that much further ahead by the time Drupal comes around.

I didn't know this was a competition :)

~silverwing - need to upgrade a friends wp2.x site to 2.7 - might do their automatic upgrade to see how well it works...

grandtradition.net’s picture

You and I both know it's a competition when it comes to new webmasters choosing which CMS to roll with...

or even old webmasters that are new to the cms lifestyle! ;) HA!

silverwing’s picture

It's about finding the best tool for the job. And it really doesn't matter to me if someone picks Joomla or WordPress or Drupal.

~silverwing

WorldFallz’s picture

Not that I know of-- and I don't think it's likely there will be. Apparently doing this through the webserver (which is what the drupal ui is, afterall) is a huge security risk and many prominent drupal devs are adamantly against it.

There is the http://drupal.org/project/plugin_manager module, but I'm not sure if it handles updates.

BTW, the 'drush pm update' updates all your contributed modules. You don't need to type it once per module-- it will do all of them.

Core drupal updates are a lot less frequent than contribs, so I don't think it's such a hardship to have to handle that separately. I'd want to anyway.

EDIT: corrected the module link.

grandtradition.net’s picture

Do you think someone would be willing to be paid to develop this amenity?

WorldFallz’s picture

I don't know if that's an option either-- the couple of modules i know of that tried to do this or something similar in any way other then the approach taken with plugin_manager were summarily removed due to security concerns. I'm sure you could contract a developer to do this for you, but if it's not a community project you'll be solely responsible for getting it upgraded as time goes on.

Emyr42’s picture

This is a feature that's very well implemented in SMF, without requiring that the site be put into maintenance mode.

dellintosh’s picture

I use it on my client sites, and it's working great for handling core updates. It's called Aegir, and although it takes a little time to set up (took me around 30 mins) once that's done it drastically improves my speed of maintaining multiple Drupal installs. :) Check out the video to get an idea of how it works.

http://groups.drupal.org/aegir-hosting-system

And once this module (plugin_manager) gets done, we'll have an easy way of updating our modules too! :)

Hope that helps,
-dellintosh

emax’s picture

I do not see even one real "argument" against an automated update. All the points given above are just matters of personal preferences. I cannot even see one real technical reason against auto-updates.

As there is a detailed description for a systematic procedure to update a drupal installation, I cannot understand why it should not be possible to put this into a piece of software. It sounds arrogant to me, to claim that all drupal-webmasters shall be able to handle the updates as easygoing as the "I-am-a-guru-I-do-not-need-auto-updates" experts.

For me, it ended up that way, that I finally abandoned all drupal updates - it was to much effort and I hit problems twice due to a step I missed in the update procedure.

AND THAT'S FOR SURE THE BIGGEST SECURITY RISC.

I finally switched over to WP for that reason, which is a real pity: from a conceptual point of view I really prefer drupal, but the inconveniences and security-riscs associated withg manual updates prevent me from using it.

The bottomline is: Drupal Is missing a safe update procedure that does not pose the riscs of manual work. It is therefore risky to use Drupal for a website.

garg_art’s picture

Many PHP based open source give you capability to update the module or even core by using the simple browser.
Does any one have a pointer on why Drupal does not allow this considering it has such a great talent pool and large following.

devilhoused’s picture

This is an interesting topic because I sometimes want to move on from client projects without the worry of whether they can handle the task of site maintenance. At the time, it does keep my involvement fresh and on a customer-relations level it helps build rapport.

In my opinion, Drush makes for the easiest means to update Drupal.

Although it requires an understanding of command-line, Drush really helps to simplify download and installation processes. Drush does conduct core updates, and together with Drush Make it is possible to prescribe Drupal installations, pulling together resources from separate locations. The steps are simplified with the parse of a single text file. The sources the text file names are then downloaded. Per the Drush Make project page: "In practical terms, this means that it is possible to distribute a complicated Drupal [package] as a single text file."

For instance, I develop on a test server to make sure that everything is satisfactory, and then I create backups of web sites and transfer to the web servers. Many of the popular web hosts provide a command-line shell for file transfers while I'm at it. I also like FTP. Check out various resources online like this one. Ciao!

picxelplay’s picture

I would suggest using Acquia's (Dries's company) distro of Drupal. http://acquia.com. It comes loaded with more than the core, and has auto updates.

----
Sudo Kill Cylons

XTCHost’s picture

Has anybody written a PHP script to initiate Drush to do automatic updates via cron?

hopeeg’s picture

I agree with you picxelplay as that help developing Drupal sites, and the best offers a blog

mgifford’s picture

This is being called for in other places. From Google I got here. Generally if the community is going to develop something it really needs to be in the issue queue.

"What Drupal badly needs but doesn't have is an automatic updater that rolls out security updates by default."

http://nakedsecurity.sophos.com/2014/10/30/millions-of-drupal-websites-a...

I couldn't find a related issue queue about this, so created https://www.drupal.org/node/2367567

Codeblind’s picture

I've actually been toying with the idea of creating a node.js deamon that automatically runs security patches. However, I don't know if the Security team has an RSS feed pointing directly to patches. I'd really need something like that to hook into to do the job reliably.

Road Kill’s picture

I had multiple sites hacked and was not impressed with how the alert was handled. I had 10 websites hacked and spent the last two weeks cleaning and removing malicious code. The fact that the sites were not up to date was my fault but with more than a hundred sites to update an automated core update feature would be most welcome. I think if we had this feature then this would not have been such a big problem now.

I hope that in the next Drupal 7 release we can see this feature implemented as I think it would be irresponsible not to do so.

WorldFallz’s picture

Personally I would never enable an auto update for production sites, though I can see why some might. However, keep in mind that core can be updated quickly and easily with "drush up drupal" which can then be scripted fairly simply for multiple sites. Even without scripting, I had 22 sites updated within minutes of seeing the announcement.

Anonymous’s picture

http://www.bbc.com/news/technology-29846539

Mark Stockley, an analyst at security firm Sophos, said the warning was "shocking". "Many site owners will never have received the announcement and many that did will have been asleep," he said. "What Drupal badly needs but doesn't have is an automatic updater that rolls out security updates by default."

WorldFallz’s picture

And what will be the quote when the new automatic updater updates millions of sites automatically and breaks them?

I love the bbc ... but just because someone there has an opinion, doesn't make it any more 'right'.

There's also nothing to prevent one from scripting the drush command for multiple sites and attaching it to a cron. If you want auto updates you can have them right now. The fact remains, that the most likely users of an auto update feature are exactly those users least likely to be able to fix their sites when an auto update goes bad (which it will at some point, they always do).

Jaypan’s picture

World Fallz is spot on. With an auto-updater, it's not a matter of if, but rather when, you wake up to angry users of your site because it's stopped working after an update you didn't know anything about.

There are better methods to maintaining sites to deal with the issues that people are attempting to solve with an auto-updater. Regular db updates and using GIT for file management makes it easy to maintain the file system, and revert to a DB backup in the case of a hack like Drupalgeddon. Two of my sites were hacked, and I had the back patched, with hacked filed removed, and a DB backup installed, within 3 minutes of discovering the hacks.

dimmech’s picture

Ideally yes, developers should be on top of situations like this so that they are immediately alerted and are able to respond. I personally am not that developer. I have a handful of sites that I did for myself and a few others and was not even aware that my sites were compromised until my provider took me offline for spamming. The nature of the drupalgeddon hack in many cases was to exploit then patch the vulnerability so as to have exclusive and evasive access for further intrusion. There are still a great many sites owners unaware of what has happened. Even if you were on top of things as a developer, you may have been asleep, away or otherwise indisposed at the time of the announcement and by the time you found out it was too late.

On the one hand I agree that auto updates are dangerous. But a better response must be sought out. Initially I thought that there should be a means by which the security team could issue a severity level and have drupal respond by putting the site into maintenance mode. That would not have worked in this case, so what actually would be needed is to monitor security updates using a process running outside of drupal on the server. Perhaps deployed as a plugin to popular web hosting managers or a cron script. Depending on the severity of an issued alert by d.o. It could be configured to force the site into maintenance mode (if that will block the attack vector) or stop the http service altogether and alert admins. If something like this were possible, I'd commit to monetary contribution to aid in the funding of such an effort. Shoot, I'd pay a monthly service fee for the peace of mind that I won't have to go through something like this again.

Greg Boggs’s picture

While we're waiting for automatic updates to move along in the Drupal 8 Core, you can use this bash script and Drush to automatically install security updates for Drupal 7 core and contrib: https://github.com/thenewgroup/auto-updates

If you find a bug, or want an additional feature, Pull Requests are welcome. The script does not support or maintain core/contrib patches. You'd need to reapply those manually after a security update ran. You should also consider using Varnish if you're going to use this script because Varnish will make any update related bugs not obvious to your users.

manuelBS’s picture

Since Drupalgeddon I started with a concept of a service to automate Drupal updates "from the outside" (instead of a module that updates Drupal itself "from the inside". Some important criteria was to integrate the update process with our development and deployment workflow as well as automated re-applying of patched modules/core/themes.
Beside Drop Guard is a service I also opened the concept in a blog post in case someone is interested in details. It can be found at http://www.erpal.info/blog/blog/drupal-security-how-to-deliver-drupal-up...

I am very happy about any feedback or requests for personal demos hopefully to make the Drupal eco-system more secure.

Chris Dart’s picture

Everyone has rationales as to why this isn't necessary, but as someone who has to manage over 40 sites, some not hosted on platforms with SSH access or drush installed, it is time-consuming and frustrating given that the WP sites I manage take care of this all by themselves. This is the kind of project that needs a major developer sponsor because it needs to be done right, but we're operating in isolation from the larger developments in CMS. While Drupal 8 uses best computer science practices for a framework, it sucks at ease of use. Meanwhile WordPress keeps getting better and better and easier and easier.

Look at how easy it is to install a plugin in Wordpress compared to Drupal. Is it easy in drupal with drush? Yes, but WP has a command-line installer like drush, and a UI installer that allows users to browse plugins and install them right from within their site.

We can attempt to rationalize away all the reasons auto-updating is a bad idea, or not necessary, but one of the key reasons Wordpress is so popular is because it is incredibly easy to set up. Is it developed using programming best practices? Not even close. Does it work well? Yes. Is it getting as powerful as Drupal? Yes. It gets harder and harder to justify using Drupal when a tool is emerging that is proving more nimble and user-friendly.

That said, I suppose it is possible to set up a shell script and cron to develop a "poor mans" security updater. A script that runs

cd /web-root
drush sql-dump > ../backup-drirectory/$(whoami).sql 
#in my case most of my sites are chrooted under their own linux user accounts
drush up --security-only -y > logfile.log

would be a good start.

The infrastructure is all there for this to work, but it would require several keen minds working on the issue to develop it.

groovedork’s picture

Interesting. I was wondering if there are any easy to use brute-force hacks or work-arounds for this. I don't care anymore how it works or if it's an official part of Drupal, I just need some websites to auto-update. I'd rather deal with borked updates (since automatic backups ARE a solved problem), than have unsecure Drupal instances that are joining Russian botnets.

guillaumev’s picture

I started working on a service which uses your github repository (with your Drupal code inside) to automatically provide pull requests with new versions of modules. It's still in a very early state, but it can be found here: https://drupdate.io. Feel free to try it out and send me your feedback.

heshanlk’s picture

I think there are services like https://www.mydropwizard.com/ and https://www.webotics.io who does the same thing for you, I assume both do it manually so there are human hours associate with the process. Fully automated process might not be ideal with current versions of Drupal I believe (D7 and D8, D9 could be better who knows)

Heshan Wanigasooriya
Github

aphenine’s picture

One of the things I'm noticing with Drupal's direction of travel is that it's making life increasingly difficult for anyone on shared hosting. Access to the command line seems to be a minimum and most themes now require node-tools which just don't exist on servers unless you pay for them.

I'm not a big developer, I develop a few sites occasionally for friends and family because I know how to do it and because it's fun and because I know my way around HTML/CSS. Unfortunately, I'm getting the impression that Drupal does not like me.

This topic is very much in that vein. Yes, if you have access to your own server, updating is easy, and now there are auto-update tools that can be bought in. I even get why an auto-updater from within Drupal is considered a security risk.

However, could those of us who don't do Drupal as our day job and don't have access to the countless tools you all seem to take for granted actually have something too, please? Because I'm still sure that, at my end, an automatic updater is better than not having one, and the more I look into writing websites, the more I'm convinced being a one person web dev IS a security risk. Yet, somehow, didn't everyone start here? Or am I missing something?

VM’s picture

However, could those of us who don't do Drupal as our day job and don't have access to the countless tools you all seem to take for granted actually have something too, please?

you do. There are other CMS's out there that aren't pushing the envelope or dancing on the razors edge with each new release

Chris Dart’s picture

Typically, once a Drupal site is configured, it can be hosted on most any shared hosting environment. It seems that Drupal community has decided (explicitly and/or tacitly) to focus more on enterprise-level site development. Platforms like WordPress have a low barrier to entry, but with complex sites, the skills required are comparable to Drupal (albeit quite a bit different).

WorldFallz’s picture

If anything, in the intervening time since the op, Drupal seems to have gone even further away this type of gui updating. The increasing reliance on composer is def pushing it in the direction of not shared hosting friendly, or even shared hosting hostile at this point. In no particular order, see

To be fair, there is #2845379: Provide optional composer integration but don't force users to understand how to use composer, but the relentless march to composer appears to be far stronger than the effort to make it friendly/accessible to shared hosting and site builders.

aphenine’s picture

Thank you for that super-constructive comment. Those links all proved really good to read.

If I'm understanding them correctly, Drupal itself is using Composer to bundle its dependencies. Module authors are also using Composer to bundle libraries with their modules (I experienced this personally with the one Drupal 8 site I've done). In order to avoid duplication, Drupal merges the Composer system from the modules and its build Composer system in order to prevent duplication.

While this seems effectively simple and sensible on the surface, it means that Composer has become a core part of Drupal's module management system, not just Drupal's build system. This effectively means Drupal is unbuildable without Composer (screwing over the site builders), Composer will have to be integrated with module management tools (tricky, also not doable on simple webhosts and a serious security risk using Composer in a way it was never meant to be used) or moved to an entirely new system that's Drupal only (defeating the "Get Off The Island" approach).

Is that a fair summary of the whole issue?

WorldFallz’s picture

Yep, that's pretty much how I understand it. There are already a couple of drupal modules that simply cannot be installed without composer. The https://www.drupal.org/project/address module is one-- not exactly an 'advanced' module. Fairly basic functionality imo. So we've already, by default, pushed ANY user to having to navigate the morass of composer to test drive it with address functionality.

One of those threads states that there are resource limitations (composer is a resource hog) that many shared hosting won't/can't accommodate and the response has been, 'you should only be running composer on a local site anyway'.

So now in order to test drive drupal 8 not only do you have to figure out composer, you are also expected to have a local *AMP stack, with composer installed, to do it. If folks hesitated to take drupal for a test drive before, now they will SURELY RUN from it, lol.

it's only anecdotal data, but just look at the reported installs for addressfield and address: 140k to 8k. That's like a 5% percent conversion. It's difficult to find another such ubiquitous module that has versions for 7 and 8, but metatag shows something similar: 326k to 29k, an 8% conversion.

Whatever the 'official' position about not leaving behind site builders and non-professionals may be, this type of ground roots change ends up being the deciding/overriding factor. It doesn't matter that core isn't strictly requiring composer/local development-- if main, bread and butter, must-have modules like address are requiring composer and therefore a local environment, that is an ipso facto requirement for Drupal.

Requiring composer and a local environment will probably serve as the death knell for non-professional/hobbyist use of Drupal. I can only speak for myself, I'm a professional Drupal developer, but I will no longer be using Drupal past version 7 for friends/family/hobby/non-profit type sites.

Def seems like a case of expert blind spot.

aphenine’s picture

Thank you.

Those stats are interesting. I also bumped into a must-have module that used Composer as a replacement for code it used to install via the Library module. It was irritating as hell, though I got it to work after getting SSH access and downloading Composer onto the shared host. Tedious as hell and never again for a quick build site.

I switched to Wordpress for my current site. Annoying, but much, much better for shared host building.

sprite’s picture

It seems notable that this thread has existed for eight (8) years.

For Drupal7 neither core nor module updates have historically been relatively straightforward.

For Drupal7, I long ago achieved the ability to maintain a master superset of Drupal7 core and numerous modules, so that a single primary update could be done, and then an archive of that copied to every Drupal7 site, where only the .htaccess, settings.php, php.ini and a few other files needed to be kept in place after copying in a fresh code archive.

For Drupal7, mass module updates are easily accomplished right from the admin GUI as well, including automatically going to the update.php script page for database updates, making that a relatively straightforward process once it is well understood by a maintainer.

For Drupal7 even with patched versions of certain modules, maintaining a standard superset code archive across dozens of Drupal7 sites is possible, even with a different theme running in each one.

-----------------------
For Drupal8 maintenance, everything changes.

As has been noted, CLI access is a mandatory prerequisite.

When the CLI composer utility - composer update - command works alright it can be relatively painless, but when composer runs into problems, composer is very brittle, unforgiving, and a real annoyance.

As others have noted, the composer utility is also a huge memory hog, in addition to being an - issue command - and check back in half an hour type of affair, even on a Xeon powered server with SSD hard drive, and many GBs of memory.

One response to the above is that it was a huge mistake to essentially have reinvented Drupal in Drupal8, with an architecture of dependencies to complicated it needs automation to keep track of them. Then when the automation tool is as poorly constructed as is the composer utility, the problems are compounded.

IMO, the composer utility is ungainly, archaic, software that smacks of something thrown together back in the 1970s. Modern 21st century software, including CLI utilities, should understand and fix the problems ti encounters on its own, rather than just reporting the problems it encounters and giving up, which is what the composer utility does when it encounters problems. A truly well written "composer" utility, (should have been named conductor), would better understand the underlying dependencies it is supposed to maintain and be able to adjust configuration files and make choices on its own, while at the same time being able to actually inform users how to resolve any conflict it finds, rather than only reporting them.

-

For those of us with appropriate servers and CLI access, it is possible to live with the failings of Drupal8's maintenance design.

However, I have to admit that I am still in the stage of using Drupal8 only for research and development, but not for production websites, not just because of the composer utility matter, but because there are just too many gaps in the contrib module area to make Drupal8 workable at this point in time ( Commerce being a big part of that gap).

-

Notwithstanding all the foregoing, IMO, CMS software needs to be as user friendly and GUI oriented, even at the admin level, as the GUI friendliness the websites it builds hope to present to public website visitors.

spritefully yours
Technical assistance provided to the Drupal community on my own time ...
Thank yous appreciated ...

guillaumev’s picture

Hi,

The solution which I'm working on, Drupdate, is already running on 4 platforms (powering about 10 sites) and so far has been really a timesaver for me, as it does the module updates semi-automatically, providing a PR automatically each time a module needed update.

I am in the process of offering it as a service, https://drupdate.io. I really hope it can be an added value for you and your projects. If you think it can, let me know by subscribing to the mailing list on https://drupdate.io.

guillaumev’s picture

Just to keep people updated on this: Drupdate is now a github application, which you simply need to install on your Drupal 7 github repository. It will send you automated Pull Requests with module updates. Give it a try here: https://github.com/apps/drupdate.

Jaypan’s picture

Very nice idea!

ressa’s picture

jmev’s picture

For D8, it's possible to export the configuration of the site, including the list of enabled modules. Can't this configuration be used to pull that module from the git repository? I think even shared hosts allow you to have files uploaded by your site scripts. This would, in theory, replace the need for composer, as far as installing contributed or other publicly hosted module and theme repos.  Is there already a module in place that does something similar to this, and if not, how difficult would it be to create one (that conforms to Drupal 8 and OOP standards, of course)?

Jaypan’s picture

There is an automatic updates initiative right now that Dries spoke about at Drupalcon. So it’s coming.