We explicitly forbid using updatecode on drupal core but it seems like both updatecode and updatedb are up to the task. I haven't tried it, just requesting this :)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

seanr’s picture

I could really use this as well. I actually have major problems updating core right now. If I try to download drupal and then move it into the wwwroot directory (so it should overwrite the existing files, the mv fails saying "mv: cannot move `dir' to a subdirectory of itself, `../dir'" even though that's not what I'm doing at all.

greggles’s picture

What are the steps that need to happen? For a lot of people it's:

1. Download core
2. Remove some or all of robots.txt, .htaccess, and sites/ directories
3. cp -r drupal-6-xx/* drupalroot/
4. Run update.php

Is that what we want Drush to do? I think that steps 2 and 3 are where we'll have some debate. Perhaps they can be handled with options.

I agree, that this would be Drupal shell Nirvana.

nadavoid’s picture

I have a small shell script that does a version of step 3. What it adds is: it deletes the destination directories before copying the new files in, to ensure that any removed files are really gone. Here it is (as executed from web root of site):

rm CHANGELOG.txt; cp ~/private/download/drupal/drupal-6.10/CHANGELOG.txt ./;
rm cron.php; cp ~/private/download/drupal/drupal-6.10/cron.php ./;
rm index.php; cp ~/private/download/drupal/drupal-6.10/index.php ./;
rm install.php; cp ~/private/download/drupal/drupal-6.10/install.php ./;
rm robots.txt; cp ~/private/download/drupal/drupal-6.10/robots.txt ./;
rm update.php; cp ~/private/download/drupal/drupal-6.10/update.php ./;
rm xmlrpc.php; cp -r ~/private/download/drupal/drupal-6.10/xmlrpc.php ./;
rm -r includes; cp -r ~/private/download/drupal/drupal-6.10/includes ./;
rm -r misc; cp -r ~/private/download/drupal/drupal-6.10/misc ./;
rm -r modules; cp -r ~/private/download/drupal/drupal-6.10/modules ./;
rm -r profiles; cp -r ~/private/download/drupal/drupal-6.10/profiles ./;
rm -r scripts; cp -r ~/private/download/drupal/drupal-6.10/scripts ./;
rm -r themes; cp -r ~/private/download/drupal/drupal-6.10/themes ./;

I realize it's unnecessary to delete the individual files such as cron.php, but I have those in there just for the visual consistency of the script.

I've had no problems at all with this method.

seanr’s picture

Strongly opposed to #3 - if people don't keep their contrib modules in the sites/all/modules directory, #3 will cause all of them to be wiped out. Here's how I did it:

drush dl drupal;
cp -ar drupal-6.*/* ./;
rm -rf drupal-6.*;

That will overwrite all of the existing core files without touching anything the user may have added regardless of where they put it.

greggles’s picture

@seanr @nadavoid - people often modify robots.txt and .htaccess files. Your solutions ignore that which makes them non-solutions to me.

To be more clear about my proposal:

drush dl drupal
rm -rf drupal-6.*/robots.txt
rm -rf drupal-6.*/.htaccess
rm -rf drupal-6.*/sites
cp -r drupal-6.*/* ./
rm -rf drupal-6.*

This is how Acquia upgrades are created (more or less).

nadavoid’s picture

@greggles - good catch. So removing the robots.txt line from my example script would fix the issue raised by local customizations.

My example script only affects the directories listed, so unless someone has put files in core directories, their customizations should remain unaffected.

Your script is certainly much cleaner and more straightforward. I'd definitely prefer to see yours or something close to it.

The only problem I see is in the case that a file is removed in a new release. I don't have a specific example, so I'll make a hypothetical one. Suppose /modules/block/block.admin.inc is removed or renamed in an updated release. The cp -r command doesn't delete existing files. It only updates existing files or adds new files. So after an update, /modules/block/block.admin.inc still remains in the installation, even though it's no longer part of core. (gosh, I hope I'm making sense. I feel like I'm rambling.)

I'm pretty sure this is the reason that step 6 in Drupal's UPGRADE.txt file is "Remove all old files and directories from the Drupal installation directory".

I'm sure this scenario is rare in core updates, but it's pretty common in contrib module updates. So, in core, is this a big issue? If so, I see two solutions:

1. issue a rm -rf for every directory that needs to be updated. These would be: includes, misc, modules, profiles, etc.
2. add an option to the cp -r command to delete files that are not being copied in. I'm not aware of an option that does that, but I sure would like to find out, if it exists.

I'm very unfamiliar with drush itself, so I'm probably in a little over my head in this issue. And if so, I'll step back and be quieter. :)

greggles’s picture

@nadavoid, that's definitely a weakness of my approach and you explained it clearly. I think UPGRADE.txt is really focused on major releases (5.x to 6.x, etc.). It's quite rare for core to remove a file and I think that this could just be a documented "known weakness" of the script.

The last time I tried it the Acquia installer had all contribs and Acquia modules in a "modules/acquia" directory which would make removing the "modules" dir a problem.

Owen Barton’s picture

I think it would be best to do some more work on the updatecode command first #433950: Modernize updatecode command so that backup becomes a version control engine, and that the command (and the plugins) can roll back in the event of an error more elegantly before we approach core upgrades here.

seanr’s picture

I'm fine with #5. Hadn't really thought of people modifying those files (since I rarely do), but that makes perfect sense.

mlncn’s picture

FileSize
1.8 KB

I'd like to take a crack at this, the update code part.

I think the most robust way is to generate and apply a patch. This may be too many requirements for Drush core, but this approach may have a home in Drush contrib in any case, so I'm going to try it.

Here is the shell script version of this approach, attached.

ben, Agaric

moshe weitzman’s picture

I think this patch based approach makes good sense. Lets hear what others have to say about the approach and also how it should be implemented as a drush command.

greggles’s picture

I see that as a reasonable option, but ideally would like the ability to do either a patch upgrade or a full upgrade.

seanr’s picture

I'm with Greggles. One concern - will the patch approach properly update the version number seen by the update module?

mlncn’s picture

Assigned: Owen Barton » mlncn

@seanr: yes, as the patch is based off downloaded tarballs of Drupal core (not using CVS), so the resulting code is exactly as would be achieved by replacing with the latest downloaded tarball, all packaging information included. (Except -- don't tell anyone -- if you have made small hacks to core they may survive with this method.)

seanr’s picture

Very cool (except for that sound I'm hearing of a kitten dieing somewhere). ;-)

rsvelko’s picture

The time to do this right has come.

What do we got so far: (LET this be the main issue for this task - let all other issues listed here are sub-ordinate)

1. comment #10 above with a bash script doing a diff/patch file to apply onto the fresh drupal core we update to:

- link: http://drupal.org/node/434944#comment-1803900
- patch: http://drupal.org/files/issues/version-upgrade-diff.txt

2. The isCoreHacked issue - #528016: Create ishacked drush command

- link to arcstwewart's comment - http://drupal.org/node/528016#comment-1867138
- and his latest patch there: http://drupal.org/files/issues/drush.iscorehacked.v4.patch

- link to snufkin's comment: http://drupal.org/node/528016#comment-1883018
- and the patch: http://drupal.org/files/issues/drush-ishacked.patch

3. My vision on that matter is the following:

General thoughts :
- bash scripts are best for prototyping and should be used initially. Later the code would be migrated to drush and php
- all you people - put as many stuff in 'sites/' as you can - and ease the life of the drush developers and actual site updaters!
- I mean: 1. put your modules in sites/all ,
2. move the cache/site1, cache/site2 dirs of 'boost module' to each site's dir in sites (try to say this on 300ml vodka :) )
3. move the backup folder of drush to the 'sites' dir (if we update contrib module residing in sites/all/modules/mymodule - the backup should go to sites/all/backup , if it is in sites/site1/modules/mymodule the backup goes to sites/site1/backup - cause drush was invoked with a specific multisite in the cmd params )
4. There are some files that usually people add to the root drupal dir - like phpinfo.php . This one for example cannot be put in sites - simply because we are lazy:)

What is the problem and the solution?

- problem 1 /use case 1 - check if core hacked and move some diffs out of core (for example changes made to a hacked core .css file can easily be pasted into the theme .css or a new child theme or a new custom module...) - keyword "out of core" , keyword "easy core update".
- problem 2 / use case 2 - we need easier update - meaning we seem to worry that core is hacked when we come to update core ... not sooner :)

use case 1 is a sequence of actions that are a subset of use case 2.

The right solution should have these elements:

- at each step we must know what is the situation - is our core/contrib hacked?
- If the diff says YES - we should know what's wrong, how bad it is? It is our cute core - we want to help it recover.

Point 1: The core /contrib update/unhack process should be semi-automatic with a anti-hack hack-removal/migration functions.

Point 2: There are 3 main players: proj_live , proj_orig and proj_new (for example - my live drupal 6.12 site, a fresh 6.12 dl-ed from drupal.org and a fresh 6.13 ...) and the workflow should be like:

- diff live against orig and while resolving this do the same things to "new" simultaneously
(like copy a file/dir to both orig and new)

Point 3: We need the functions of the debian apt-get installer which once asked me about a locally modified version of /etc/apache.conf - "Overwrite with new version / Leave as it is / Check and resolve in the background and diff again then"

I've almost written the complete script. Till later.

rsvelko’s picture

I am finishing the diffmerge part with the semi-automatic diff resolution...

joachim’s picture

Drupal docs tell you to change the scripts in /scripts if your host requires you to run cron via lynx or something, so /scripts needs to be preserved too.

rsvelko’s picture

FileSize
3.07 KB
3.03 KB

Just so you grasp the idea and see the draft I attach the current progress.

- the "drupdatecore" is a "drush update core" bash script that will be generalized to cope with core and contrib projects

it uses :

- the "diffmerge" script is a general purpose 3-file by-design-semi-auto interactive diff merger (post apt-get update style - i.e.

"There are some diffs. For action press a number. (after each action you will see the new diff that results)

[1] resolve in background manually
[2] resolve in background with help from me (copy-paste multi-file cp, rm commands we create for you)
[3] cp $curr $orig
[4] cp $orig $curr
[5] rm $curr
[6] rm $orig
[7] produce a patch ...
[8] quit
"

The main benefit implemented so far is that you no longer have to manually download drupal 6.12 and 6.13 and write all other commands ...

Still much work needs to be done.

To test - copy both commands in your path (currnetly they do not interact in any way)
and go to a drupal dir and execute the drupdatecore one...

No harm will happen whatsoever - read the code if you want.

Tested on debian 4.0 .

rsvelko’s picture

once again please read my code to see the general ideas I go after.

For example the semi-auto merging would use an external file with definitions of which dirs/files should be pre-auto resolved - like the sites/, backup/, scripts/ part... that is a no-brainer and should not be resolved manually ....

rsvelko’s picture

note1 : I pasted above:

"
[1] resolve in background manually
[2] resolve in background with help from me (copy-paste multi-file cp, rm commands we create for you)
"

- I rethinked the way this should happen - no background user operation - but an copy-paste-editable multi file commands and editor invocations without backgrounding are the right way.

PS The whole idea is while merging the curr/orig diff to capture all actions and redo them on "new".

babbage’s picture

Yep, would love to see this in Drush. I'm surprised no one has linked to this tutorial yet in discussing how to do this:
http://drupal.org/upgrade/downloading-drupal-command-line

I would suggest that we either integrate that approach as much as possible, or if we feel there is anything in that tutorial that is incorrect we also correct it. Obviously the more sophisticated approach here allowing for diffs to be resolved is great if you have had to hack core (shudder) for some reason, and that's a great addition to the Drush approach.

Edit: Actually, I meant this tutorial: http://drupal.org/node/297496

rsvelko’s picture

I've just created this task #543564: Update drupal.org docs to reflect recent drush core/contrib shell updates capabilities which we will accomplish after closing up here.

rsvelko’s picture

@22:

I've read both tutorials - they bring nothing new to the ideas we have.

PS Currently I am considering that all diff resolutions will be aided by the patch command - previously I thought cp/rm is ok - but now I changed my mind....

I am currently rewriting the drupdatecore script to generalise it for contribs too. And I am thinking over the UI of the diffmerge resolver...

I need this one closed before proceeding: #543570: Need a better pm-releases command - with latest stable and current installed versions

rsvelko’s picture

Update: (this issue has become twitter-like - sorry )

The right names/invocations for these commands are:

1. core/contrib updater - shell: "drupdatecode proj1 proj2 ..." , drush : "drush updatecode proj1 proj2 ..."

2. mergediff3 - this should be a separate project on sourceforge.net - I searched there and cannot choose from the many there -

HELP! - somebody know of a similar project we can use? (I mean a semi-auto-semi-manual 3 files/dirs interactive shell-based diff merger? )

Edit: a graphical tool like kdiff3 MAY be an option here - but graphical tools (KDE...) on a server machine is a bad security practice.

babbage’s picture

@24 they bring nothing new to the ideas here, but would enable users right now to implement shell-based updates to Drupal Core, something not yet available in Drush. But something like #543564: Update drupal.org docs to reflect recent drush core/contrib shell updates capabilities was the main point of my comment.

rsvelko’s picture

moshe weitzman’s picture

Status: Postponed » Active

Lets not create dependencies where they need not exist. drush dl already knows the latest stable for core.

@rsvelko - i am having trouble following your comments and code. Lets all try to keep our replies simple and straightforward and similar for the code.

rsvelko’s picture

FileSize
5.13 KB

OK, this newer version of the shell script I've attached is quite usable.

Features:
- interactive (when selecting versions and type of project and type of diff) and verbose
- works from the root drupal dir but uses time-stamped subdirs of /tmp for operation
- works for core, modules, themes
- can be used as an iscorehacked command
- with no args - project=drupal
- accepts several projects - like 'drupdatecode admin_menu rules image'
- tries to guess the current and latest stable versions but leaves the possibility to override them
- downloads projects and outputs the diff automatically (normal diff or kdiff3 )

PS. 3way Diff resolution can be currently done via kdiff3 or manually by the user after he knows and interprets the diff

moshe weitzman’s picture

Assigned: mlncn » Owen Barton
Status: Active » Needs review
Crell’s picture

I'm confused. This is in the Drush issue queue, but what people keep submitting are bash scripts. Is the intent to bypass Drush entirely for core updates, or is this still in skunkworks phase?

I've been working on a drush command to do a core update; it doesn't work yet, but I'm getting closer. :-) (The basic idea is to do a normal download/unpack, but instead of untarring the entire tarball I explicitly untar everything in the package except htaccess and robots.txt. I don't have it expanding to the right directory yet, unfortunately.)

rsvelko’s picture

FileSize
5.37 KB

In my lengthy posts above I tried to outline the safe and smart way to do updates - via downloading and diffing - and just then overwriting and deleting sites/ dirs and such...

Bash is easier for me to prototype the future drush functions (which are very close to waht bash does well) and even actually release some working script..

I just could and I did it. While you do/develop sth like these bash scripts - you inevitably encounter unforeseen obstacles and by solving these riddles you can help the drush team...

The new release of this script fixes minor version detection issues.

babbage’s picture

Looking forward to this being in Drush.

When the patch arrives, I think we need a different name for the command than "drupdatecode". Apart from anything else, given our familiarity with "Drupal" to me it reads first as "drup-date-code" not "dr-update-code"... Not wanting to bikeshed, but better to throw it in for consideration now perhaps rather than later...

Will review the bash script soon.

rsvelko’s picture

as I said above (in my lengthy postings, sorry :) ) the right name for that drush command is "drush updatecode" - the updatecode command exists in the moment and deals with contrib modules only + it overwrites the whole module's dir without diffing... Furthermore the "updatecode" command is used by the "update" command that calls updatecode and updatedb to glue them and do the job...

rsvelko’s picture

FileSize
5.33 KB

here is the script with a major bug-fix: the final diff was previously diffing against sites/all/module/project_name - instead of against sites/all/modules - note the "s"!

Some other minor improvements too.

Crell’s picture

FileSize
3.23 KB

So I've also been working on this internally at Palantir, and have managed to get the following command working. I couldn't figure out how to make the code deeper down in drush_pm behave, as it seems to be much less modular than I'd want it to be, so I had to copy out lots of bits and pieces instead.

It's probably not the right long-term solution, but it works for now so I'm posting it in the hopes it helps someone else. :-) Dissect and use as appropriate. Hat tip to sdboyer for his help with the proper tar syntax.

mcrittenden’s picture

Subscribe.

Agileware’s picture

Subscribe

justin.hopkins’s picture

Subscribe.

Although I'm not currently, I've weighed the benefit of checking all of (and only) non-core files into an SVN repository (essentially .htaccess, robots.txt, sites/*, and a few others - am I the only one with one or two homebrew php scripts residing in the drupal root?). Would having these things checked into SVN make a script along the lines of this discussion a bit easier to tackle?

Owen Barton’s picture

Way back in #10 I pointed to #433950: Modernize updatecode command which I think is the correct first step here. This is to fix up updatecode so that the backup function becomes a (simple) version control plugin.

Once we have done that we can safely and more logically proceed to add new functionality of the kind that is discussed here. I think the 3-way diff is too ambitious for an initial patch (especially in terms of the user interface), and we should start with a simple 2 way and add 3 way later on. We need to make this as simple and intuitive as possible - even if we can just deal with the case of adding files (such as libraries etc) this will be a real time saver. This definitely needs to work for modules and themes, as well as core.

The main backend pieces of this would appear to be adding the ability to diff the working copy (new code) and the most recent revision (the API could allow any revision, at least for proper VCSs) of a directory. For the backup system this would move the current directory to backup then expand the new tarball in place and diff against it.

For the frontend piece I think the apt-get model is probably the right direction to start with though, offer files up one by one then choose between - (K)eep my version, Use the (M)aintainers version, view a (D)iff of the changes. It would be really great if you could select these options in your .drushrc (or even optionally remember these choices), so that it is easier in future - for Drupal core we could add default settings for .htaccess, sites, robots.txt etc.

ChrisRut’s picture

Subscribe

Shai’s picture

subscribe

rsvelko’s picture

I've noticed a new module was born - Hacked! - http://drupal.org/project/hacked - just to mention it here...

xPosting our issue and code to the hacked issue queue...

AdrianB’s picture

Subscribing

SeanBannister’s picture

sub

sagannotcarl’s picture

Subscribing. I'm happy to help test when it gets into a patch.

johnbarclay’s picture

FileSize
11.08 KB

This is a simple approach that allows specifying top level files and directories to ignore or replace. That is, you can set robots.txt or sites or profiles to either be ignored or replaced by the new drupal core patch. You can also set an overall default of "ignore" or "replace".

It has no version control or diffing. In my case, I'm playing a hosting role rather than a developer role, so version control is not part of my role. And we don't maintain/patch installs that hack core. When I have my developer hat on I check stuff in and out of version control with tools other than drush.

I've only tested it on windows server 2008. The drushrc.php file might have somthing like:

  $options['corepatch_file_prefs']['default_action'] = "ignore";  // ignore|replace default approach to directories and files
  $options['corepatch_file_prefs']['ignore'] = array("profiles","backups","drushrc.php","CVS","sites","files", "images",".htaccess","robots.txt","web.config","favicon.ico");
  $options['corepatch_file_prefs']['replace'] = array("includes", "misc", "modules", "themes",
        "scripts","CHANGELOG.txt","cron.php","index.php","install.php","LICENSE.txt",
        "MAINTAINERS.txt","update.php","UPGRADE.txt","COPYRIGHT.txt","INSTALL.mysql.txt",
        "INSTALL.pgsql.txt","INSTALL.txt","xmlrpc.php"
        );

in it to specify file and dir prefs.

moshe weitzman’s picture

Status: Needs review » Needs work

Thanks for the code. Setting this issue to 'needs work'. See Owen's comments in #40 for our goal here.

rsvelko’s picture

My bash script from #35 + Owen's view in #40 (about the apt-get diff resolution simple script ) are going to do a splendid job soon.

My script has worked for me 100% stable 100s of times now and it only misses the convenient diff-resolution apt-gett-ish part ...

(It misses its porting to php/drush too :) )

Basically my unhack bash script does a 2-way diff between your drupal site and the same version drupal tarball. (detecting the version automatically with a manual override if needed). It works for modules too and autodetects that this is a module ...

Does anybody know of a tool that does only the diff-resolve part ?

PS. There should be a command that just checks for diffs - most of the time I do not want to upgrade/update - just to check...

themselves’s picture

Subscribing. We've built heaps of Drupal sites, and we plan on basing all of the update functionality around Drush. The last remaining piece of the puzzle is updating Drupal core!

timwood’s picture

sub...

Owen Barton’s picture

For anyone wanting to work on this you will notice that #433950: Modernize updatecode command is closed for a little while - backup is now a version control (in quotes) engine. I think the next steps are these:

1) We need to change the backup folder structure so that rather than:
backups/123456789/sites/all/modules/mymodule/mymodule.info
we use
backups/sites/all/modules/mymodule/123456789/mymodule.info
and also add a convention so core goes in backups/core/123456789 or similar.
These changes will allow us to easily determine the most recent backup for a project or core.

2) We need to make sure the version control engines can find a clean instance of the core or project to diff against. It is may be easiest to generate a clean copy on demand (rather than saving a duplicate copy on install/update) by inspecting the project version. We may want to cache tar.gz files to make this a little quicker. For svn/bzr the nicest approach would be for them to either generate a tag with a specific name, do something clever by checking svn log for the modules info file (although that assumes that is not manually edited) or to record the revision number after the clean module was installed (in a text file?). We could start with the same approach as backup to start with though (just call wrapper functions).

3) We need a way for version control engines to return a structured array of differences to pm, which would then allow the user to decide what to do with each one, it would then include these responses so the package handler engine can act on them, e.g. by excluding a file when updating. A good initial step here could be skip the differences and user interaction part and simply allow users to feed the engines a comma separated list of excludes when updating. We would need to change how the package handlers make updates (e.g. wget would need to extract to a separate directory and then copy files in one by one, rather than simply dumping it on top as it does now).

greg.1.anderson’s picture

I was just wondering if you were still in-progress on changes, or if you were done as of #433950. I'll post here if I start working on this; others please do likewise.

Re 1) That would work. I think it would also work to keep the existing structure, walk the whole backup tree once, and build an associative array mapping from module => most recent module path. Your change is probably a little faster for updates, but I think that the existing folder structure is better for backup management (easier to dump old backups), and I think that's more important.

Re 2) For version control engines, can't you just do an update and let the version control system turn dead kittens into in-file conflicts? If we do anything more complicated than that, I would say that it would be sufficient to preflight for conflicts before doing the update so that we can present a warning.

Re 3) For wget, yes, sure. For version control engines, I have the same response as #2. Why code all that extra complexity? Getting a simple update mechanism that will help all of the users who have conflict-free installs is better than trying to make the first version save the dead kittens. I'd vote for tackling things like this in a follow-on improvement so that we can provide basic functionality sooner.

greg.1.anderson’s picture

Status: Needs work » Needs review
FileSize
13.41 KB

Okay, all you subscribers: it's time to put on your testing hats and try out this patch on a non-production Drupal site.

This is a simple implementation of drush pm-updatecode drupal that is designed to give some basic ability to do easy minor core updates with a minimum of code changes to the pm module. Owen's suggestions in #52 are great enhancements, but they are not covered in this version.

Here are some characteristics of this patch.

  • Drupal core cannot be updated at the same time that non-core modules are updated. drush pm-updatecode will behave as it currently does (notifying of available core upgrades only) if there are non-core modules that need to be updated first.
  • drush pm-updatecode drupal will upgrade Drupal core. drush pm-updatecode will do the same if there are no non-core modules in need of updating.
  • Drupal core cannot be updated if there are any non-core modules enabled. drush pm-updatecode will automatically disable your non-core modules for you (after warning and prompting) before proceding with the update.
  • If you run drush pm-updatecode directly, it will warn you that you need to re-enable your modules after running updatedb. (n.b. it will cause problems if you try to enable them before you run updatedb). If you run drush pm-update, your modules will be re-enabled for you automatically,
  • drush pm-update will run drush pm-updatecode a second time if there is an update of Drupal core available.

Under the covers, the updatecode process for drupal is done exactly the same way that modules are updated today. First, a new folder is created and named after the new release (e.g. 'drupal-6.16'). The contents of the drupal root directory is then moved into this new folder, excluding the file/folders .htaccess, robots.txt, backup and sites. Once we've made our pseudo-module directory, we run updatecode as usual to update this one folder to the correct version. Once the update has completed, we again move the contents of this folder, minus the same list of exclusions as before, back into the drupal root directory, and delete anything that is left behind in the temporary folder.

One known limitation: if you're using the wget package handler, and you've added some new files to your Drupal root that are not usually there (maybe some extra static resources or whatever), these items will be moved to your backup directory and will disappear from your drupal root. You can find them in your backup directory and copy them back manually for now; eventually this will happen automatically as part of the update process.

So, this patch is still in progress and really should be in the 'needs work' state, but I set it to 'needs review' to catch the attention of the many subscribers of this issue, as I'd like to get a wide range of feedback on this code from a number of different sites and site configuration styles. If this approach to updates of Drupal core works well enough, I'd like to see this go into drush-3.0, which hopefully will be tagged as a stable release by DrupalCon.

greg.1.anderson’s picture

Switch back to a core theme before trying this too.

Edit: drush update will try to update core even if there is an error updating modules and themes, which it should not do. The above version of update core does not ever get changes made to robots.txt and .htaccess, which is undesirable. A 3-way-merge similar to what Owen recommends should be done at least for these two files. Finally, and this should be obvious enough, but the core update won't go well if drush disable can't disable one of your modules or themes, which of course shouldn't happen, but is causing me trouble at the moment.

Getting closer, though.

greg.1.anderson’s picture

Status: Needs review » Needs work

Setting to 'needs work', at least until the items identified above are resolved.

Crell’s picture

I'm curious why you're doing a full disable of the entire site before upgrading. I've done minor version core updates over a hundred times without disabling all modules first, and not once has it been an issue.

greg.1.anderson’s picture

@Crell: The pragmatic answer is "because UPGRADE.txt says you should". Now, I've gone on record in the past making exactly the same observation that you did above. I suppose the question is, what are the odds that a module will or can hook a function that is called anywhere from the time that the Drupal core code is update until the time that updatedb is called? In theory, no one should be using the site through the usual web interface prior to the run of updatedb, but perhaps admins using the web interface to run updatedb might pull up some pages in order to get updatedb to run. If some module hook should fail catastrophically during a page load due to some db schema change that breaks said hook, that would be a little bit inconvenient. It does seem that the odds of any problem are very low, and my experience in this area matches yours. Still, I felt that I should fall back on the pargmatic answer for an 'official' update script.

Should pm-updatecode respect a command-line option that skips the pm-disable step when updating core? It seems even less likely that problems would be encountered when using drush to update.

moshe weitzman’s picture

I'm on record as thinking it silly to go offline for a minor upgrade. I'd just as soon skip that step.

Crell’s picture

Since you're a fool to update the live production site directly anyway (you want to update your offline copy and then push it via drush rsync or a VCS), I also think it's better to skip that step and all of the issues that come with it.

And if you did want to do it on live: drush pm-updatecode drupal && drush updatedb

The potential breakage window can't get smaller than that.

greg.1.anderson’s picture

@Crell: drush pm-update does the equivalent of drush pm-updatecode drupal && drush updatedb, but with one fewer drush bootstrap, so perhaps the breakage window is slightly smaller with former. ;)

Since the consensus is that disabling modules is silly, and since there are known disadvantages to disabling them (update fails if enable or disable fails, could lose the enabled/disabled state of all of your modules if drush updatecode drupal aborts without rollback...), the patch below will not disable modules unless you specify the --disable-module flag on the command line. Even this option may be eventually removed, depending on feedback.

pm-update will now only continue on to do pm-updatecode drupal if the update of the non-core modules was successful.

I decided it was better to just allow .htaccess and robots.txt to be overwritten by the upgrade, so I took them out of the exceptions list and added a warning that changes to these files must be manually preserved. Trying to preserve changes in just one file pretty much implies everything in #52, and I'd like to see a simpler core update working in a shorter timeframe. I think that the current implementation is reasonable in terms of convenience-to-upgrade vs. inconvenience in modification management.

Tonight I'll add preservation of non-Drupal files stored at the Drupal root.

This patch upgraded my test site cleanly to 6.16.

greg.1.anderson’s picture

Status: Needs work » Needs review
FileSize
14.64 KB

Here's an updated patch that also preserves non-Drupal files stored at the Drupal root. Seems to be working pretty well now, but could use more testing in more environments.

This patch upgraded dev copies of two real sites cleanly to 6.16.

ramirez.gerardo’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed and tested by community. Works great! Should save me hours of maintenance work.

Thanks Team Drush!

moshe weitzman’s picture

I'd just as soon remove the --disable-modules functionality for now. We'll put it back is a compelling reason arises. Otherwise, looks good.

greg.1.anderson’s picture

Status: Reviewed & tested by the community » Fixed

Committed without '--disable-modules'. Thanks.

Shai’s picture

Awesome work folks...

Will the project page soon list a "dev" version which includes the patch?

Or will we see beta2 soon?

And if I get impatient... I'm presuming that what was committed was update-3 from message #62. Yes?

Again... thanks for supporting sanity within the Drupal developer community.

Shai

greg.1.anderson’s picture

There's only a few additional things I want to see in drush-3.0 beta2 (e.g.#737610: Inheritance for site aliases); I'll be pushing for that release to come out soon so that we can have 3.0 stable released for DrupalCon. There probably won't be a dev release prior to beta2, but that's up to Moshe.

I committed update-3 with some minor deletions. I'd recommend that you just use drush-HEAD, but if you want to use beta1 with update-3, that would work too.

quicksketch’s picture

This is a great update, but drush up drupal will delete any .svn directories you may have, making this approach incompatible with SVN. I opened a separate issue at #757790: drush up drupal deletes .svn/.bzr/vcs files within core directories.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.