Review and test the upgrade guide for Drupal 7

Taking this issue for the next couple of hours. Will update with notes on my progress with upgrading a site according to the instructions.

Did a read through of the entire book outline having to do with upgrading Drupal, and I see the challenge before us. My sense is that it needs a complete rewrite, to have maybe 3 "top"-level items instead of the 10 that have specifically to do with upgrading + the 3 that are alternate approaches. The top 3 in the upgrade outline, each its own single page, would be:

1. how to upgrade Drupal through SFTP/phpMyAdmin (probably the most applicable instructions to web hosts out there)
2. how to upgrade the site via the command line using 'mysql', 'mysqldump', 'tar -zxvf', and whatnot
3. upgrading Drupal using the command line using a CVS checkout

I'd like each document to include the following sections:

• a backup strategy (make it part of the official steps to upgrade, not merely "recommended")
• an upgrade planning strategy, including how to research whether contributed modules are available in the version the maintainer is upgrading to
• MAYBE mention that upgrading is an opportunity to do a site audit and to organize the site's files properly, like moving contributed modules out of the Drupal root and into sites/all/modules
• MAYBE a staging site strategy, to try an upgrade out before proceeding on the live site
• things to check afterwards, such as going to the Drupal status report

The document and follow-ups at http://becircle.com/how_upgrade_drupal_5_drupal_6 have the right approach to a thorough and sustainable upgrade procedure, and I'd like to see a condensed version in the handbook.

Relinquishing the issue, though I will come back to it this week.

I just wanted to note that a while back I submitted a rewrite of the D6 upgrade.txt file because I found the original was somewhat confusing and ambiguous (http://drupal.org/node/368516).

I'd update it again for D7 but it's not looking like anyone has the time to review it so I'm not sure if there would be much value.

I will throw my 2 cents in here and suggest that fields in core is probably going to dramatically complicate the upgrade path for D6 > D7. Very difficult to tell at this point exactly what that will look like.

Here is a conversation about this issue in the queue...

#366364: [meta] Data migration from D6 contrib CCK fields to D7 Field API

And a specific quote from that issue (bjaspan)...

I do not think this can be a normal hook_upgrade_N() function because it is unlikely that all field modules a site is using will be available to upgrade all at once, so the upgrade process will require multiple passes. Thus, it should be a module with a UI, perhaps showing all the D6 fields & modules, the D7 fields and modules they map to, and the status of each (upgraded/not upgraded).

Emphasis added.

I've got a preliminary checklist page for d7 upgrade : http://drupal.org/node/548922

If anyone else is working on this, wants to work on this, has completed it -- lets flesh it out together.

I'll help out - going to comb through the formatting, compare with upgrade.txt, etc. - will save changes as new revisions so if there's multiple people workin on it it'll be happytimes.

Just thought I would chime in here (partially at the request of emma jane).

I've talked to a couple of people including Richard Eriksson about this but have not posted this publicly.

Basically, I want everyone to know that you can feel free to take liberally from http://becircle.com/how_upgrade_drupal_5_drupal_6 if there is copy/text there that will help fill in content for portions of this documentation. At very least some of that content can be used as a 'stub' for some of the sections of the upgrade guide.

For what its worth, I think the skeleton at http://drupal.org/node/548922 is looking good, but will need to be broken into separate pages and lots of bullets that are there now could use some expansion.

e.g. for each of the following say where you find this information out.
___ Determine which version of Drupal is running on the site
admin/reports/updates - explain
___ Identify and write down the modules that are on the site
admin/build/modules - explain
___ Identify and write down the modules that are active and are NOT used on the site
admin/build/modules - explain
___ Understand which theme is active on the site
admin/build/themes - explain
___ Determine if any modules or themes have been customized
diffing modules - explain
___ Determine if Drupal base code has been customized
diffing core - explain

I've started some of this work. See the revision notes for the handbook pages.

crap - just reading this last comment, andremolnar you are totally duplicating the work i've done over the last couple days. :-/ not really sure how i'm going to integrate the changes now, but will try not to totally overwrite the stuff you've done. if you want to take this on instead, just say so and i'll happily move onto something else!

Ok, all my work to date has been integrated, I did some major reformatting and rewriting of the language of the pages I edited to match up with the Styleguide guidelines. Just a suggestion, it would be a REALLY good idea to update content to the style and formatting guidelines as we go so it doesn't have to be redone later. If you're not familiar with the guide, it's really helpful and detailed: http://drupal.org/node/338208.

I may do a little more work on this, but I'm going to switch and focus on patch reviews till code freeze, and then will come back to more documentation work. Onwards! :-)

Perhaps the upgrade documentation should also be broken out by OS and web server, or at least web server, as it is with the installation requirements, eg

For Apache on *nix:
For Apache on Windows:
for IIS on Windows:

Command line stuff is quite different across OSes, .htaccess (Apache) and web.config (IIS) are different, etc.

arianek. Sorry, I'm not sure which bits I duplicated (had you created unpublished pages?). I didn't do anything really new (besides creating the sub-pages). I simply started to organize the stub content that had been entered on the upgrade page.

At any rate, as a general rule I'm on board with 'be bold' when editing wiki type documentation. If anyone has issue with any of my edits (big or small) be bold and blow them away (provided that it improves the overall quality of documentation ;). I won't be upset.

@chicagomom can you think of specific parts of the upgrade process that are OS specific? I'm not sure I can off the top of my head. Still, if there are, then it might be best to include notes in those specific areas rather than maintain multiple upgrade docs that are 98% the same.

That is wonderful news - and very gracious on behalf of becircle. Thank you!

@andremolnar - no i'd been reworking the content locally (as i thought bekasu and i were the only ones working on this section) and was about to update, when i saw it had been broken up into pages - it's fine though, i didn't have too difficult of a time piecing in the bits i'd rewritten and reformatted, and i don't think i deleted any of the content you'd added, so altogether i think things are looking improved! anyway i'm switching gears to help with patches again till code freeze, so it will be a couple weeks at least until i check back in on this.

@andremolnar you're correct - it's mostly things like cp is not a valid command in a Windows environment. But rather than having Windows user instructions hanging off the "main" documentation though, it would be better to just note within the documentation what to do if you're using Windows. Eg

Use a basic copy (and paste) to copy all the files (including the .htaccess or web.config file) from your Drupal directory to the backup directory:

Unix: cp -rp /path/to/drupal_site /path/to/backup_dir

-rp = recursive and preserve permissions

Windows: robocopy c:/path/to/drupal_site c:/path/to/backup_dir /MIR

(or whatever your local install drive letter is)

/MIR - copy directories and subdirs including empty ones

One thing the current documentation is not good at is telling users when the documentation is referring to Unix-specific commands. Even a label would help. Option 2 on the above-mentioned page, for example, goes through creating a tar archive - not something you can do with Windows unless you have specific add-on utilities installed.

This applies here as well. And also on this page, "IV method 1: Use the contributed database insertion script on this page: also refers to Unix-specific commands, but it's not identified as such.

I'm almost finished dismantling the HOWTO section on install, upgrade, etc. Many pages have been moved to the "Archive" book, but some have moved over to the Install and Upgrade guides. Please use content as appropriate, shuffle pages into the right place, or put the page in the "Archive" book if it is no longer relevant.

I dropped off the face of the earth for the past two weeks as a couple of unexpected projects took over. I will be back on this in the coming week.

I finally cleaned up and expanded half the pages.

Still TODO:
New Site Preparation: http://drupal.org/node/570160 - Needs to be expanded and handbook writing stylized.
Actual Upgrade: http://drupal.org/node/570162 - Needs to be expanded and handbook writing stylized.
After Upgrade Checklist: http://drupal.org/node/570164 - Needs to be expanded and handbook writing stylized.

Maybe someone can help me track down a link or be willing to write a handbook page:
On http://drupal.org/node/550130 I added this line (among many others)
Has the module moved to Drupal core? Drupal 7 core has incorporated several key contributed modules into its release. For a full list see [link]

The problem is that I can't find the new core module inclusions documented on any handbook page and changelog.txt doesn't specifically name all the modules.

Help as much as you can.

I've created a related issue: http://drupal.org/node/615634 "Update CHANGELOG.txt to explicitly name new core modules and core modules that have been removed."
That issue is required for completing upgrade documentation as new core modules that have been moved in from contrib will impact on the upgrade path for people currently using those contributed modules in their 6.x site.

Need to use clean urls for documentation otherwise people cannot know where they are. The tree map is helpful but clean urls would be so much more as they are more easily searchable and stickable http://drupal.org/node/15365

The other thing is that the actual documentation is at http://drupal.org/node/548922 and its not mentioned in the actual description but way down in one of the comments. Even if this was conceived about a year back it would have been better to have a URL/node reserved for the same and made sticky. This would have been easier to know things around.

It would be nice/better to see this happening down the road as well.

The upgrade.txt page http://drupal.org/node/550126 should have had also the upgrade txt file attached or linked to the cvs/svn/git version of the file (if the file is being updated all the time.)

i'm thinking this should be one of the primary items to focus on, cool if i priority +? (feel free to reset if not)

I've gone through the existing upgrade guide, shifting stuff, cleaning up, numbering the pages that represent the major steps. It still needs a lot of work. Perhaps some or all of those step pages could be merged and I'm still not happy that there's a lot of overlap between http://drupal.org/upgrade/tutorial-introduction, http://drupal.org/node/29161 and the rest of the section. There are also a lot of comments to be merged in.

Mostly I've been trying to make it easier to grok what we have and make needed changes or even replace the whole thing if necessary. I do want to stress that it would be a bad thing if we wind up maintaining two upgrade guides - d7 and pre-d7 if they contain essentially the same information (and so far it looks like that's the case). We should be aiming for only one guide to upgrading which could include a page or two about special considerations for d7 (i.e. fields).

The upgrade steps in UPDATE.txt and at http://drupal.org/node/570162 from #8 to #10 (#9 seems missing) are confusing.

If I weren't confused by them I could probably do the rewrite.

We are supposed to copy our old sites directory—complete with contributed d6 modules and themes—to the new sites directory? Then verify the new configuration file (which file is that? settings.php? then it should say so) is "correct." What? How is anyone supposed to know it's correct? By what standards? Also, which version of the file is it? Is it the newly-copied d6 configuration file (which really means it's the old configuration file, right?) or is the user supposed to make changes to the file supplied in the d7 distribution?

Better would be to say something along the lines of "the database settings are correct"—at least that provides some context.

Then according to http://drupal.org/node/570162 the user is supposed to empty the sites folder. First, it would be helpful not to keep going back and forth between "directory" and "folder." I'm OK with either but pick one and stick to it. Second, is the user really supposed to delete everything in the directory that was just copied there? If that's the case why bother copying the directory or editing or verifying files in there in earlier steps?

Thanks!

I have started cleanup at the main landing page http://drupal.org/upgrade.

Edit: This is the diff so far. I have not spent much time checkin for grammar errors then I have focuses on removing clutter like unnecessary words and sentences, confusing sentences and the overall order of sections to improve the flow of the document. More UL-lists and H2 to make the structure clearer though as soon some of the paragraphs are more coherent we might drop some of them and/or move some of them back to H3.

hey steinmb - thanks for helping with this!!! i skimmed through the diff (since my last work on the page) last night actually ;) but will have a more in depth look when i get a chance!

feel free to keep at it and just make notes here of which pages you've worked through, and i'll review it all down the line for typos, etc.

hi again steinmb:

i was a little nervous seeing how much you'd changed on http://drupal.org/upgrade, but i think the content changes are very good. you definitely refined a lot of it, and i think it's a bit clearer now. good work! i made some edits:

- big cleanup of the grammar and typos (it would be great if you're doing a lot of edits and english isn't your first language, if you can run the edits through a good spell checker first, as i think it would catch a lot of the grammar errors too, we want to be careful not to introduce too many errors into very frequently used pages!)

- changed some examples to using mainly 6 and 7 (since 5 will no longer be supported).

- update manager is enabled by default, so made that reflected in the steps for monitoring for security updates

thanks for starting back on this, if you work on other pages, post here and i'll try to keep up reviewing them or find others to tag team with you!

Thanks for cleaning up the language after me ripping the page apart :) Have not found the time to read through the changes you did but they all sounds sane. I was e.g. unaware that D5 was to get removed from the doc.

Normally my written language is better but I got a late start with the article, did my first edits at 3am with the intention of only doing a few minor edits but after a few minutes came to realise that it had a lot of issues and perhaps was due for a major rewrite. I ended up doing the last edits at about 5am and was too tired to do any additional work on the spelling and grammar, and for that apologise.

Anyway I', glad that you liked the changes and I would love to team up. I did a few futile attempts getting in touch in the doc IRC channel but without any luck.

Cheers
@steinmb
Norway

Yes, the content changes were beneficial. It's not a big deal for me to clean up the grammar, but since I don't always get to it for a while, and it's a popular page, I didn't want to leave it too long!

As long as you note in the issue queue that there's something that needs review though, it'll get followed up on. Definitely keep working on as much as you like, especially in the upgrade guide, I haven't had a chance to do much with it. D5 docs don't need to get completely eradicated, but definitely 6/7 should be the primary content coming up.

I'm one of the main people working on the handbook, so I'm basically teamed up with everyone. ;) As long as you comment/post on the Docs issue queue I'll see it. And if you want, try IRC first thing in the morning perhaps - since I'm on the west coast of Canada, our zones don't overlap well, but I'm often working on Docs in the evening my time.

D7 beta is getting closer so I plan to spend some more time looking into our doc.
The next book page after http://drupal.org/upgrade is http://drupal.org/node/29161 that definitely need updating but I do not have edit access. How do I gain access to it? Do I need to be a part of the documentation team?

Bump!

Hi! Just though to ask what's the preferable work flow here if you like to help and find something that could be added/changed in the guide? Do you open a new specific issue for it or this issue is generally used? Is it ok to go ahead making changes (at least when it's obvious like spelling/grammar) or we bring it up here to discussion first?

Like I found on http://drupal.org/node/550130 (Make an Upgrade Plan)

Identify the modules that are installed (including ones that are installed and not enabled). To do this, you can either look at the listed modules on the Modules admin page (Administer > Structure > Modules), or in your sites/all/modules directory.

where D7 menu structures are used, while it clearly must be "Administer > Site Building > Modules" as we are talking about a D6 site here and not a D7 one?

I'm just starting with some D6->D7 upgrade testing and (finally) found http://drupal.org/node/548922 (Drupal 7 Upgrade Guide) so thought I could try to give a helping hand where applicable.

Yettyn:

indeed we are working on the D7 upgrade guide outside of the main guide so that it's not taken as a "finished" product.

if you want to help with it, you can make changes/additions and note them here (as people can view and revert revisions easily), but if it's just typos or formatting no need to note that on this issue. If you want to discuss changes first, feel free to post on this thread, as others working on it will see your comments.

also, please review the handbook styleguide if you're going to be working on it: http://drupal.org/node/338208

there is also a whole bunch of other D7 stuff to work on here if you are just looking for things to help out with! http://drupal.org/node/515870#handbooks

thanks!

could do to address some of http://drupal.org/node/808172 as part of this

The following pages need edit (I do not have edit rights):
- http://drupal.org/upgrade/copying-your-live-site-via-GUI (missing phpMyAdmin screenshots)
- http://drupal.org/upgrade/copying-your-test-site-via-GUI (formatting issues under mod_rewrite code snippets)

LE: On Make backups page, there is a link to http://drupal.org/upgrade/do-i-upgrade-the-db. I'm getting a 403 when trying to access that page.

fyi - this seems to be the active thread for the cck > field upgrade path #781088: Updating CCK Fields and Data from D6 to D7

TO ANYONE WHO HAS BEEN WORKING ON THE D7 GUIDE: Help!

I just skimmed through the current upgrade guide and the D7 version (which I haven't worked on in any detail for a year or so), and they are different enough that I'm really not sure what to do with them to get a finalized (ie. final as it can be until all the full contrib upgrade paths are settled) version.

Based on discussions at the meeting a few weeks ago, we want to do everything possible to avoid having separated sections/pages per version, which means merging the two versions.

And it seems like Andre, Richard, Lee and others have done a bunch of work to make the D7 version have a more approachable and well-organized structure. But because of these changes it doesn't really match the current version anymore... (or maybe it does and it's just not obvious?)

I think they need to be merged, then touched up/reorganized a little, and then re-reviewed against upgrade.txt to be ready, but if anyone sees a different path to done-ness please speak up. ;)

I'm going to call it a night on this, but would love any suggestions (or insight on end plans for this that anyone had in mind) on how to proceed. We need to have an as-good-as-possible version of this up when D7 launches in just over a week! I think I'll be doing some more work tomorrow, but will be offline traveling most of Weds.

Thanks in advance!

alright, just discussed with jhodgdon and webchick (and others) in drupal-contrib, and we've decided to keep the guides separate. i'll move this into the /upgrade section this afternoon and then let the release post writers know where to link to. assigning to myself, please nobody move anything around for now!

alright, the 6 > 7 section http://drupal.org/documentation/upgrade/6/7 has been moved into the main upgrade guide section, and i separated the 5 > 6 pages into its own section of the upgrade guide.

i think that there could stand to be some more rearranging, reviewing, etc. i'll keep working through the formatting and page arrangement for the whole section as time permits.

if anyone has a chance, it think it still needs to be reviewed back against upgrade.txt and to be tested as much as possible.

i've filed a separate issue for the contrib to core upgrade/migration stuff so this one can be closed ahead of it: #1009126: D6 CCK to D7 core field migration docs

(unassigning from myself)

had to revert some changes to a page, follow up here #1016834: Review changes to Upgrade process page if you're able to re-review and add any content back with proper formatting. thanks!

Hi y'all...

Ariane has asked me to post the notes I wrote out explaining what I had to do for a recent D6 > D7 migration here. This is with respect to migrating node content specifically:

1. CCK module remains in D7, primary purpose of which is the CCK field migrator: http://drupal.org/node/781088, maintained by KarenS. (this link is also clearly on the CCK project page)

2. I had to do two things to get this migration code working. A) turn off the PHP filter (drush dis php) because any PHP snippets in the node bodies that call functions which no longer exist in D7, or otherwise break, will break the migration code. B) manually drop the SQL tables that the migration code will fail with error because the field tables already exist (field_data_cckfieldname and field_revision_cckfieldname for each cck field). This latter point isn't mentioned in the thread anywhere, not sure why. If anyone comes across this I'd be interested to find out if they were required to do it too.

3. taxonomy, including content taxonomy, and locations migrated fine automatically.

4. field_covert module (http://drupal.org/project/field_convert) is the leading contender for migrating non-CCK content from contribs that aren't otherwise handled by the core upgrade, where the field should go into the D7 field format (vs. staying in its old contrib propietary format). Contrib maintainers are to write the conversion plugin modules for the field_convert module. The only one that I know thus far that exists is to upgrade legacy image (ie. non CCK image fields), referred to as image­_legacy (obtainable currently only through the CVS D7 image module).

If anyone has any questions that I might be particularly suited to answer, p.m. me!

Shiraz
Affinity Bridge, Vancouver

Reviewing at Docsprint

At the DrupalCon Chicago doc sprint -- I replaced the instructions on the D7 upgrade process page (http://drupal.org/node/570162) with a slightly modified and reformatted version of the steps from UPGRADE.txt. olafveerman is continuing to work on reorganizing the content in the upgrade guide.

Together with arianek and gravelpot, we decided on a more intuitive structure for the upgrading section. A lot has been moved and merged already, but unfortunately I did not get to finish everything. Three general issues could still be worked on:

1. After 'Upgrading from Drupal 5 to 6', a new section could be added about the steps after the upgrade process. This could contain:
   • http://drupal.org/upgrade/testing-the-new-site (merge http://drupal.org/node/570164 into this)
   • http://drupal.org/upgrade/copy-your-test-site
2. Review the 'Before you Begin' section
3. Clean up the http://drupal.org/node/635460. This section contains quite some redundant pages, including:
   • Create a Test Site
   • http://drupal.org/node/340073

tags

Can anyone here confirm that the initial task on this - to review and test the upgrade guide for D7 - is done now? I know it's had several reviews, and it sounds like people are following it to successfully upgrade sites.

If that's the case, we can open a separate issue to track comment #45 - reorganizing the section further. (In fact olafveerman, if you can open a new issue and post a link here that'd be great!)

Yes, gravelpot finished the review of the Drupal 7 upgrade guide during the Drupalcon Docsprint.

Created a new issue for comment #45: http://drupal.org/node/1126034

Fantastique! Thanks all!