Improve instructions for updating composer.json and /vendor

Added #2494073: Prevent modules which have unmet Composer dependencies from being installed to IS.

Also, if you say composer update you'd rightly expect the listed dependencies to be updated. But they won't be, because composer-installer puts Drupal-flavored stuff (including their composer.json files) all over the file system beyond where composer expects it to be. That's fine for initially building a codebase based on an existing composer.json file, but breaks composer when its time to update dependencies. That's why this exists: https://github.com/paul-m/drupal-merge-plugin

Basically the reason its hard to document is because it's broken.

Here's what happens when you follow these instructions on a new installation of Drupal 8.2.x (after running composer install):

1. +++ b/core/UPDATE.txt
   @@ -64,7 +64,21 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +3. Determine if your project is composer managed project. You can do this by
   +   running "composer info drupal/core" from the command line. If this is
   

   $ composer info drupal/core
                                    
     [InvalidArgumentException]     
     Package drupal/core not found  
   

2. +++ b/core/UPDATE.txt
   @@ -64,7 +64,21 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   the command line in the root directory and skip to step 7. Replace X and Y
   

   X and Y where?

3. +++ b/core/UPDATE.txt
   @@ -64,7 +64,21 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +     composer update drupal/core --with-dependencies
   

   $ composer update drupal/core --with-dependencies
   Package "drupal/core" listed for update is not installed. Ignoring.
   Loading composer repositories with package information
   Updating dependencies (including require-dev)
   [etc]
   

I would like to point out that while its not an official Drupal project by any means, BLT handles this for you.

1. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +3. Determine if your project is composer managed project. You can do this by
   

   How about "Determine if your project is managed by Composer."?

2. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   successful, you have a composer managed project. If you don't have composer
   

   composer managed or composer-managed?

3. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   successful, you have a composer managed project. If you don't have composer
   

   Seems Composer takes a capital 'C' when we're talking about the project rather than the binary (e.g. page title/footer on getcomposer.org)

   Also, should we say composer-managed instead? If yes, would the capital 'C' not apply (since in this instance it could be because we're referring to the binary)

4. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   it will be present in the "replace" section).
   

   I'm not sure about that. You could have "drupal/core" added under the "require" section only, e.g. https://github.com/anavarre/drucker/blob/master/orchestration/files/comp...

   So perhaps it's either/or?

5. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   If the project is not managed by composer skip to step 4.
   

   Capital 'C'?

6. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   If the project is a composer managed project run the following command from
   

   Perhaps "If the project is managed by Composer"

7. +++ b/core/UPDATE.txt
   @@ -64,7 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +     composer update drupal/core --with-dependencies
   

   Since it's a new line, should we add a '$' for the command prompt?

8. +++ b/core/UPDATE.txt
   @@ -113,10 +126,15 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +6. Re-apply any modifications to files such as .htaccess or robots.txt.
   

   For composer-managed projects, should we mention drupal-scaffold's 'excludes' directive to prevent people from having to remember that .htaccess or robots.txt can be ovewritten?

9. +++ b/core/UPDATE.txt
   @@ -113,10 +126,15 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +     composer require drupal/address
   

   Same as above, should we add '$' here?

I think you don't mean composer-installer - I think you are talking about https://www.drupal.org/project/composer_manager which as the project page says is:....

No, I'm talking about the use case documented here: https://www.drupal.org/node/2822349

This is actually the main use case you'd want if you've got a codebase and want to start managing it with composer, as you might if you're using pantheon or whatever and you ended up needing drupal/address. Combine its solution with the subtree split of core, and we're not worried about how to document this any more, because it's reasonable.

Setting to needs work because of the review in #8

1. +++ b/core/UPDATE.txt
   @@ -64,16 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +3. Determine if your project is managed by Composer. To determine this run the
   

   Could be simplified to "by running the following command:" and would prevent repeating "determine" twice.

2. +++ b/core/UPDATE.txt
   @@ -64,16 +64,20 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   Composer installed and "drupal/core" is present "require" section of
   

   How about:

   in the "require" section of your composer.json file, then the project is managed by Composer and will require you to install the Composer executable in order to update it.

re: #7 perhaps something like:

Alternatively, when Drupal is managed by Composer https://github.com/drupal-composer/drupal-scaffold can help you with excluding files you'd like to preserve always when updating Drupal.

We should update the composer docs to mention it

It's mentioned at https://www.drupal.org/docs/develop/using-composer/using-composer-to-man... so it seems we're good here.

As far as I'm concerned the patch looks good now (assuming it comes back green).

+   If the project is managed by Composer, run the following command from the
+   command line in the root directory and skip to step 7.

Step #4 includes things like looking at the release notes and seeing if any manual changes need to be made to settings.php. Why would those be skipped?

+3. Determine if your project is managed by Composer by running the following
+   command:
+
+     composer info drupal/core

This assumes a global installation of composer. But if you follow the basic instructions at https://getcomposer.org you wind up with a local install instead, and the command is php composer.phar info drupal/core. Maybe at least list both of those as alternatives?

It may actually be better to just put something in the Status Report that tells you if your project is managed by Composer (it should be very easy to have code in Drupal that parses composer.json and checks whether "drupal/core" is present in the "require" section or not). Then it would not be necessary to mess with the command line to figure it out. Could be a separate issue.

Even better. I'll leave it up to @David_Rothstein to RTBC, though.

1. +++ b/core/UPDATE.txt
   @@ -64,7 +64,22 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +3. Determine if your project is managed by Composer by running the following
   +   command (replace /PATH/TO with the appropriate location for your system):
   

   This assumes the user has the ability or knowledge to "run a command", which is not part of the current expectations for users who install from a tarball.

2. +++ b/core/UPDATE.txt
   @@ -64,7 +64,22 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   If this is successful, your project is managed by Composer. If you don't have
   +   Composer installed and "drupal/core" is present in the "require" section of
   +   your composer.json file, then the project is managed by Composer. You need to
   +   install the Composer executable in order to update it.
   

   This could be misread as telling people who don't have Composer that they need to install it.

I like the new file organization. But:

1. With the patch applied, the information about looking at the release notes and seeing if any manual changes need to be made to settings.php is still missing from the "UPDATING CODE WITH COMPOSER" section (it's only in "UPDATING CODE WITHOUT COMPOSER") so the issue I raised earlier still needs to be addressed.

   Possibly it would be better to have a separate section about applying manual changes that is after the "updating code" sections, since a lot of the information would be duplicated otherwise.

2. +   On a typical Unix/Linux command line, this can be determined by running the
   +   following command (replace /PATH/TO with the appropriate location for your
   +   system):
   +
   +     /PATH/TO/composer info drupal/core
   

   I think it should say "replace /PATH/TO/composer with the appropriate location" instead, since as mentioned earier a default basic installation of Composer will actually have the executable named "composer.phar" (not "composer").

3. +2. Determine if any modifications to files such as .htaccess or robots.txt and
   +   re-apply them.
   

   Missing some words - should be "Determine if there are any modifications" instead?

4. +   excluding files you'd like to preserve always when updating Drupal.
   

   "always preserve"

Needs work per #24.

I ran through the process for non-composer-managed, since that's where the pain seems to be for a lot of people.

I re-ordered a few things and tried to be a little more more hand-holdy.

I noticed that there was no special instruction for extension directories, so I called those out in the copy-over steps.

We still have outstanding: #24.1 and #24.2, but 3 and 4 I got.

There's no good d.o documentation that says: Here's how you install composer enough to deal with these kinds of instructions, such as downloading composer.phar to DRUPAL_ROOT and using it. So we can't link to it.

Also, telling people to say composer require drupal/address really blurs the lines between whether the installation is managed by composer or not.

Shouldn't it be mentioned in UPDATE.txt that if your Drupal 8 site doesn't rely on Composer, it is also possible to update with just drush up? See: Update procedure in Drupal 8

I noticed some unhappiness with the 8.5.0 update, like this: https://twitter.com/geerlingguy/status/976840449339088897

So I thought I'd review this issue again. :-)

I didn't get to the composer update path, because the non-composer path needs some love, too. I did git checkout 8.4.x and git clean -fdx and installed. Then I followed all the instructions to update from an 8.5.0 tarball.

1. +++ b/core/UPDATE.txt
   @@ -64,15 +64,69 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   Copy all the files, but do not accidentally overwrite your modules, profiles,
   +   sites, or themes directories.
   

   This is still super awkward. The only way you can 'accidentally' do this is by following the instructions.

   Should say: "Copy all the files except...."

2. +++ b/core/UPDATE.txt
   @@ -64,15 +64,69 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +   You can find the release notes for your version at
   +   https://www.drupal.org/project/drupal. At bottom of the project page under
   +   "Downloads" use the link for your version of Drupal to view the release
   +   notes. If your version is not listed, use the 'View all releases' link. From
   +   this page you can scroll down or use the filter to find your version and its
   +   release notes.
   

   'Your version' could refer to either your old site or your updated site. Should be: "You can find the release notes for the updated version at..."

   Also, will there be a section in the notes that explicitly says, 'Changes settings.php'? Go to the 8.5.0 page and try to figure out from what's there whether the settings have changed.

3. +++ b/core/UPDATE.txt
   @@ -87,6 +141,39 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +5. If you did not have access to the command line, use an FTP client to upload
   +   the files to your web root.
   

   "...including the vendor/ directory."

I do not think that instructing users to run /PATH/TO/composer update should be the suggested method to update Drupal Core in the UPDATE.txt file. It should be something along the lines of the following to restrict the updates to Drupal Core only:

composer update drupal/core --with-dependencies

Otherwise, you are instructing users to update every module and project defined in their composer.json file, which is probably not what they are looking to do.

There's also now a --with-all-dependencies flag on composer update now, which is likely what people want to be using moving forward.

I don't fully understand the difference between the two options, but that doesn't help. The problem are cross-dependencies of symfony components that are *not* dependencies of drupal/core, most of the time it's symfony/config. It will try to update all others that we depend on but then conflict on that, which it is not trying to update as we don't depend on it (but console or behat do).

What worked for most is this: `composer update drupal/core symfony/* --with-(all-)dependencies`.

Also, the last 10 or so people commenting on #2943546: composer fail to upgrade from 8.4.4 to 8.5.0-alpha1 where people that did not use drupal/core but still tried to update with compoer and it looks like the merge plugin is for example making problems there. This is explained in the patch, wondering if it's clear enough or if we should do something like "If you get error X, then you are likely not using drupal/core, see Y on how to update drupal core without composer"

I think we are RTBC. We have documented composer update as the standard though we also have info. on how to only update core. Let us get it in. It will never be the final draft.

+++ b/core/UPDATE.txt
@@ -64,15 +64,51 @@ following the instructions in the INTRODUCTION section at the top of this file:
+   Note, if you want to only update drupal/core the following command will
+   probably work:
+
+     /PATH/TO/composer update drupal/core symfony/* --with-all-dependencies

Are we sure that symfony/* is needed here? #36 explains the initial rationale for that, but are we sure those problems weren't a consequence of https://github.com/composer/composer/issues/4795, which is now fixed with --with-all-dependencies as per https://github.com/composer/composer/issues/6661?

1. +++ b/core/UPDATE.txt
   @@ -64,15 +64,51 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +1. On a typical Unix/Linux command line, run the following command from the root
   +   directory (replace /PATH/TO/composer with the appropriate location for your
   +   system):
   +
   +     /PATH/TO/composer update
   

   Maybe say --no-dev, or add a comment that it's a good idea to add --no-dev to production for security reasons.

2. +++ b/core/UPDATE.txt
   @@ -64,15 +64,51 @@ following the instructions in the INTRODUCTION section at the top of this file:
   +     /PATH/TO/composer update drupal/core symfony/* --with-all-dependencies
   

   Using drupal-project, you can't update only drupal/core. Or you can try and it doesn't do anything.

To my mind, this issue *is* part of the composer initiative.

Given how long that's taken, and how long this issue has taken, I think it's fair to say we need to tell users about --no-dev. I'd write it but then I can't review it.

Regarding --no-dev: https://www.drupal.org/blog/drupal-8-will-no-longer-include-dev-dependen...

Followup for related changes: #3000374: Improve Composer update workflow in UPDATE.txt

The 'needs documentation' tag isn't really satisfied. In #44, @alexpott points to https://www.drupal.org/docs/user_guide/en/security-update-core.html with a frowny-face, so that could use some love. The fact is that currently Drupal doesn't have a supported Composer update workflow, which is why we have: #2958021: Proposal: Composer Support in Core initiative

That will give us a chance to define a workflow and also to document it. So I'm taking away the needs docs tag for this critical issue.

Agreed with getting this in as an interim improvement. We can open follow-ups to refine further.

Committed/pushed to 8.7.x and cherry-picked to 8.6.x. Thanks!