Disallow composer.json and composer.lock from being indexed

I don't really see a reason why we should not even include it in our .htaccess, as this adds much more security, than the robots.txt here does.

Yup, this makes the files unaccessbile via the web on my install. Having the additional layer in robots.txt makes sense IMO -> RTBC

Adding to both .htaccess and robots.txt is overkill. Plus we should add to web.config.

Updated title to reflect #7 - it looks like the question remains: can we solve this via .htaccess and web.config? Yes, for the most part. At the same time, it seems thorough and simple enough to additionally disallow indexing via robots.txt. As a backup for non-htaccess situations, its not overkill so much as just being thorough. I believe the thing to do is combine #7 and #2 but I won't move this to Needs Work until we have that consensus.

Backporting this to Drupal 7 would be ideal, as I'd like to be able to use different versions of certain tools (like Drush) on different projects.

Attaching a D7 patch; but marking as "do-not-test" because it won't apply to the 8.0.x branch (testbot runs tests based on the Version in the issue metadata).

I can't think of a good reason why someone would want to access composer.json or composer.lock directly from the web: if anything, malicious visitors could use the information in the files to start looking for vulnerabilities. I vote we block it with .htaccess and web.config.

Looking at robots.txt, I do not see any lines which correspond with blocking rules in .htaccess or web.config. Based on this precedent, I do not think that it's necessary to disallow search engine access to composer.(json|lock) in robots.txt as well.

Missed the change to web.config in #10.

We can improve Drupal\system\Tests\System\HtaccessTest to test this.

Added composer.json and composer.lock to the list of protected files to test in \Drupal\system\Tests\System\HtaccessTest::testFileAccess().

This should work.

Updated issue summary to keep it up-to-date with the latest decisions in the comment section.

Whoops, used code tags instead of del tags on Remaining Tasks that were Done.

+1 for this change

I actually think this is Information Disclosure. According to #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json you should be able to modify this file therefore could potentially include raw data or links to custom repositories. I'll bump this to major and tag with needs security review.

Feel free to move down if you disagree.

patch doesn't apply

Rerolled and submitted for testing, based on the assumption that we want users to receive a 403 when attempting to access composer.json and composer.lock.

    // Ensure composer.json and composer.lock cannot be accessed.
    $file_paths["$path/composer.json"] = 403;
    $file_paths["$path/composer.lock"] = 403;

patch rerolled in #22

We have test coverage, so this looks perfectly for me.

Committed 8a0a5c1 and pushed to 8.0.x and 8.1.x. Thanks!

I think we should mention this change in the 8.0.2 release notes as it is a change to the .htaccess file.

Is there one in Drupal 7?
We don't have test coverage, this looks good to be in.

Yeah, I don't think we need automated tests for something like this.

Committed to 7.x - thanks!

I think we should mention this change in the 8.0.2 release notes as it is a change to the .htaccess file.

This was tagged for the 8.0.2 release notes, but it looks like it wasn't in 8.0.2 at all - rather it's scheduled for 8.0.3. So I'm changing the tag accordingly.

I would suggest in general that the Drupal 8 release notes start including something like the standard line used in the Drupal 7 notes ("No changes have been made to the [list of files] in this release, so upgrading custom versions of those files is not necessary") and then of course highlight cases like this issue (where a change was made) after that.

Here are some recent examples for reference:
https://www.drupal.org/drupal-7.40-release-notes (which had some changes)
https://www.drupal.org/drupal-7.41-release-notes (which did not)