.git repository should not be included with composer package

I've tested with "composer require --prefer-dist drupal/coder:^8" on the empty folder and you're right. Other packages such as "vendor/symfony/yaml", "vendor/squizlabs/php_codesniffer" doesn't have the .git folder included.

According to Drupal's package repository documentation, this package type ("phpcodesniffer-standard") is not supported.

This might be confusing the repository and causing it not to issue dist zips.

Also Related: #2919786: drupal/coder in composer.lock broken resulting in broken composer install

I get a "lingering diff" in the coder folder, after downloading Drupal core 8.7.x, and preparing for development:

# download Drupal
git clone http://git.drupal.org/project/drupal.git drupal-8.7.x-dev
cd drupal-8.7.x-dev/
git checkout 8.7.x

# composer update, for some reason I need to run it twice
composer update
composer update

Try to commit everything, to get ready for development, git diff should give no result, but it does:

$ git add -A :/
$ git commit -m "first commit"
$ git diff
diff --git a/vendor/drupal/coder b/vendor/drupal/coder
--- a/vendor/drupal/coder
+++ b/vendor/drupal/coder
@@ -1 +1 @@
-Subproject commit 984c54a7b1e8f27ff1c32348df69712afd86b17f
+Subproject commit 984c54a7b1e8f27ff1c32348df69712afd86b17f-dirty

A work-around is to delete the /vendor/drupal/coder/.git-folder, but that shouldn't be necessary ...

Here is a link from Composer documentation explaining how to handle dependencies when packaging:
Should I commit the dependencies in my vendor directory?

Thanks for the link @benoit.borrel. I arrived at the same solution, and until the .git folder issue in coder module is solved, activating .gitignore before git add -A :/ will exclude the vendor folder from being committed:
mv example.gitignore .gitignore

Here is my attempt at a process to get ready for Drupal core patching and development in the fewest steps as possible:
https://www.drupal.org/node/1137316#comment-12717835

@mathieuhelie in #4
Composer installer plugin dealerdirect/phpcodesniffer-composer-installer (which is supported by Coder since 8.2.11) installs phpCS standards it finds among installed packages. According to their README.md it

search for phpcodesniffer-standard packages in all of your currently installed Composer packages

So switching package type to something else than 'phpcodesniffer-standard' could break this support.

Please see:

https://www.drupal.org/forum/support/upgrading-drupal/2017-10-06/update-...

This issue seems to prevent proper Drupal updates via composer. It's open now since January. Can someone please remove the .git folder from the package? Thanks!

Is this related to changes in March 2019 that now require authentication? I cannot get past updates and installation in composer in windows because a user name and password are now required at git.drupal.org

  - Installing drupal/coder (8.3.1): Cloning 29a25627e7
    Authentication required (git.drupal.org):
      Username:
      Password:

Same issue with the coder 8.x-3.3

Authentication required (git.drupal.org):
Username:
Password:

@dureaghin

I solved the issue by upgrading to the latest Git. The following link may help you.

https://confluence.atlassian.com/bitbucketserver/installing-and-upgrading-git-776640906.html

I think this happens because we have drupal.org as git source at https://packagist.org/packages/drupal/coder

I don't think there is a way to prevent that, let me know if you have any ideas!

@klausi: What I don't understand is: Is "coder" the only module/package with that property? Really nobody else uses drupal.org as git source?

This seems to break every composer update I do, have to remove the coder directory in vendor to fix.

@rfay: As a workaround you can use this command:

composer config discard-changes true

This will add the discard-changes option permanently to your composer.json config section.

Now, you can always use:

composer update --no-interaction

Personally, I created a shell alias "up" which runs that command which is much more convenient.

Absolutely not an ideal situation, though, but perhaps it helps.

tried #16, but that too is not working, it throws error during composer update

13:26:11  + php -d memory_limit=-1 /usr/local/bin/composer config --global discard-changes true
[Pipeline] sh
13:26:12  + php -d memory_limit=-1 /usr/local/bin/composer update --no-interaction
13:26:13  > DrupalProject\composer\ScriptHandler::checkComposerVersion
13:26:13  Loading composer repositories with package information
13:26:13  Updating dependencies (including require-dev)
13:26:52  Package operations: 0 installs, 5 updates, 0 removals
13:26:52    - Removing dnoegel/php-xdg-base-dir (0.1)
13:26:52    - Installing dnoegel/php-xdg-base-dir (v0.1.1): Downloading (connecting..
13:26:52    - Updating psy/psysh (v0.9.11 => v0.9.12): Downloading (connecting...)
13:26:52    - Updating guzzlehttp/guzzle (6.4.1 => 6.5.0): Downloading (connecting...
13:26:52    - Updating chi-teck/drupal-code-generator (1.30.5 => 1.31.0): 
13:26:52    - Updating drupal/coder (8.3.6 => 8.3.7): 
13:26:52                                                                                                                                                                                   
13:26:52    [RuntimeException]                                                                                                                                                             
13:26:52    The .git directory is missing from /home/jenkins/workspace/vendor/drupal/coder, see https://getcomposer.org/commit-deps for more information  
13:26:52                                                                                                                                                                                   
13:26:52  
13:26:52  update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [--with-dependencies] [--with-all-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]...
13:26:52  

There is definitely something wrong with how Coder is being packaged. If you look at the project's repo (and download a tarball of the 8.3.x-dev branch), you'll see there is no .git folder in either. However, when downloading a tarball of the 8.3.5 release from drupal.org, the .git folder is bundled with the release. Not saying it is specifically Coder's fault, but it might be worth moving this issue to the infrastructure queue.

Note that the drupal.org tarballs are not relevant because Coder is installed with Composer from packagist.org.

Maybe we should open a Composer issue to remove .git folders after installing a package? They might consider that a feature, I could not find a fitting issue at https://github.com/composer/composer/issues

Adding my two cents...
Our task runner for deployment removes all .git directories because there are a handful of other packages that do this same thing, and it makes deployment unusable. This worked until we upgraded to drupal coder 8.3.7 and now we're getting this error from composer:

The .git directory is missing from /builds/wvuf/FS/foundation-services/vendor/drupal/coder

FYI:
forcing a downgrade to 8.3.6 fixes this issue.

I'm having this issue as well.

@klausi asked for advice in #13, does anyone have advice for them? Any other module maintainers out there using drupal.org as the git source in packagist and not having this problem of leaving a .git file in the composer package?

What seems to be happening to me is that drupal/coder:^8.3 is being cloned, not downloaded, despite having prefer-dist set. Only module this is happening with.

I'm experiencing the same as #22 :(

Using version ^8.3 for drupal/coder
Using version ^1.8 for behat/mink
Using version ^1.4 for behat/mink-selenium2-driver
./composer.json has been updated
Gathering patches for root package.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 1 install, 0 updates, 0 removals
Gathering patches for root package.
Gathering patches for dependencies. This might take a minute.
  - Installing drupal/coder (8.3.11): Cloning 67c1bcf2d6 from cache

I've found the most easy way to replicate this is by creating a new Drupal 8 project. I'm using the latest version of 1.x composer 1.10.17.

thom@thom-xps:~/code$ composer create-project drupal-composer/drupal-project:8.x-dev drupal-8 --no-interaction
... deleted lines...
  - Installing sirbrillig/phpcs-variable-analysis (v2.9.0): Loading from cache
  - Installing drupal/coder (8.3.11): Cloning 67c1bcf2d6 from cache
  - Installing seld/phar-utils (1.1.1): Loading from cache
... deleted lines...
thom@thom-xps:~/code$ cd drupal-8/
thom@thom-xps:~/code/drupal-8$ find . -name .git
./vendor/drupal/coder/.git

I think what is happening is that drupal-composer/drupal-project has a script to always remove .git repositories from dependencies in the vendor folder. This is done for people that commit the vendor folder into git, then you don't want another layer of .git directories there.

If you don't commit the vendor folder in your setup then you can remove the script "DrupalProject\\composer\\ScriptHandler::removeGitDirectories" in composer.json so that only this remains:

        "post-install-cmd": [
            "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"
        ],
        "post-update-cmd": [
            "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"
        ]

I also think the the drupal/recommended-project composer package does not set up such a script, so you might also want to use that for new Drupal projects.

It seems like this was introduced in on of our projects from https://github.com/drupal-composer/drupal-project/issues/223 , so this was not even part of the drupal-composer/drupal-project template in our case.

Anyway, this issue is about people that commit their vendor folder and for them the .git folder is annoying. Sorry for the noise!

Not quite, this issue is about: *Only* the composer package of coder contains a ".git" folder. No other composer package for *any* other Drupal component does that. So, simply: It shouldn't. Plain & simple. To whom it is annoying or not, is IMHO irrelevant. There are a multitude of reasons why you wouldn't want that.

The .git directory currently seems to be included because this project doesn't include a dist version of the package, which means Composer will always fall back to the source, which includes the .git directory. This also means that the .gitattributes file is ignored. This currently happens because https://packagist.org/packages/drupal/coder currently watches https://git.drupalcode.org/project/coder.git, which doesn't have any context about releases and only has information about the git tags.

FYI: This is breaking our composer install on Pantheon. Drupal 9.1. Still a problem, and unclear how I can fix it.

Hey all, I found that adding this to the scripts section of composer.json seems to resolve the issue.

"post-install-cmd" : "find ./vendor -mindepth 2 -type d -name .git | xargs rm -rf"

Obviously there is something wrong with the module itself and we shouldn't have to do this, but it does seem like a functioning work around.

I am seeing this on an up-to-date Drupal 9 project. I get this all the time while working on the repo with SourceTree:

git --no-optional-locks -c color.branch=false -c color.diff=false -c color.status=false -c diff.mnemonicprefix=false -c core.quotepath=false -c credential.helper=sourcetree checkout -b updates/08-2021 --track origin/updates/08-2021 
Switched to a new branch 'updates/08-2021'
Branch 'updates/08-2021' set up to track remote branch 'updates/08-2021' from 'origin'.


git --no-optional-locks -c color.branch=false -c color.diff=false -c color.status=false -c diff.mnemonicprefix=false -c core.quotepath=false -c credential.helper=sourcetree submodule update --init 
fatal: No url found for submodule path 'vendor/drupal/coder' in .gitmodules
Completed with errors, see above

I actually deleted the submodule reference from the repo, and it just came right back the next time I ran Composer updates.

I came across while working with upgrade_status. As per the instructions for that module, I required the dev dependencies with composer require --dev drupal/core-dev:9.2.6 --update-with-all-dependencies. It installed, among other things, vendor/drupal/core, which was not able to be committed to the repo. Had I left it in that state, the autoloader would have referenced non-existing classes.

It's been a year since #28 correctly identified what is happening here. Can we get a maintainer to make an official release and stop pulling in the package as a git sub-module?

@klausi I believe this has been correctly identified and can only be fixed by a maintainer.

So it seems this is happening because drupal.org git repositories don't have a nice dist integration with packagist. We could switch packagist to use our Github repo at https://github.com/pfrenssen/coder , but it is tied to a personal Github account from pfrenssen.

We did this before in the past until I took a break from Coder maintenance, handed over the repository and wanted to archive/delete it. This caused a critical cascade for Drupal core and thousands of Drupal sites having this repository referenced in their composer.lock files. See #2919773: Fail to download drupal/coder because github.com/klausi/coder has gone missing for all the fun.

It is a bit unfortunate that packagist only really works with Github repositories and not other repositories like drupal.org for us.

Maybe we can move https://github.com/pfrenssen/coder under the offical drupal organisation on Github https://github.com/drupal ? Then it would be a bit safer from deletion than in a personal account and we could use that.

I will ask drumm from the Drupal association if that is possible.

I think the transfer to github won't help much, because drupal/coder is available on packages.drupal.org and composer 2.x uses "Canonical repositories" by default. See the docs on https://getcomposer.org/doc/articles/repository-priorities.md

When Composer resolves dependencies, it will look up a given package in the topmost repository. If that repository does not contain the package, it goes on to the next one, until one repository contains it and the process ends.

Therefore an exclude "drupal/coder" in your projects composer.json is required to load it form packagist (and github).

Coder currently isn't included on packages.drupal.org, because this caused issues (#3135034: 8.9.x-dev tarball INCLUDES CODER AS A MODULE (!) because Composer packages.drupal.org definition incorrect for Coder 8.3.9) in the past. This means that the current drupal/coder package is already loaded from packagist.org, we would only have to move the source from drupal.org to GitHub.

@Arkener: You are right. Sorry for the confusion.

Have this issue on a D9 site as well.

I am confused as how to fix this issue.

Tried deleting composer.lock and that didn't help.
I do not see the coder folder in the vendor folder.

Do I need to add something to the composer.json folder?

Any help is greatly appreciated.

We've got around this by removing .git directories from docroot/ and vendor/ using composer scripts. This will remove these whenever they get added by composer operations.

First add the following to the scripts section of our composer.json:

"scripts": {
	"post-install-cmd": [
	    "DrupalProject\\composer\\ScriptHandler::removeGitDirectories"
	],
	"post-update-cmd": [
	    "DrupalProject\\composer\\ScriptHandler::removeGitDirectories"
	],
}

Then in ScriptHandler.php, add the following:

protected static function getDrupalRoot($project_root) {
	return $project_root .  '/docroot';
}

protected static function getVendorRoot($project_root) {
	return $project_root .  '/vendor';
}

public static function removeGitDirectories(Event $event) {
	$drupal_root = static::getDrupalRoot(getcwd());
	exec('find ' . $drupal_root . ' -name \'.git\' | xargs rm -rf');

	$vendor = static::getVendorRoot(getcwd());
	exec('find ' . $vendor . ' -name \'.git\' | xargs rm -rf');
}

Ok, https://github.com/pfrenssen/coder has been stable for a couple of years, so I think we can risk it to use as source for Coder on packagist again. That should help people that have composer install problems when a git checkout from drupal.org was used. Now it should use Github zipballs again.

I also sent a message to pfrenssen on Drupal Slack to never delete the repository as I did :-D