Add gitignore(s) to Composer-ready project templates

Quoting myself from #2982680-110: Add composer-ready project templates to Drupal core:

Re #104

Prompting may be better for little details.

👍

That's also the philosophy that Drupal Console seems to use when generating things, for example to generate an Entity Type there is two distinctly different commands, one for generating a Config Entity Type and the other for generating a Content Entity Type, the questions you get asked are mostly similar but are all relating to small details about the configuration of the Entity Type and what features it supports.

diff --git a/composer/Template/README.txt b/composer/Template/README.txt
index 1d0c793203..3818c709b8 100644
--- a/composer/Template/README.txt
+++ b/composer/Template/README.txt
+  example.gitignore:
+  Includes the files managed by Composer into your git repo.
+  Useful if you prefer a "monorepo" where site updates are deployed using only
+  git. This is the same example.gitignore shipped with traditional Drupal
+  archive releases.
+
+  example-composerdeployment.gitignore:
+  Excludes all files managed by Composer from your git repo.
+  Useful if you prefer keeping your repository limited to your project's code,
+  and you use a more complex deployment process to build your complete site.

I like these descriptions, concise yet clear, +1

Diff and quote above are by @mbaynton in #2982680-104: Add composer-ready project templates to Drupal core.

My preferences for which possible solution should be used:

1. Add interactivity to our composer create-project, such that users are prompted to select the .gitignore appropriate to their persona, or opt out of a .gitignore completely.

I am very much in favour of this.

2. Have two differently-named example.thisorthat.gitignore files, neither of which is called .gitignore at the project's starting state.

I am also in favour of doing this in addition to option 1, I think doing both is useful, because further down the road the setup might change so the gitignore might need to change, so providing the example files would be helpful.

3. Create more template project variants differentiated by this, e.g. drupal/recommended-ci-deployed-project.

I'm hesitant about this because we could end up with a spiders web of templates just for differences between a single file, and I think 1 and 2 are better solutions for the problem.

4. Have one well-documented .gitignore with a big commented-out section covering persona b)

Doing this instead of option 2 might be better, reduces the number of overall files which need to exist.

Ultimately my opinion is: definitely do 1, along with either 2 or 4, and don't do 3.

Any prompting must be disabled via --no-interaction. Ideally, it should also be possible to provide default values, e.g. via environment variables.

Overall, I am kind of -1 on prompting in Drupal Core's project templates. When using a full-featured CLI tool such as Drupal Console or Drush, there is an entire config system available for defaults, environment variable overrides are possible, and so on. Composer is a full-featured CLI, but it does not offer us a good mechanism to pass values through to our customization scripts when using composer create-project.

Furthermore, if you are targeting your site to a very specific / opinionated hosting platform, e.g. Acquia or Pantheon, then ideally your host will provide a template that has been pre-customized to suit the environment. An example of this is example-drops-8-composer.

It seems better to me to provide a reasonable default with comments and instructions on how to customize things. We can put interaction in third-party tools and allow hosting providers to document how to best use Composer for their particular system. If we try to cover all contingencies in core, there's a risk that the user won't understand how to answer the question, and we'll have a lot of extra code to maintain. We don't want to have to involve a core committer every time someone wants to fine-tune how a template project installs for some specific environment, do we?

+1 #7

Generally I’m in favor of adopting to convention over configuration, because of better practices and decisions I don’t need to take myself.
However Drupal is not built to be conventional (there are many ways to do one thing, and sometimes you need to figure out which path to take on your own)

However, comment #7 raises some really good points and if I’m not part of the defined personas I would be lost in what choice I need to make, but also additionally do my own configuration afterwards anyways, or as Greg puts it try to pursue a fine tuning on the template.

Big +1 on sensible defaults and instructions how to configure for my own use cases.

Thanks for a great write up @mbaynton and outlining the different scenario options.

I am generally in favour of streamlining as much as possible and consolidating on an agreed best practice to simplify things, and reduce the likelihood of errors as much as possible. So I am in favour of shipping with a default .gitignore file, based on the best practice of Drupal, as well as for the simplest solution, to lower the barrier of entry for people just starting with Drupal.

This would mean as little interaction as possible during installation, with sane, best-practice defaults used.

I have read many places that having Composer installed on the server is bad practice. Yet, the current .gitignore in the Composer template for Drupal projects excludes many directories.

So, if Composer shouldn't be installed on the server, the default .gitignore should include all necessary files for git-based deployment from repository to server. Example .gitignore settings for other scenarios could be included at the bottom of the included .gitignore file, commented out. Having an extra example file like example.thisorthat.gitignore would only add clutter.

In essence, as comment #7 and #8, I vote for option #4

Have one well-documented .gitignore with a big commented-out section covering persona b)

#3084298: Make composer installer paths consistent between drupal/drupal and the project templates adds modules/custom/${name} to the installer paths. To support this, we should use the following .gitignore pattern:

# Ignore custom modules managed by Composer.
# Enable custom modules committed to the repo on case-by-case basis.
#modules/custom/*
#!modules/custom/module_in_repo

#10 I am wondering if we really should have modules/custom be ignored. I believe it is more common to keep your custom modules directly in the project's repository rather than maintaining them in their own separate ones. https://github.com/drupal-composer/drupal-project does not ignore the custom module path for instance.

I personally always commit my private modules directly to the repository; however, in my case, a "private" module is always a site-specific module. I do not have private modules shared between multiple sites. This use case is common in agencies.

If we were doing this from scratch, I'd put drupal.org Packagist modules in `contrib`, commit site-specific modules to `custom`, and put private Packagist modules in `private`. However, the type `drupal-custom-module` was invented and added to Drupal some time ago. I therefore think that folks are probably using it. Since the use-case is going to be primarily for private modules, it's difficult to do a survey.

It is possible for these two conventions to co-exist per #10. There typically are not very many site-specific modules in a site, so most sites won't have too many directories to un-ignore. Those sites that are outliers can always decide to change their .gitignore to match their use case. We are also suggesting that these ignore rules be commented out, so sites that don't want them can just leave them commented out.

In summary, I am not opposed to the idea in #12, but since the path referenced in #10 has a corresponding type that is (presumably) in use, that represents a certain amount of inertia, and I'm not sure it's worth making the change.

I created an issue, #3085354: Better default behavior for scaffold-created .gitignore files to track the .gitignore behavior described in #14.

Thanks @baynton, that looks like a great starting point for a .gitignore file.

Like you write, there are not a lot of comments yet. Perhaps we could add a bit about something that puzzled me a bit, namely the line with the exclamation mark? Perhaps add this, taken from https://git-scm.com/docs/gitignore:

An optional prefix "!" which negates the pattern; any matching file excluded by a previous pattern will become included again.

Also, I suppose the .gitignore file from composer/Template/LegacyProject/.gitignore will end up in the root of the installation after running composer -n create-project drupal/recommended-project:^8.8 my-project, right? I tried running the command, but it returned with Could not find package drupal/recommended-project with version ^8.8. ...

@ressa Either use 8.8.x-dev or add -s dev or use ^8.8@dev to get the dev version of that project. Basic composer 101 !

Thanks for helping out @mmjvb, all three options you suggested worked just fine.

Thank you for sharing the method for setting up a local Composer repository @baynton. It does seem a bit convoluted, and a bit of a chicken-or-egg-firsts situation. I will probably not try to replicate it, but still it will undoubtedly be quite useful to have the method documented. It might even be referred to from the Using Composer with Drupal guide?

I am glad to hear that .gitignore will be placed in the root of the installation.

Using composer create-project from on-disk templates is something that is only needed for folks developing a template project. Perhaps a good place for this would be in TESTING.txt in the composer/Template directory.

Basic composer 101 again: No need to make it more difficult.

mkdir my_test_project
cp /path/to/local/composer.json my_test_project
cd my_test_project
composer create-project

This assumes those packages only contain a composer.json. Obviously, you need the folder contents when they start to have more than just a composer.json.
Also, several existing distributions already use --no-install on the create-project command. That is for allowing to make the desired changes on the downloaded package. Folow up with composer install. That is the way to get rid of drupal/core-recommended and replace it with drupal/core.

Here's an updated patch with comments added to just the legacy project. Once we have wordsmithed the one project to our satisfaction, we can duplicate equivalent text in the recommended project as well.

Setting to 'needs work' because a) it does, and b) there's no need to run the tests on this patch; I changed only a .gitignore file.

@mmjvb If you're planning on participating on core development of composer initiative issues, please participate in a constructive manner that isn't insulting or demeaning. Running composer create-project on a directory with solely a composer.json is documented in a singular line of all of the composer documentation, and is neither "composer 101" nor "Basic". Its an obscure piece of trivia that you happen to know.

That obscure piece of trivia isn't relevant in this case, because, as you said yourself

This assumes those packages only contain a composer.json

This issue is about *adding* a file to those packages for .gitignore, and as such isn't suitable for testing the way you described.

@mixologic It is these kind of responses from you that makes me stay away from that project as much as possible.

I'm a little concerned that some of the discussion here, as valuable as it is, might be taking the issue a little off track. The discussion around how best to document and setup ones local environment for contributing to the Composer initiative development is of course valuable and important, but it should probably be addressed in a follow-up issue.

@mmjvb @Mixologic please consider reading the Drupal Code of Conduct before allowing this disagreement to escalate any further.

Going back to the primary topic, one thought that popped into my head is that in both the LegacyProject and RecommendedProject templates we would have gitignore files which are almost identical, the only difference being (as far as I can tell) the inclusion of "/web" for at the beginning of directories for one of the gitignore files. I wonder if instead of having these two static files having the scaffold plugin generate the gitignore file with the correct paths based on which template was used would be a good idea. One advantage of doing that would be that it reduces the risk of these files becoming out of sync in the future, and means updates to the gitignore only have to be made in one place, rather than two.

Anyway, think I'm going to lye down, my cold seems to be getting the better of me 🤒

I agree that it is always worth aiming for avoiding extra and duplicating work, but if it will delay the patch too much for inclusion in the next release, perhaps the task can be postponed for later refactoring? Except, of course, if it's very simple to do.

I hope you feel better soon @AaronMcHale!

@AaronMcHale Sorry to hear that you consider this topic going off track. My understanding is that this issue is about what to do related to a .gitignore in the to be introduced project templates. Looks like the OP considers this issue still open for debate. Clearly a .gitignore is related to how you manage your project. The point I made was there is no need for the mini repository. Copy your own template and use composer create-project or composer install in your new project. Consider the mini repository shooting with a canon at a fly.

Surely, the way you manage your projects influences your opinion about the alternatives provided.

@mmjvb #19 is really about testing this issue in a context where #3085354: Better default behavior for scaffold-created .gitignore files could be tested. That bug was discovered while working on this issue. The situation is that composer create-project will create a git repository at the target location, but only if it creates the project from source rather than from a dist. When doing a dist install, Composer simply unpacks the archive, and no git repository is created. When there is a git repository, the scaffold plugin behaves differently. The details are outlined in the linked issue.

@mbayton and I discussed testing that issue and this issue at BADCamp. The technique in #22 is not sufficient to test #3085354, because it does not create a git repository, which is a necessary precondition to expose that bug. I am not experienced enough with Composer to have known in advance that composer create-project can even be used on a bare composer.json file. I am still unclear on how create-project differs from a composer install in this instance; I think it would be necessary to read the Composer source code to discover the difference. In any event, though, a basic composer install is sufficient to do basic testing of the .gitignore in this issue. It is really good that @mbayton ran this code through composer create-project with a packages.json file, so that the code would run in-situ as it will once this patch is committed. We would not have found #3085354 otherwise. However, it is not necessary to test like that every time. Simpler techniques will suffice for most runs through the iterative development process.

The interrelationship between the different parts of this rather complex system are why I feel like the topics being raised here are not out of scope for this issue. The simple question of what we should put in our .gitignore touches on many different parts of the system, and shining light on some things that have not been considered yet. It's good to try to keep things on topic and focused in an issue, but sometimes it's better to expose important points in the main thread that everyone is following. By all means, we should put the fine details in follow-on issues such as happened with #3085354.

Finally, regarding #26, I think this is a fine idea, although it should be discussed more fully in a follow-on issue. My initial impression is that it won't happen before the alpha code freeze, and even after the freeze, it might be considered to be too much complexity to do template replacement on scaffold files. We are currently doing this for the autoload.php generation, but only there. I'm undecided on this point at the moment, though.

And Finally^2, I think I have already made the point that we're beyond basic Composer here, even for basic issues like what to put in the .gitignore. Everyone is working hard here, so let's all be nice, and be nice about being nice, k? Thanks everyone.

I hope you feel better soon @AaronMcHale!

Thanks @ressa!

Re @NEW in #28

Surely, the way you manage your projects influences your opinion about the alternatives provided.

I supose it would, I provided my initial thoughts in comment #5, I'm assuming you're referring to the options explained in the issue summary?

Re @greg.1.anderson in #29: Ah I now see more clearly why more of the recent discussion may have been more on-topic than I had initially thought.

No need to read the code, its documented:

create-project#

You can use Composer to create new projects from an existing package. This is the equivalent of doing a git clone/svn checkout followed by a composer install of the vendors.

There are several applications for this:

You can deploy application packages.
You can check out any package and start developing on patches for example.
Projects with multiple developers can use this feature to bootstrap the initial application for development.
To create a new project using Composer you can use the create-project command. Pass it a package name, and the directory to create the project in. You can also provide a version as third argument, otherwise the latest version is used.

If the directory does not currently exist, it will be created during installation.

php composer.phar create-project doctrine/orm path 2.2.*
It is also possible to run the command without params in a directory with an existing composer.json file to bootstrap a project.

By default the command checks for the packages on packagist.org.

Options

--stability (-s): Minimum stability of package. Defaults to stable.
--prefer-source: Install packages from source when available.
--prefer-dist: Install packages from dist when available.
--repository: Provide a custom repository to search for the package, which will be used instead of packagist. Can be either an HTTP URL pointing to a composer repository, a path to a local packages.json file, or a JSON string which similar to what the repositories key accepts.
--dev: Install packages listed in require-dev.
--no-dev: Disables installation of require-dev packages.
--no-scripts: Disables the execution of the scripts defined in the root package.
--no-progress: Removes the progress display that can mess with some terminals or scripts which don't handle backspace characters.
--no-secure-http: Disable the secure-http config option temporarily while installing the root package. Use at your own risk. Using this flag is a bad idea.
--keep-vcs: Skip the deletion of the VCS metadata for the created project. This is mostly useful if you run the command in non-interactive mode.
--remove-vcs: Force-remove the VCS metadata without prompting.
--no-install: Disables installation of the vendors.
--ignore-platform-reqs: ignore php, hhvm, lib-* and ext-* requirements and force the installation even if the local machine does not fulfill these.

Appreciate the elaboration on the circumstances. Wasn't aware of the consequences of using source. Not convinced its relevant here. Agree that the contents, number of and what provides them are relevant here. Sounds like a responsibility for the new scaffold plugin to me.

It is also possible to run the command without params in a directory with an existing composer.json file to bootstrap a project.

This does not describe in what way composer create-project is different than composer install when used in this way. I think for most purposes, composer install should be sufficient for testing this issue. I don't think that further examination of composer create-project is necessary, unless someone wants to help out with #3085354. If you have any concerns about the test coverage of that issue, please comment there.

That is documented as second line of the documentation:

This is the equivalent of doing a git clone/svn checkout followed by a composer install of the vendors.

OK, so we have two .gitignore files and they're essentially the same, just that the recommended project one uses a web/ path.

I can imagine a use case where someone wants to update their site to use composer, so they create-project a a legacy project and then just copy all the files into their existing Drupal repo.

Then they're angry because we messed up their .gitignore file. But wait, they're using git so they can revert, problem solved, we’re all happy again.

In the use case of a tarball site that has a preferred .gitignore configuration, our composerizer tool will likely create-project using the legacy template and then copy the existing files back over the scaffolded ones which is similar to the previous use case.

This leads us to some ambiguities about how to handle this situation, such as whether the composerizer tool should fix it, etc. Our idea of an MVP for the composerizer is that it should be able to detect the difference and warn the user about it with a suggested course of action.

Given that we’ll do this in the composerizer tool, we can go ahead with adding .gitignore files here, and add this scenario to the spec for the tool.

So RTBC yay.

+++ b/composer/Template/LegacyProject/.gitignore
@@ -0,0 +1,112 @@
+# is to register them in Packagist and give them the type

I think people do this with private composer repos', like private packagist and/or satis repos locally. Maybe this could be "register them in a package repository"

Either way though +1 on RTBC. There'd be no real way to efficiently document exactly how everything is done, so probably better to do this, iterate as necessary.

#39: Yeah, a custom module does not necessarily need to be private, so I thought I'd just stick with the simplest explanation. If someone comes here wondering about private modules, they can search for how to add private repos to Packagist, which will get them to Private Packagist and Satis and etc.

My initial 2 cents:

1) The documentation here is excellent. I understand what each* section of this is trying to do, and feel confident I could make adjustments within the examples for my site.

2) That said, keeping said documentation/examples in sync going forward between the two files sounds like a pretty tough maintenance burden. It'd be ideal if we handled this with _one_ file, e.g. "example.gitignore.txt" or something, like we do in e.g. sites/example.sites.php, that could serve as the basis for both. (I see there's a concrete reason for the discrepancy, which is that one's examples start with web/ and the other's doesn't, but I wonder if we could capture that in the docs and direct people to do the right thing, instead.)

3) I said each* ... the asterisk is that I don't understand what "add all of the scaffold file locations" means, and I'm not sure we should expect average site builders, especially those new to D8/Composer, to know what that means, either. It /seems/ to mean all of the "top-level" files/directories that are not /vendor. So it'd be nice if we said that (or some other similar instruction that's actually correct ;)).

4) Similarly, this also feels like a maintenance burden because I could see the list of top-level files changing and us forgetting to update these docs. I doubt it, but have to ask... is there any way to programmatically (e.g. automated test; an automatic-generated hook_help() page that we could point to?) ensure that the list here is always up to date with what's "live" in core?

Not bumping down to "needs work" since most of these are questions, though it would be nice to "English-ize" the scaffold piece a bit, unless you think this is a me problem vs. a general "site builders out of the loop on Composer stuff" problem. :)

2) It was suggested that we could scaffold these files into place. If we did that, though, we would have to add a templated scaffold files feature to the scaffold plugin. I was reluctant to add the complexity, but it is something we could consider.

3) The scaffold files are mostly at the top level, but there are some in sites and sites/default, for example. You are right that this list can change over time.

4) If the user .gitignores their vendor directory, and does not .gitignore all of the scaffold files, then the scaffold plugin will edit the user's .gitignore file and add ignores for any scaffold file that should be ignored. Maybe rather than trying to keep the scaffold file section up-to-date, we should instead just document this behavior and tell users what to do with the .gitignore changes added by the tool. (Commit them.)

Setting to 'needs work' for #42.4.

Created #3086819: Allow template substitutions in scaffold files to serve as either a follow-on or blocker.

Do we want to ask every user of 8.8.0 that starts a project with Composer to do that chore (well just about, who doesn't use git ;)), or can we ship the static files and create a follow-up?

If the gitignore gets generated based on the template that the user starts out with, we wouldn't need to ask every user though, we would only need to start getting interactive if we decide to do approach #1 in the IS.

For #3086819: Allow template substitutions in scaffold files I would say that's a (REALLY) nice to have follow-up. I do think we still need to reduce this patch to only one "example" file that gets copy/pasted, even if it has stuff the site builder needs to reconcile like [webroot]/path, though I'm happy for another committer to say I'm wrong about that.

#3086819: Allow template substitutions in scaffold files is green, so it could use docs, community reviews, etc.

Here's a patch addressing #47, assuming that #3086819: Allow template substitutions in scaffold files is committed.

I had an idea that would allow us to use /[web-root]/core/ instead of /[web-root]core/, but it would require another update to #3086819. Maybe that could be done as a bugfix follow-up. If these do not make it in for 8.8.0-alpha1, then I'll go ahead and make the improvement.

Here is a proposal to provide a scaffolded example.gitignore to replace the existing example.gitignore file. The theory is that this is documentation, and can be backported to the 8.8.x branch. On the 8.9.x branch, once #3086819: Allow template substitutions in scaffold files is committed, then the /[web-root]/ pattern in the file will be automatically renamed to / or /web/, as appropriate (or to some other value, as stipulated in the "locations" section of the drupal-scaffold metadata. This file is still scaffolded as "example.gitignore", as the user must examine it and decide what edits to make to match the strategy for their particular site.

Note also that we are deliberately changing example.gitignore here. It is a goal to ensure that the tarballs produced by the scaffold operation have high fidelity with the tarballs produced with the historic automation tools; however, I feel that it is okay to decide on a case-by-case basis to change the contents of existing files if new contents is more appropriate in Drupal 8.8.x / 8.9.x.

@greg.1.anderson so just to clarify, what you're proposing in #51 means that when someone creates a new Drupal site, regardless of which template they use, a .gitignore file wouldn't automatically be created but instead an example.gitignore file which they can modify then move/copy to .gitignore?

@AaronMcHale That is correct.

What are the interactions between this issue and #3086819: Allow template substitutions in scaffold files ?

If the latter gets in, does this one need to change? If this goes in, and then #3086819: Allow template substitutions in scaffold files is committed, do we come back and reopen this to leverage the new features? Im not sure what might be postponed on what.

If #3086819: Allow template substitutions in scaffold files goes in, we just need to come back here and edit some of the comments. This file will be correctly processed by that patch.

I didnt mean to actually set this RTBC before, because of my questions in #55, but they're answered, so now it is. Groovy.

Re #53 and #54: sounds good, thanks for clarifying :)

This issue needs an issue summary update so the latest state of the patch - once that's done the issue can go back to rtbc.

Just ran into this as I'm starting work on switching a number of projects to use and recommend the drupal/recommended-project template, but realized after running git add -A that it added all kinds of files that should not be in a git repo (regardless of the validity of committing all the vendored files).

The git output included a large number of warnings out of the box:

warning: adding embedded git repository: vendor/behat/mink-selenium2-driver
hint: You've added another git repository inside your current repository.
hint: Clones of the outer repository will not contain the contents of
hint: the embedded repository and will not know how to obtain it.
hint: If you meant to add a submodule, use:
hint: 
hint: 	git submodule add <url> vendor/behat/mink-selenium2-driver
hint: 
hint: If you added this path by mistake, you can remove it from the
hint: index with:
hint: 
hint: 	git rm --cached vendor/behat/mink-selenium2-driver
hint: 
hint: See "git help submodule" for more information.
warning: adding embedded git repository: vendor/behat/mink
gitwarning: adding embedded git repository: vendor/drupal/coder
warning: adding embedded git repository: vendor/drupal/core-composer-scaffold
warning: adding embedded git repository: vendor/drupal/core-project-message
 cwarning: CRLF will be replaced by LF in vendor/pear/archive_tar/tests/dirtraversal.phpt.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in vendor/squizlabs/php_codesniffer/src/Standards/Generic/Tests/Files/LineEndingsUnitTest.css.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in vendor/squizlabs/php_codesniffer/src/Standards/Generic/Tests/Files/LineEndingsUnitTest.inc.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in vendor/squizlabs/php_codesniffer/src/Standards/Generic/Tests/Files/LineEndingsUnitTest.js.
The file will have its original line endings in your working directory
warning: adding embedded git repository: web/core

Committing the vendor stuff, I'm not sure if there's a way around some of those warnings. But it's certainly not a pleasant experience.

I think Drupal core should be opinionated here, and I'd rather see all the vendored files excluded by default. They could be in a gitignore section that says "If you want to commit all your site's files to your git repository (warning: you should know what you're doing—this could introduce a security risk), you should comment out all the lines below."

The problem is, most (all?) devs I've worked with throw the kitchen sink at local development, meaning modules like devel, php, etc. are all enabled and present locally. If we have everything committed by default, we don't have the protection of those modules not being installed if someone installs only prod dependencies with Composer.

But either way, we definitely need to ship something (because it's even more important to gitignore the files dir, settings.php, etc., and right now they aren't!), so if that bikeshed is going to stir up another month's waiting on this, please disregard for now.

Following your link, it looks like you're using this line:

composer -n create-project drupal/recommended-project:^8.8@dev my-project

If you remove the @dev, you get the stable release of Drupal 8.8.0, and you also don't end up with embedded git repos. You can also add --prefer-dist to make it more explicit that you don't want repos:

composer -n --prefer-dist create-project drupal/recommended-project:^8.8 my-project

Then git add -A doesn't report any embedded repos.

If you just want the latest version, this also works:
composer create-project drupal/recommended-project my-project -n

@Mile23: Indeed, using prefer-dist and stable results in only one warning, which is much less intimidating:

warning: CRLF will be replaced by LF in vendor/pear/archive_tar/tests/dirtraversal.phpt.
The file will have its original line endings in your working directory

@ressa: True, however, even in documentation I feel a lot safer at least specifying a major version (e.g. ^8) because a lot of little dependent things are not yet tested against Drupal 9.

@geerlingguy: One thing that is not clear to a lot of folks is that the version you specify on the command line with composer create-project is not saved anywhere in your project. It is only used to select the initial version of Drupal to install.

Composer will use the version you specify on the command line to select some version of drupal/recommended-project. The version that it selects has its own composer.json file that includes drupal/core-recommended: ^8.8. It seems logical to most folks that if they use ~8.8.0 to install drupal/recommended-project, that future updates will continue to use that constraint, but that isn't the case; the ^8.8 from the template will be used, unless you change it yourself after the project is created.

It's kind of weird, but that's how Composer works.

Also, related to all of the git warnings, Composer will add a .git directory to any vendored dependency that it installs from source. It has to install dev dependencies from source, and the behat/mink components you reference above have not had a stable release in over three years. Folks from the community have been asking for a stable tag periodically, but it has not been provided.

Regarding #61... @geerlingguy was working off the change record, not the documentation. I updated the CR to reflect that there is more up-to-date documentation.

We still need to deal with what sort of .gitignore stuff we care about in this issue. :-)

I created a project over the weekend using this.
I was unaware that there was no .gitignore so I ended up adding not only the vendor files that are apparently contested but also site files, settings.php etc.

I think when it comes to something like .gitignore it's better to be more aggressive with the ignore if you look at the two failure modes when creating a project.

Failure mode 1 no .gitignore:
You end up with all the default/files, settings.php, and vendor.

Resolution for failure mode 1
Delete the files.
Adding those deletions to git
Adding the .gitignore

Failure mode 2 aggressive .gitignore:
You end up with no vendor files in git that you expected.

Resolution for failure mode 2
Modify .gitignore to stop ignoring vendor
Add those files

The only other comment is regarding modifying a scaffold file if you decide you want to keep the vendor files. However there are already conventions for having a modified scaffold file with composer.

Looking at these two options I think it's clear that having the more aggressive .gitignore makes sense as it's easier to add a missing folder than removing many files and directories you mistakenly added.

Another possible solution, but probably more work would be to have two composer create profiles, like we have with standard and minimal.

One has no gitignore, one has gitignore.

This is a long comment just to say that I used this over the weekend and it felt unfinished and not ready for use. I didn't realize there was a different school of thought on including vendor or not, but I think of the two options, needing to modify something to include something is less painful than removing something you inadvertently included.

#66 makes some good points.

Also, I'm not sure why .gitignore is a scaffold file. Why don't we just add two different .gitignore files, one to the recommended template and one to the legacy template? We could add a test to ensure they stay in sync. That would probably be better than keeping this blocked on the template substitutions issue. Also, making this a scaffold file isn't a good idea (even with template substitutions) because .gitignore is highly project-specific and usually modified. It wouldn't be desirable to to have a project's .gitignore be overwritten every time there is some minor update to the file contents in the assets file (or worse, prior to #3092563: Avoid overwriting .htaccess changes during scaffolding > security problem landing).

Yeah. We moved away from #35 after #41 in an attempt to use templating to keep the .gitignore's DRY, but the templating didn't go in.

So, I think we should in fact go back to #35. Let's compare and see if #50 made any substantive changes to what was ignored (I don't remember) and also update the issue summary and add a change record. Then we can review and move this back to RTBC.

Whoops, I think I spoke too soon in #69. There's another thing that is bothering me here.

Currently, if you download Drupal from drupal.org (e.g. version 8.8.1), then you will get (once you untar it) a Drupal site that has no .gitignore file, but that does have an example.gitignore file. If we committed #71, then there would be both an example.gitignore AND a .gitignore in the tarball downloads (because these are now generated from drupal/legacy-project). There has never been a .gitignore in the download files before. If we added one now, then folks who are still using drush pm-update to update their sites will find that each Drupal update will start overwriting their .gitignore file. Do we want this for Drupal 8.9.0? Do we want this for Drupal 9.0.0? Perhaps we should consider that the .gitignore should appear in the template projects, but should not appear in the downloads. At a minimum, we should decide what the aspirational behavior is.

We also have not addressed #41.2. At a minimum, I think we should have a test that detects variations in the template project .gitignore files with the example.gitignore file at the root / in the scaffold assets directory. (Note that it would be easier to write this test if the non-comment, non-blank lines of example.gitignore always exist in the project template .gitignore files.)

We should also take some time to discuss the implications of #66. The example.gitignore file is set up by default to ignore /vendor/. In a way this is an odd choice for sites that are usually going to be maintained with drush pm-update (perhaps the primary use-case for the download tarballs), and yet somehow this was the default recommendation. Back in 8.0.0, we had:

# Ignore core and vendor when managing dependencies with Composer.
# core
# vendor

In 8.1.x, this was changed to:

# Core's dependencies are managed with Composer.
vendor

In 8.8.x, we have no .gitignore at all in the template projects, but we still have the example.gitignore file, and it recommends ignoring /vendor/. While I recognize that there are valid use cases for committing vendor, I'm not sure that it's best to reverse the recommendation from the example gitignore file, which has recommended ignoring vendor since 8.1.1. Perhaps we did this because it would be easier for a user to remove comments (to convert commented-out vendor into an ignored vendor) than to have documentation on where to add the comment characters for users who want to commit vendor et. al.

I updated the issue summary with things that we still need to consider before calling this issue RTBC.

Part of me agrees with an aggressive approach, because as it was pointed out, it's harder to remove something that you already committed in that even if you remove it by commit it still exists in past commits, but it's trivial to commit something that was never committed in the first place.

That said, core doesn't ship with a settings.php, only a default.settings.php, settings.php is created during install. All I'm saying with that statement is that it's a precedent.

That's it, that's all the thoughts I have right now.

I was a little surprised to see that there was no "starter" .gitignore file included when I began using the drupal/recommended-project project template. If it is the easiest solution to build, can't we just put an example.gitignore in the root of both Composer installs as well as tarball downloads, which people can rename themselves to .gitignore if they want to use it? @greg.1.anderson writes:

Perhaps we should consider that the .gitignore should appear in the template projects, but should not appear in the downloads. At a minimum, we should decide what the aspirational behavior is.

I agree that probably only template projects needs a .gitignore, so leaving it out from the tarball is an option. I would think that not many users who drush pm-update their sites use Git, but I could be mistaken.

After reading the comment by @nicxvan in #66, I agree with having an aggressive, more exclusionary approach as default, yet also documenting the more inclusive way, like how to handle the vendor folder. Like geerlinggy writes in #60:

I think Drupal core should be opinionated here, and I'd rather see all the vendored files excluded by default. They could be in a gitignore section that says "If you want to commit all your site's files to your git repository (warning: you should know what you're doing—this could introduce a security risk), you should comment out all the lines below."

#74

It feels all that's been said is enough to have fleshed this out...
@ressa +1

I'd like to see example gitignore in the tarball plus recommended-project. What's the harm?

Can we ship this please.

In parallel to this issue there was also a PR in github about the same thing: https://github.com/drupal/recommended-project/pull/2#issuecomment-616630579 with several upvotes. Thanks to @drumm for pointing to this issue right here.

I am also in favor of an example file, like already discussed!

What is the community's thoughts on this enhancement to these excellent patches?

# Ignore files generated by IDE
/.idea/
/.vscode/


# Ignore Packages
*.7z
*.dmg
*.gz
*.bz2
*.iso
*.jar
*.rar
*.tar
*.zip
*.tgz

# Ignore logs and databases
*.log
*.sql

Hello,
Sorry for my extremely long absence (months). I decided to do some community work today as I've been having to work on Drupal 8 projects in the last few weeks.

#78
I don't think generic non-drupal stuff belongs in a project .gitignore.
that should be user centric on the user's global .gitignore_global file.

About the opposing schools of gitignore this largely depends on the project in my opinion.
For old unsupported Drupal 6-7 projects, or complex big projects, or projects that lack CI i might use the "commit all" approach" including vendor, or everything but vendor.
Then for everything else, including proper pipelined CI projects, small projects, projects without any obsolete or missing **disapeared from web** dependencies i will use the "commit only custom and composer.json" approach.

I think having proper wording in the .gitignore might help alleviate concerns.

I'm providing some input for consideration as i've just pushed for upstream changes at github/gitignore and gitignore.io:
- https://github.com/toptal/gitignore/pull/297
- https://github.com/github/gitignore/pull/3386
- https://github.com/toptal/gitignore/pull/296
- https://www.gitignore.io/
- https://www.gitignore.io/api/drupal

Feel free to comment if i should change any wording in those merge requests or paths in the proposed changes.

Perhaps if these are good enough or after fixing if needed they could be reused here?

I just created a Drupal 9 instance with Composer, and see an example.gitignore file is included, which is great. But it is located in the web folder, and also seems very limited in its scope ...

My user experience:

Because there were no .gitignore nor example.gitignore files in projects created by composer create-project drupal/recommended-project, I was copying .gitignore from older projects. By doing that I was adding all the scaffolded files under VCS. Like in the good old times 😅And of course I completely missed drupal/core-composer-scaffold existence and all of its cool features.

I just want to say that .gitignore should be definitely shipped with drupal/recommended-project 🙏

I added a note to Starting a Site Using Drupal Composer Project Templates mentioning the file example.gitignore in the web/ directory. Anyone who reads the docs should have an easier time than @ressa (Comment #81).

Thanks @benjifisher for adding that note mentioned in #84 to the info page.

My user experience:
I want to have the .gitignore in project root instead of the web root. So i was unsure if i just should cp the example.gitignore and keep it in the webroot directory (where the example file is located) or should move it to project root.
(I simply moved it to project root, but i wasnt sure if thats the "recommended default way" ).

Yeah #84 thank you for pointing this out, it's a little unclear if the example.gitignore is supposed to go inside the /web directory or in the repo root? The /vendor path only works if .gitignore is in the root level, in which case all the other paths need to be updated to include /web.

#85 I ended up using the example .gitignore inside the /web subdirectory, and creating a second `.gitignore` at the repo root containing /vendor.

I would also love to have this patch committed. Thanks to @benjifisher (#84) for adding the documentation

I would also love to have this patch committed.

I agree @zuhair_ak, it would be great, but a few Remaining Tasks were added in January 2020 (see @greg.1.anderson's comment) which we probably need to decide on, before this can move on.

I shared my point of view in #74, and those who want to improve the chances of moving this issue along and include an example .gitignore file in the root of a fresh Drupal installation should feel free to share their opinion on the Remaining Tasks.

I would like to see both project root and web root files, neither of which would be literal .gitignore files. Rename the existing example.gitignore file to something like example.webroot.gitignore and add example.project.gitignore, perhaps, to make it unambiguous. This is the model for eg settings.local.php.

I'd also suggest ignoring vendor by default, as well as files created from scaffolding assets. The suggestions along those lines in this article by @james.williams make sense to me.

I ran into this recently while noticing that the Drupal 9 example.gitignore file will only work if in the project root. Since I create Drupal sites via Ansible currently, I'm copying my own customized version of that into the project root. I still like the prospect of two example .gitignore files, one geared for web root, and one geared for project root. I'd probably have Ansible copy the project root one and have Ansible make any changes I'd want.

It's so sad that this issue has stalled.

I just started a new Drupal 10 project, and using the .gitignore file from the MR here worked beautifully.

I really appreciate all the hard work which went into getting it. Not only are the rules very precise, so that only relevant files are committed, the comments are also extremely helpful and enlightening. For example in helping to choose whether to include custom modules and themes as separate repositories, or simply commit them directly to the repository of the site where they are used.

This is the kind of information which is invaluable for a new Drupal user, helping them get started well, on their Drupal journey.

I hope this issue can be restarted, so new Drupal users can have a .gitignore file included in freshly created projects before too long. It will save them much time and trouble down the road.

Bringing this back to life @ DC Pittsburgh. Learned way too much about the Drupal composer project templates. Some brilliant stuff for sure.
First, I wanted to share this nifty command I came across to make testing this easier (ie: create-project from a local directory)
composer create-project --prefer-dist drupal/recommended-project="@dev" 3082958-test-`date +%s` --repository '{"type": "path", "url": "drupal-3082958/composer/Template/RecommendedProject", "options": {"symlink": false}}'
source

I tested MR1627 using the above method with both legacy and recommended project and they worked well. I also think the approach of a .gitignore file with commented lines makes the most sense.
My initial hesitation with this was in the multiple gitignore files. But looking back at #3082958 it makes sense that we would have different gitignore files for each project template. The rules for those are pretty tightly coupled to the template they come from.
We have the example.gitignore file that gets scaffolded in from drupal/core, but that only has rules for sites/default/… I played with the idea of putting everything in example.gitignore and scaffolding it to project-root instead of web-root, but I think that will make the file overwhelming as noted in 3082958.
Now that we have a few years of composer deployments under our belts I think we can make these changes with a little more confidence. I +1 the approach in MR 1627. Using this approach has the added benefit of being compatible with projects like acquia/drupal-recommended-project who also provide a gitignore file via composer template (and not via scaffold)
In terms of remaining tasks I think we might want to consider opening another ticket to deal with the non-composer gitignore files. If we keep this ticket scoped to the composer templates we can probably move this along

Thanks for the thorough review @crasx. I recently started a Drupal 10 project, and the example .gitignore worked really well (comments in #96).

Getting this committed will help users with limited Drupal and Git experience start out on the right foot, and be a step in the right direction of attracting more Drupal site builders and increase Drupal's potential user base, as proposed by Dries in his blog post Drupal is for ambitious site builders.

Another relevant ambitious site builder issue would be getting a README included in the Recommended project repository: #3144211: Add README.md to composer project templates. Maybe that issue can get some attention at Pittsburgh2023 as well? :)

Can the MR be updated for 11.x please.

I tried re-basing via the Gitlab GUI (as someone else also recently did) which resulted in disaster, and I am getting Failed to create branch '3082958-add-gitignores-11x': invalid reference name '11.x' from Gitlab when I try to create a new branch ...

So I am attaching a good old patch, what is essentially #70 again.

Thanks!

Fantastic, thanks @smustgrave and @crasx for moving this issue closer to completion!

Is the project root .gitignore file meant to replace the webroot example.gitignore? The project root file looks to me like it does everything the webroot file does except ignore /vendor/, which doesn't initially exist in the webroot anyway.

@brad.bulger it is not. The example.gitignore in Drupal root is used when a user downloads Drupal as a zip, the others are used during composer create-project. Do we maybe want to add a post composer create-project hook to remove example.gitignore?

Also linking #3294241 which removes the legacy project.

I see. I'd think comments in the files saying something like what you just said would suffice.

Updating status, planning on circling back to this soon

Added some docs to the top of gitignores

Thanks @crasx, it's more precise now.

It looks like core/assets/scaffold/files/example.gitignore also needs to be updated. I also see now that Composer is not capitalized a few places, and there was a +80 character line, so here is a re-roll.

Tests are green and change looks good still.

I am doing triage on the core RTBC queue.

1) The Issue Summary has a proposed resolution but there are 4 remaining tasks. In reading the comments I see that in #89 @ressa, pointed out that they need to be decided on. From #89 onwards I don't see discussion of those 4 points. Have those items been resolved and I missed it? Was it done in a Slack discussion?

2) There is a change record here which has not been touched since Jan 2020. That should be reviewed for accuracy and to update the branch/version numbers. Adding tag and setting to NW for this.

3) I applied the patch and read the comments. I thought the new paragraph in the 2 example.gitignore files could be easy to read if it were in plain English. I have made a patch as a suggestion. I am not running tests, that can be done if there is agreement to keep the changes.

I just remembered that because this is changing a file managed by the site, it should have a release note snippet. Correct me if I am wrong.

I've added that heading to the issue summary.

Thanks @quietone.

1) The Issue Summary has a proposed resolution but there are 4 remaining tasks. In reading the comments I see that in #89 @ressa, pointed out that they need to be decided on. From #89 onwards I don't see discussion of those 4 points. Have those items been resolved and I missed it? Was it done in a Slack discussion?

About deciding the structure (the first three items) we probably need @mbaynton, @greg.1.anderson, @Mile23, and @Mixologic to chime in, and make a final decision.

About committing /vendor and related files by default or not (the fourth item), since #60 by @geerlingguy December 2019, the consensus in the comments is to ignore /vendor, and these users have stated that they prefer to not include /vendor:

• @geerlingguy
• @nicxvan
• @ressa
• @tyeth
• @lpalgarvio
• @Leksat (I think ?)
• @brad.bulger

Currently, the patch includes /vendor, so as an example, I am attaching a patch which reverses this, and excludes /vendor, and also directories generated by Composer and scaffold files. What do you all think about that?

Ambitious Site Builders use Git, and it would be nice if they had a great .gitignore file in their new Drupal 10 projects.

Not sure who could make the call about those remaining tasks. May be worth posting in #contribute on slack?

I don't use Slack, but If you could nudge it along by mentioning it in that channel, that would be fantastic.

+++ b/composer/Template/RecommendedProject/.gitignore
@@ -0,0 +1,113 @@
+# When a development team creates one or more custom modules that
+# are intended for use on more than one site, the typical strategy
+# is to register them in Packagist and give them the type
+# `drupal-custom-module` instead of `drupal-module`. This will cause
+# Composer to install them to the directory `modules/custom`.
+#

That seems rather specialized to go in uncommented.

On all project I've worked on, web/modules/custom is committed to the repo.

If we have our own modules in a Composer Packagist, they might go in web/modules/COMPANY_NAME for example.

That makes sense @joachim, do you have a suggestion on how to reword it?

In the meantime, here is a re-roll, to match changes in file locations, and extension.

The Needs Review Queue Bot tested this issue.

While you are making the above changes, we recommend that you convert this patch to a merge request. Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

I tried this and failed (see #112). Should we stick to an old school patch?

Tagging for framework manager to hopefully get this moving.

I'm moving the patch to an MR

@ressa I was able to convert your patch to an MR.

Hiding outdated patches.

Perfect, thanks @nicxvan :)

+++ b/composer/Template/LegacyProject/.gitignore
@@ -0,0 +1,114 @@
+# Ignore directories generated by Composer
+#
+# See the "installer-paths" section in the top-level composer.json
+# file.
+# ------------------------------------------------------------------
+/drush/Commands/contrib/
+/composer/
+/core/
+/modules/contrib/
+/themes/contrib/
+/profiles/contrib/
+/libraries/
+
+# Generally you should only ignore the root vendor directory. It's important
+# that core/assets/vendor and any other vendor directories within contrib or
+# custom module, theme, etc., are not ignored unless you purposely do so.
+/vendor/

I'm a little unclear on this. Wouldn't core/assets/vendor always be ignored if /core/ is ignored, as suggested in the previous section? Same with contrib directories.

It was added in comment #50 by @greg.1.anderson, and taken from the example.gitignore file added in 2018, so perhaps it can be handled in a follow up issue?

Maybe a framework manager can give it a look over? It would be great if this improvement got committed in 2024.

Learned something new.

So for the framework manager to take a look can the pros/cons on each remaining task be listed in the summary vs needing to read 130+

+1 on #120. We should not be instructing people to make new packagist entries for their custom modules, and we should not ignore the custom modules directory.

We also need to summarize decisions here in the IS, so a maintainer can review it easily.

I agree, the default .gitignore file should be aimed at the broader audience, and managing modules via Packagist is rare.

Issue summary is still needed though.

The fact that there is no example.gitignore in the project root when using the drupal/recommended-project Composer template has always struck me as very odd, especially considering that one exists in the Drupal docroot.

I've been complaining about this for a few years now (mainly while muttering to myself) and unfortunately for me (heh), with a little help I was able to find this issue. Seeing how close it is to the finish line, I'd love to be a part of getting it done.

I just reviewed the existing issue summary and the last 70-ish comments. FWIW, my feelings are as follows:

1. I don't see any need for a project-level example.gitignore to be part of the Drupal core tarball. I really think it is important for one to be included as part of drupal/recommended-project. The fact that we've named it "recommended-project" makes it a bit opinionated, no? So, there's no reason why we shouldn't have an example.gitignore that recommends that Composer dependencies are ignored by Git.

2. I think a more aggressive approach (as @nickvan detailed in comment 66) is the way to go.

3. As I just alluded in my first thought above, I think it should be an example.gitignore and not a .gitignore file. I'm a bit confused though because the existing MR [!5622] has a title of "Add example gitignores", but it actually adding two .gitignore (not example.gitignore) files. Is this just a case where the title is old/incorrect? (related to the first "remaining task" listed in the issue summary)

Regarding the second "remaining task" (.gitignore in tarballs) in the issue summary, is this really something we still need to consider four years later? I think the whole point to this issue is to help folks newer to Drupal who are using drupal/recommended-project (or similar) to set up their code base properly. I think that if you're cloning Drupal core (or downloading the tarball - {{shudder}}), you probably don't need any help with setting up a .gitignore file.

Regarding the third remaining task (strategy to exclude .gitignore in downloads), I don't see any discussion about this in previous comments. Is this still an issue? If we decide to use example.gitignore instead of .gitignore, how does that impact this task?

Regarding the fourth remaining task (vendor committed by default), I think this is the easiest to decide - we should ignore the vendor directory. The fact that the current MR does this makes me think that this task is obsolete.

Finally, my only comment on the existing MR is that if we're including a rule to ignore PhpStorm files, then we should probably do the same for Visual Studio Code files, as the most recent Drupal Local Development Tools survey shows 55.4% of respondents using PhpStorm and 36.9% using Visual Studio Code - not an insignificant amount.

Let's figure out the next steps we need to complete in order to get this ready for review. It seems that updating the issue summary's remaining tasks is a priority, so any guidance that anyone can provide on that front seems like it would be most helpful.

thanks,
-mike

Thanks for nudging this issue along @ultimike, let's get it moving :) About your comments

I think a more aggressive approach (as @nickvan detailed in comment 66) is the way to go.

I agree, and that's how it is now, after #116, where I summarized the general opinion (aggressively ignore) and created a patch accordingly.

As I just alluded in my first thought above, I think it should be an example.gitignore and not a .gitignore file.

The Issue Summary has this sentence, which I agree with:

Have one well-documented .gitignore per project template that meets persona a)'s needs without modification, and has the additional rules desired by persona b) commented out. [...]

I would prefer to not clutter the folder with an example.gitignore file, but prefer a hidden .gitignore file, as it is in the current patch.

About PHPStorm and VSCode, I agree -- let's add ignore rule for Visual Studio Code files as well.

About the two example.gitignore files in the current patch, I have no opinion. Perhaps @mbaynton, @Mixologic, or @greg.1.anderson can weigh in, if they are necessary, or can be removed?

My main goal is to get an aggressively ignoring .gitignore in the project root of drupal/recommended-project, out of the box.

I decided to try out this gitignore for recommended-project in a recent project and am pleased to see it works well.

Something though I found odd, the .env line is commented out, I don’t think we should comment this out. I can’t think of a single case where you would ever want to commit the .env file, so if you have one you’re pretty much always going to be uncommenting that line. Or worse, forget to uncomment it and accidentally commit your .env file, which could be a real problem!

Let’s do ourselves a favour and have the .env always ignored by default. I think that’s in the spirit of this being a recommended setup.

tl;dr - I vote for an example.gitiginore file instead of .gitignore !

I reckon the ignore file would be customised on many (perhaps most?) projects, which would require tricks to get around the composer scaffolding, which can be tricky and frustrating if & when updates are made to the original version of the file in future. It would become an extra thing for core release managers to keep track of changes to, just as they currently have to explicitly state when files like .htaccess are changed. (I suspect putting more responsibility on them is unhelpful & makes the Drupal project less sustainable.)
I do not see the .gitignore file as a 'functional' file, it's much more of a development / project-management file - so more like settings.php, which we have an example.settings.php version for too.

But regardless of how any one of us sees this file, the very fact that there are opposing schools of thought - as evidenced by the original report in this issue's summary, and the issue this one came out of - makes me think that we really shouldn't be imposing an approach too much, but instead providing an easy place to start from. For me, that means an aggressive example.gitignore, rather than confusing users with a .gitignore file that will easily lead users to have partially-built codebases that would not be functional. An incomplete codebase would also encourage running composer on production, which I don't believe is a good idea either.

---

Meanwhile, let's be careful about getting confused about what drupal/recommended-project actually is. It is not supposed to be a opinionated recommendation of a default codebase. That's what distributions are/were. Instead, it is simply about setting up the codebase with the recommended directory structure, with a relocated document root, and with versions of dependencies that are 'recommended' in the sense that they have been tested with Drupal core. I suggest that we carefully restrict this issue's goals to the scope of the two personas in the issue summary rather than getting too excited about each of our own opinions ;) Even if ideas like excluding .env files etc might be good! I just mean let's ensure the motivations that drive decisions in this issue align with what the packages exist for.

Thanks @AaronMcHale, great suggestion adding .env by default, I have updated the patch.

@james.williams: I think I missed the argument before, that using example.gitignore eases the burden on the core release managers, or didn't understand it. But it's a strong argument, so I have renamed the two .gitignore files files to example.gitignore.

I still think that having an actual .gitignore is more important than an example.

If there is a .gitignore and you don't want one then the resolution is a lot simpler than if there is no .gitignore and you expected one.
I also think the use case where a .gitignore is expected is a lot higher in the Drupal community, but I may be wrong.

As a developer using composer I expect vendor, core, and contrib to be ignored. When starting a new project and those get committed it is something I have to resolve. If I am using a host with no build process and need to commit those files the resolution is a lot simpler, I just delete or comment out the default gitignore.

For my use case having an example.gitignore does not solve the problem that brought me to this issue in the first place.

Also @ressa I'm curious why this is back in a patch rather than the Merge request?

Sorry @nicxvan, I forgot that you managed to create an MR in Gitlab, which I have updated.

I agree, .gitignore would be easier out-of-the box, so I'll leave it at that again, and not convert to example.gitignore. So this commit only adds ignore rule by default for PhpStorm and VSCode folders, and .env file

Core maintainers should be allowed to veto .gitignore-files, if it makes maintenance very hard. But it would be great with some feedback from them, to clarify if this assumption is true or not.

@james.williams:

I reckon the ignore file would be customised on many (perhaps most?) projects, which would require tricks to get around the composer scaffolding, which can be tricky and frustrating if & when updates are made to the original version of the file in future.

As I understand it, when composer create-project drupal/recommended-project is run, the composer/Template/RecommendedProject/.gitignore file will be copied to the root of the project. After that, it's left to its own devices, and Composer will not update it again.

Heh - seems like we're playing ping-pong with example.gitignore vs. .gitignore...

I really appreciate everyone's speedy comments in the past few days - while that's a great sign of a healthy debate about a feature we all want to make happen, it also could lead us to the dark side (bikeshedding).

From @ressa's (thanks for keeping the MR updated, by the way) previous comment:

As I understand it, when composer create-project drupal/recommended-project is run, the composer/Template/RecommendedProject/.gitignore file will be copied to the root of the project. After that, it's left to its own devices, and Composer will not update it again.

Actually, I think that if we go with .gitignore, and then it is customized, the next time drupal-core-scaffold runs, it will overwrite the changes - just like what happens if robots.txt or .htaccess is customized. This, IMHO, is a solid reason why I think using example.gitignore is the way to go.

If we name it .gitignore, why don't we just use sites/default/settings.local.php instead of sites/example.settings.local.php (amongst other examples - pun intended)?

Back to the 4 remaining tasks in the issues summary:

1. Nothing to be added to the tarball.
2. It seems like we have consensus on a more aggressive approach.
3. To .gitignore or example.gitignore - undecided.
4. Vendor directory should be ignored.

Am I missing anything?

-mike

Thanks for the fast reply @ultimike, I do think that the current debate is healthy, and we are making progress, and feel we are pretty close now. I much prefer discussion to silence :)

About "3. To .gitignore or example.gitignore - undecided.":

Actually, I think that if we go with .gitignore, and then it is customized, the next time drupal-core-scaffold runs, it will overwrite the changes - just like what happens if robots.txt or .htaccess is customized.

That does seem to be the case ... But it looks like .gitignore can be excluded from any further updates during scaffolding, by setting the file-mapping value to false. See Drupal's Composer Scaffold > Excluding scaffold files.

I have tried to add this to the MR.

To experiment with excluding a file from getting updated in a standard drupal/recommended-project installation with an existing scaffolded file, try adding the below snippet, edit the /web/.htaccess file, and update with or without it being excluded. The file is overwritten with default set up, but excluded from updates when file-mapping is set to false.

"file-mapping": {
    "[web-root]/.htaccess": false
}

It would be great to be able to actually test the patch with Composer commands locally, and have it do the scaffolding. Tricks for using Composer in local development exists, but is only for contrib modules, so far.

I agree that not updating gitignore after the initial scaffold is ideal as well.

I'm in favor of .gitignore instead of example.gitignore and excluding it from scaffold updates.

@ressa - thanks for keeping the MR up-to-date.

It makes sense that with

"file-mapping": {
    "[web-root]/.htaccess": false
}

The .gitignore file that is being added to the drupal/recommended-project template will not get overridden each time the drupal/core-composer-scaffold plugin runs. This will ensure that if/when the .gitignore is modified, those changes are not overwritten the next time drupal/core-composer-scaffold runs.

While going with example.gitignore is my preferred solution, it is not a hill I am willing to die on and can (quite comfortably) live with this MR providing a .gitignore file by default.

With this being said, I think we can remove all of the remaining tasks, no?

• Decide if we want both .gitignore and example.gitignore in the project template files.
• Decide if we want to add .gitignore to the download tarballs (perhaps not?)
• Decide on a strategy on how to exclude .gitignore in downloads but not in the project templates, if that's what we decide.
• Decide if we want to stick with the proposed resolution (vendor committed by default), or reverse to match existing example gitignore file (which by default ignores vendor). The consensus is to ignore vendor

If so, we can do that and then update the issue summary with more up-to-date info regarding the contents of the MR. Am I missing anything?

-mike

You're welcome @ultimike, and I agree that the four remaining tasks have been resolved, so the Issue Summary can be updated.

Would overwrite be a better option for the .gitignore file?
https://www.drupal.org/docs/develop/using-composer/using-drupals-compose...

That could work better, thanks for the suggestion @joestewart. Feel free to update the MR using that method instead.

I am not even sure the current solution works, since testing Composer scaffolding locally is not simple, so I haven't actually had opportunity to run it ... (see last paragraph in #148, "It would be great [...]")

I updated the patch to this, perhaps it could work?

".gitignore": {
    "path": "composer/Template/RecommendedProject/.gitignore",
    "mode": "replace",
    "overwrite": false
}

I've updated the issue summary and have been talking to some folks in the Drupal Slack workspace about this issue in hopes of getting more voices in this thread.

I've also hidden previous patch files - let's stick with the existing MR for this issue.

Of particular note, I'm really curious to figure out if/how we need to write a test for this change.

From @crasx's comment 98 above, I'd love to manually test this using the method he described, but I can't decipher exactly what I need to do to make that happen. Can someone post step-by-step instructions?

thanks,
-mike

Removing that tag

But MR appears to have a spell error.

And also would love to see this land (hopefully by D11)

It looks like #3401988: Spell-checking job fails with "Argument list too long" when too many files are changed.

Perhaps someone can do the Git steps in #11 or #37, if that's what it takes?

May just need a rebase

I'll do it

That seems to have worked!

lot of comments to digest but this does appear to be working when used. Maybe any additional additions can be follow ups

Thanks, @nicxvan and @smustgrave - do either of you feel that a test of some sort will be necessary for this to get merged?

-mike

Also, can someone test with Composer commands locally and have it do the scaffolding, to check if the .gitignore is overwritten, or not? Or perhaps that has already been confirmed? I added it in the Issue Summary as a remaining task.

I don't understand what composer/Template/LegacyProject/.gitignore is for or where it would be used.

Also, I don't see how the path setting works in the recommended template, because the composer directory isn't present in scaffolded installs.

Great questions @longwave, perhaps the LegacyProject file can be removed from the patch?

It's a blocker for this issue, that we can't test this Composer template patch locally, to check if the scaffolding works.

@mbaynton kindly shared a method in #19, but things have changed since 2019, so perhaps someone can share an updated method? I have added two tasks in the Issue Summary:

• Document how to test Composer templates locally under Tricks for using Composer in local development
• Check locally with Composer if the scaffolding: 1. Creates a .gitignore, 2. Which is not overwritten/updated
  [...]

I will see if I can come up with testing instructions. But before I forget ...

There is some discussion of whether to call the file .gitignore or example.gitignore., and the current MR uses .gitignore. The problem with that is it can have unexpected effects because Git will respect what it says in any subdirectory of composer/Template/LegacyProject/ or composer/Template/RecommendedProject/.

TL;DR: I think we should go with example.gitignore.

I think my previous comment duplicates the start of the issue summary:

The composer-ready project templates (drupal/recommended-project) nearly shipped with a .gitignore that would have caused all the composer-managed files to be excluded from any git repository the owner of the new project might set up.

@longwave asked on the MR:

How does this path work?

In fact, the current approach does not work.

It is reasonable to guess that the drupal/recommended-project metapackage is generated from composer/Template/RecommendedProject/ by cloning that directory, since currently that means just the one file composer.json. If that were the case, then adding .gitignore in that directory would add it to the metapackage.

In fact, the metapackage is generated using the code in composer/Generator/. Looking at the code there, I think it creates a metapackage with just the one file composer.json, just as it says in composer/Metapackage/README.txt:

A metapackage is a Composer package that contains only a composer.json, and
has no other content. In other words, the purpose of a metapackage is to
provide dependencies, not to provide code or data.

I think that what will work is

1. Add a file, say recommended-project.gitignore, to core/assets/scaffold/files/.
2. Reference that file in composer/Template/RecommendedProject/composer.json.

Back to NW.

From Comment #162:

... this does appear to be working when used.

@smustgrave, can you describe your testing steps?

On second thought, referencing the gitignore file from composer/Template/RecommendedProject/composer.json does not work. The code in composer/Generator/Builder/DrupalCoreRecommendedBuilder.php builds composer/Metapackage/CoreRecommended/composer.json from composer.lock.

What does work is adding the extra/drupal-scaffold section to the array in DrupalCoreRecommendedBuilder::initialPackageMetadata().

Testing instructions

For manual testing, execute

composer run-script post-update-cmd

and confirm that composer/Metapackage/CoreRecommended/composer.json is updated as intended.

If someone can confirm this, then we should add these instructions to the issue summary.

What if we called it template.gitignore?
Is there a way to have the scaffold change the name from template.gitignore to .gitignore?
The objection to using example.gitignore is that it will not by default ignore the files that a CI approach would want ignored whereas having a gitignore you don't want is much easier to resolve.

Example implies suggestion where template implies that something is meant as a starter I think.

BTW, the other drupal-scaffold configuration in composer/Metapackage/CoreRecommended/composer.json comes from core/composer.json.

Just to clarify what I think the requirements have become.

After setting the initial .gitignore it should not be overwritten if the user updates the gitignore.
The template .gitignore should not add gitignore rules to where the template is stored i.e. it should not be called .gitignore

An actual gitignore file was settled on because it is less destructive to have a gitignore and not want one than to not have a gitignore and want one.

I agree that the end result should be that a .gitignore file (of that name) is placed at the root of the project after running the composer create-project command; And that existing project should, ideally, not be impacted.

Whether we call it template.gitignore or something different in our source control, I don't feel strongly, assuming we have a mechanism to place it at the project root with a different name.

@benjifisher - thank so much for your work on this issue. I know it has been a few weeks since we discussed this at MidCamp - I appreciate your patience with me finally having a moment to take a look at this.

@nicxvan, @AaronMcHale - appreciate your input in keeping this discussion going.

To summarize what I believe @benjifisher is suggesting:

1. From comment 168 above, add a new recommended-project.gitignore (it can't be example.gitignore because that file already exists for web/example.gitignore for drupal/recommended-project), to core/assets/scaffold/files/.
2. From comment 169 above, add a new extras section to DrupalCoreRecommendedBuilder::initialPackageMetadata():

protected function initialPackageMetadata() {
    return [
      "name" => "drupal/core-recommended",
      "type" => "metapackage",
      "description" => "Core and its dependencies with known-compatible minor versions. Require this project INSTEAD OF drupal/core.",
      "license" => "GPL-2.0-or-later",
      "conflict" => [
        "webflo/drupal-core-strict" => "*",
      ],
      "require" => [
        "drupal/core" => Composer::drupalVersionBranch(),
      ],
      "extra": {
        "drupal-scaffold": {
          "file-mapping": {
            "[project-root]/.gitignore": "assets/scaffold/files/recommended-project.gitignore"
          }
        }
      }
    ];
  }

If I understand all this correctly, we are no longer involving LegacyProject at all, nor do we need to, correct?

Am I understanding this correctly?

thanks,
-mike

Thanks for summarizing that @ultimike, based on what you're saying there I think that all makes sense, and it looks like the end result will be a .gitignore file in the project root, so I am happy :)

To prevent this issue from stalling, it would be amazing if someone with the right skills could check locally with Composer if the scaffolding in the patch:

1. Creates a .gitignore
2. Which is not overwritten/updated

Drupal 11 is planned to be released the week of July 29, 2024. It would be great to include this improvement.

Drupal CMS may finally push this forward :)

Adding database export files such as db.sql and db.sql.gz, for example created by Drush or DDEV.

Based on this comment I am postponing this on #2381091: Improve example.gitignore .

I hope that solving the other issue may help complete this "active .gitignore by default"-issue eventually (fingers crossed) since it will result in an amazing and complete .gitignore file, active by default, preventing an accidental commit of for example /vendor folder.

It was created in 2019, has 80 subscribers, and seen many comments and commits. So in a sense, the community voted by activity that this is the .gitignore solution to pursue. I feel it just needs the final push, to get it over the finishing line, and then it will have an enduring and positive effect, helping new users.