[policy and patch] Determine what should go in CHANGELOG.txt for 8.x core releases

Fixing title, changing component, and assigning to xjm to make sure she sees this.

Too many changes in the CHANGELOG.txt IMHO would not really give a huge advancement IMHO:

• There will be pointless patch rerolls
• Actual important changes will be overruled by the amount of small bits

Maybe though we should not only have change records for new APIs/ changed APIs, but maybe also something like feature change records.
This is the new feature, you might want to check out. Changelog.txt could then link to the approriate listing on drupal.org

Right now CHANGELOG.txt entries aren't being added at all (i.e. http://cgit.drupalcode.org/drupal/tree/core/CHANGELOG.txt?h=8.0.2 does not have entries for either 8.0.1 or 8.0.2). It seems like they should be going forward, even if not much gets mentioned in them?

I would say that all changes should be listed in CHANGELOG.txt, as they were with the 7.x branch,

That's not what I do for 7.x. I only add ones that are notable/important or that could affect people on upgrade (but with a very, very liberal definition of those two things).

There will be pointless patch rerolls

People usually don't include CHANGELOG.txt entries in patches though. For Drupal 7 I just add them to CHANGELOG.txt at the time of commit (could wait until release and go back and do it then, but in practice I think it's much easier to do while committing).

This is the new feature, you might want to check out. Changelog.txt could then link to the approriate listing on drupal.org

If there is a change record associated with the commit, that's a guarantee that it belongs in CHANGELOG.txt (and yes, the CHANGELOG.txt entry should link to the change record).

---

Since 8.0.x is only getting bugfixes, it seems to me like the CHANGELOG.txt for those point releases should list the most important bugfixes that went in, plus any where the change could cause an issue for people on upgrade (but in theory those should be rare).

For 8.1.0, 8.2.0, etc (minor releases) it becomes a little more interesting. Would something like #2646280: Remove IE pre-check and post-check Cache-control headers get mentioned in CHANGELOG.txt (unlikely to break anything, but in theory a small chance of breaking someone's automated tests), or should it not bother because it would risk making CHANGELOG.txt too long?

I agree with #4.

At this point I'd just point CHANGELOG.txt to https://www.drupal.org/list-changes/drupal

We could also have a page on d.o that always redirects to the most recent release node for a branch, and link to that as well.

Thanks @jhodgdon. I thought about this when I tagged 8.0.2 and decided that I don't think semver patch-level changes belong in CHANGELOG.txt at all. By definition there should not be any new features, BC breaks, or disruptive changes. With the sheer volume of issues still going into D8 compared to ever before in Drupal, maintaining a list of even "major" bugfixes seems unworkable for the overhead of that alone, plus it's redundant with VCS and the change records. If even minor changes were included in CHANGELOG.txt, there would be thousands upon thousands of lines under the Drupal 8 header already. :P That's not useful to anyone.

I do think we should update the CHANGELOG.txt to reference the resources for the full history (CRs and the commit history in VCS). It's also reasonable to continue to add to the 8.1.0 CHANGELOG as we did for 8.0.0.

This also seems like something we should run by Dries since the file contains essentially Drupal's whole history.

Going to retitle to something less opinionated about the solution. :)

Here's a patch with links.

Then I'd suggest we'd just update our documentation to state that CHANGELOG.txt contains a list of new features and improvements for major and minor releases, since the release note link (already a URL redirect we add to D8 releases) will always contain the important highlights for the given patch release as well as the full commit list.

Adding an issue summary.

Between our Git history, https://www.drupal.org/list-changes/drupal, and our release notes, I think we can safely remove CHANGELOG.txt. It's an outdated concept.

Having said that, it would be nice to have a page on drupal.org that documents when major concepts or features where introduced. For example, it's nice to be able to look up that we introduced i18n in 2001 with the release of Drupal 2.

I agree removing it would be better than keeping it in place but not maintaining it.

However here is the alternative perspective of why a CHANGELOG is important: http://wptavern.com/keep-a-changelog-project-aims-to-standardize-best-pr...

One way to think about it is to take a step outside Drupal. Imagine (perhaps this won't require much imagination) that you have a website with some random external plugin (like a WYSIWYG library or a jQuery carousel) and one day you find you need to update the library to a newer version. It turns out the version currently on the site is two years and a dozen releases out of date. The code changes are completely incomprehensible to you. So what you do (really the only thing you can do) is skim the CHANGELOG.txt file to see what changed in the last dozen releases that sounds like it might cause problems or be useful for the specific way your site is using the library.

While it may be technically possible to get the same information by going to an unfamiliar website and browsing through dozens of pages there, the reality is that the changelog file is the simple, standard way to do it.

I think for a lot of non-Drupal-experts updating a Drupal website itself, the above scenario is exactly analogous to the situation they find themselves in.

But it's never going to be comprehensive. git blame is comprehensive.

Again, I disagree that the CHANGELOG.txt is "not maintained". I just think that entering patch-level bugfixes would make it unusable. There are about 100 commits in every D8 patch release right now.

http://wptavern.com/keep-a-changelog-project-aims-to-standardize-best-pr... refers to "messy git logs". Our git logs are not at all messy. And a VCS is actually a much better way than reading a text file to debug when something changed, if that's the developer's goal.

https://wordpress.org/news/2009/07/improving-your-plugin-changelogs/ (linked from that post) says:

We feel that all software should have a changelog that details, at a high level, what changes have been made in each version so that the user can make an informed decision about when to upgrade and how much testing they should do with their site.

Patch-level bugfixes are not "a high level".

However, it also appears that that is only advice for plugin developers -- WordPress core does not appear to include a CHANGELOG that I could find. Nor does django. Joomla just has a line pointing to their VCS log on github.

Docker actually does have a human-curated one, grouped by topic:
https://github.com/docker/docker/blob/master/CHANGELOG.md

Symfony appears to have a non-human-curated log of all commits not labeled "minor":
https://github.com/symfony/symfony/blob/master/CHANGELOG-3.0.md
https://github.com/symfony/symfony/compare/v3.0.0...v3.0.1

The options we have are:

1. Do not maintain a CHANGELOG. Keep a stub file pointing people to our VCS and release history.
2. Maintain a CHANGELOG only for majors and minors, requiring someone to go though and decide what features merit mention.
3. Maintain a CHANGELOG for patch releases of only criticals and issues tagged for release notes mention, and automate it.
4. Require the release manager to read through 100 commits and decide which are worth mentioning for each patch release.
5. Dump the entire release notes from drush rn or the git log in the CHANGELOG.txt.

For me, 1-3 are the only acceptable solutions. 4 is an unacceptable amount of work, especially for D8, and 5 will create an unusable text file that is... I checked... already 300 lines long and 32K of text since 8.0.0 alone.

The only way I could CHANGELOG.txt entries for patch releases working is if they are automated. I recently scripted making an HTML list of criticals and "8.0.N release notes" issues, and it would be simple to output that into CHANGELOG.txt format as well. But that requires Drupal.org staging DB access, and it's still putting the same information in 2+ different places, and it's yet more burden on the release manager.

Isn't the idea of #12 to more or less move it to a page on d.o (which could be linked from core) and maintain it there? That page can then also link to git log and change records for lower-level changes. Hard to tell how it'd go, but it seems worth trying.

3. Maintain a CHANGELOG for patch releases of only criticals and issues tagged for release notes mention, and automate it.

Just to be clear, this is basically the option I'm advocating for. I don't think all changes should be in CHANGELOG.txt, just the important ones.

For 8.0.3, for example, there are a total of four issues that have been tagged:
https://www.drupal.org/project/issues/search/drupal?issue_tags=8.0.3+rel...
https://www.drupal.org/list-changes/drupal/published?version=8.0.3

If there are none for a particular release that's fine too; there can just be a standard "Numerous bug fixes, API documentation improvements, and additional automated test coverage" CHANGELOG.txt entry in that case. It's happened sometimes in the past.

What I do for Drupal 7 is mention issues in CHANGELOG.txt at the time of commit. Then for the release notes, I just copy-paste from CHANGELOG.txt. So maintaining CHANGELOG.txt is not actually any extra work (since everything in it would have needed to be written for the release notes anyway).

I think an automated option #15.3 sounds like a good idea. But I'm also not adverse to 15.1 or 15.2. I also think if we feel that an issue needs release notes then it is worth getting the release note entry written before committing. This is what has happened on #1475510: Remove external dependencies from the core repo and let Composer manage the dependencies instead

I also think if we feel that an issue needs release notes then it is worth getting the release note entry written before committing.

Just to clarify I think nearly all issues don't fit into this - only issues where the change is disruptive to build processes and people's workflows. If there is little to no disruption then I think release notes are mainly noise.

Agreed with #19. And by definition, it should be rare to nonexistent that any disruptive change occur in a patch release.

I'm not opposed to maintaining a CHANGELOG for minors, either on d.o or in a file. But I don't think we should for patch releases.

We will need to sort this out for 8.1.x.

There may be cases where site administrators use this file to help determine the current version of a site, without the need to log in. D6 and D7 had the latest version number within the file which was handy. D8.1.10 CHANGELOG.txt only lists to 8.1.7.

If there is another easy way to obtain the current core version without logging in then I'm eager to know.

@Stephen Ollman The version constant will also be listed in core/lib/Drupal.php.

I think he's talking about a scenario where the site administrator doesn't have easy access (or any access) to the codebase.

For example I can figure out what version drupal.org is running by looking at https://www.drupal.org/CHANGELOG.txt regardless of my ability to access the site code or Modules page.

It's strange to have CHANGELOG file with missing versions. If no relevant issues should be displayed according to policy, can we at least display the version + a message and a link to complete list of changes ?

The Drupal 8 committers and Dries discussed this issue recently. We agreed that, for Drupal 8, we will no longer create a changelog that partially duplicates a mixture of the release notes and the git log. We will instead include links to pages with the relevant information in a stub file in the codebase. We also want to retain the collaborative process of drafting the release notes in a human-readable and organized format, so we will still use the Drupal.org issue queue to draft release notes in a markdown format that we will then convert to HTML and place in the release notes node.

This decision won't affect Drupal 7, which can continue to use the CHANGELOG.txt format at D7 maintainers' discretion.

We've started implementing #29 with the 8.4.0 alpha and it seems to be working so far.

https://www.drupal.org/docs/8/understanding-drupal-version-numbers/drupa... is a draft handbook page for the historical CHANGELOG. (It still needs to be added to the book outline).

Ah yes, this issue was "policy and patch". Here's a patch that removes the old CHANGELOG, references the historical CHANGELOG handbook page, and adds some brief information about Drupal 8 releases.

Getting 5xx errors so this may be a double post.

We've now removed the old CHANGELOG in favor of linking Drupal.org resources.

The commits don't include a key part of the patch in #34, so here's a followup patch to propose adding it.

Drupal 8 is now End of Life.

Restoring the fixed status, set by xjm in #38, and the version.