This is a placeholder issue for all Drupal.org (websites/infra) issues that block release of Drupal 8.

See infrastructure blocker for Drupal 8.0.0 (issues tagged in issue queues outside of the core queue) (and the related issues in the sidebar). Only use that tag on issues NOT in the d8 core queue. Issues in the d8 core queue blocking d8 release are Critical and dont need a tag.

It's possible we actually could release Drupal 8 without these things being fixed, but it'd definitely suck.

This issue is postponed on the relevant drupal.org issues.

8.0.0 release candidate blockers

#2467925: [plan] Remove blockers to the "minimum viable" state of DrupalCI to ship Drupal 8 RC1
#2113957: Build server side version fallback system for translations (unless we want to keep manually updating fallback versions for core in core's PHP)

8.0.0 release blockers

See #2559711: [meta] Drupal.org (websites/infra) blockers to Drupal 8.0.0, 8.1.0, etc.

To allow Drupal to play nicely with the rest of the PHP community and Packagist we should use SemVer, current consensus is CORE.MAJOR.MINOR.PATCH.
#1612910: [PP-1, policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)

We need to understand how to push modules to Packagist under the drupal namespace. This is currently locked down to a select few people.
#2537818: Can't add to the Drupal namespace on Packagist

Not yet categorized

[none in here yet]

Fixed

#2229641: D8 usage statistics not shown on d.o
#2285063: Support for Drupal 8 logger API
#1963092: Convert .info file rewriting in packaging plugins to deal with D8 .info.yml files
#2163683: PIFT/PIFT communication broken by D9 requeued patch
#2283379: Update for complex tokens in Twig
#951114: Support all screen sizes
#2170443: [meta] Create a plan for implementing semantic versioning for core
#1867192: Testbots need to run on 5.4, 5.5, 5.6 and 7
#697220: Run tests on all supported database servers
#1424984: Port localize.drupal.org to Drupal 7

Comments

webchick’s picture

hass’s picture

Status: Postponed » Closed (duplicate)
ianthomas_uk’s picture

Issue summary: View changes
Status: Closed (duplicate) » Postponed

Duplicate isn't really appropriate, since that's in a separate project. Also, that's been marked duplicate of #1963092: Convert .info file rewriting in packaging plugins to deal with D8 .info.yml files

longwave’s picture

catch’s picture

Category: Bug report » Task

Not a functional bug in core. I think we might want a single core meta issue to track Drupal.org 8.0.0 release blockers, but moving to task for now.

webchick’s picture

Title: Drupal.org cannot track Drupal 8 usage statistics » Drupal.org blockers to a Drupal 8 release
Parent issue: #1963092: Convert .info file rewriting in packaging plugins to deal with D8 .info.yml files »
Related issues: +#2267715: [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1

Oh, sure. Let's do that.

webchick’s picture

webchick’s picture

Sorry, one more meta comment then I'll leave your poor inboxes alone. Wrong related issue, tagging for the D.o tech team.

webchick’s picture

Issue summary: View changes

I lied. :( Updating issue summary.

hass’s picture

I moved my case #2229641: D8 usage statistics not shown on d.o between several queues without getting any feedback for about nearly 2 months, before I identified the root cause myself and committed Try to see if we get some statistics done until the packaging script will be fixed. After more than 2 weeks I can say, this does not fix the reporting. There are still no D8 installations counted in GA module and I ran cron several times.

Thanks.

catch’s picture

Adding #2268449: Run contrib module branch tests against both dev and latest stable core branches. That's not a hard blocker to 8.0.0, but I'd be very wary about opening 8.1.x without it in place.

catch’s picture

webchick’s picture

rjacobs’s picture

Title: Drupal.org blockers to a Drupal 8 release » Drupal.org (websites/infra) blockers to a Drupal 8 release
Issue summary: View changes

Maybe I scan titles too fast, but it took me a few moments to wrap my head around the fact that this issue is exclusively related to Drupal.org website instances/infra, and has nothing to do with any D8 core codebase issues. The ".org" part of the title just didn't register for some reason. Given that the title appears as a link anchor in lots of places (I got here via a related issue dup link) I figured it could not hurt to make this slightly more unmistakable.

catch’s picture

xjm’s picture

catch’s picture

Gábor Hojtsy’s picture

Adding l.d.o Drupal 8 support issues as suggested by @catch, although one could say these should not only block release but even RCs as soon as we want to go out and say translators should go and translate Drupal 8. So these should be done even sooner.

catch’s picture

I've been assuming that everything linked from here blocks a release candidate.

We might find issues that don't but can sort that out a lot closer to the time.

Gábor Hojtsy’s picture

webchick’s picture

Talked to Neil about this issue on the DSWG call today, he pointed out that porting localize.d.o to D7 is also a blocker for the other localize blockers.

Gábor Hojtsy’s picture

I don't think its a blocker for any of the other blockers, we still keep building D8 supporting code on D6 now. I'm happy though its considered a release blocker, that drupal.org does not want to run an unsupported version of our own system. That's great.

hass’s picture

webchick’s picture

Not sure if this will end up being a release blocker or not yet, but cataloguing here for now, since it'll require packaging changes.

webchick’s picture

Another one. Not a hard release blocker, just a "terribly embarrassing if D8 goes out without this fixed" type of deal.

catch’s picture

Not a blocker, but it'd be good to have api.drupal.org use 8.x as the default by 8.0.1 or so.

catch’s picture

Already critical, removing tag.

xjm’s picture

Issue tags: +Triaged D8 critical
YesCT’s picture

Issue summary: View changes

Added a note in the summary about the infrastructure blocker for Drupal 8.0.0 tag.

Should give us a nicer way to view the issues, and see what queues they are in.
(we can also add "related" issues here without that needing to indicate they are blocking release. just the ones with the tag will block.)

catch’s picture

Title: Drupal.org (websites/infra) blockers to a Drupal 8 release » [meta] Drupal.org (websites/infra) blockers to a Drupal 8 release
webchick’s picture

Adding another one that's not a hard blocker but is related, per drumm.

Also tagging as needing an issue summary update, as it's unclear to the DA currently which of these are "must-have" to ship D8 and which are "really would be nice" and which are "meh."

catch’s picture

Issue summary: View changes

Made a start on categorizing the issues.

catch’s picture

Issue summary: View changes
Gábor Hojtsy’s picture

Issue summary: View changes
YesCT’s picture

Issue summary: View changes

formatting.

webchick’s picture

Just for visibility, here's the proposal on the "MVP" in terms of DrupalCI for PHP/database versions to bring Drupal 8 to a shippable state: #2467925: [plan] Remove blockers to the "minimum viable" state of DrupalCI to ship Drupal 8 RC1

webchick’s picture

webchick’s picture

Issue summary: View changes

Er, thought I did anyway.

webchick’s picture

Priority: Critical » Major

Downgrading, since this is now captured in the release checklist, per #2485119-7: [meta] The Drupal 8.0.0-rc1 Release Checklist.

mgifford’s picture

Status: Postponed » Active
mbrett5062’s picture

Issue tags: -Triaged D8 critical

Removing "Triaged D8 critical" tag, as this is no longer critical re: #39

David_Rothstein’s picture

8.0.0 release candidate blockers
.....
#1867192: Testbots need to run on php 5.3.10, 5.4, 5.5, 5.6 and 7
#697220: Run tests on all supported database servers/engines (unless we drop core support for both postgres and sqlite)

Although these are really great features, I'm not sure why they'd be release blockers... Their absence has never blocked releases before.

In any case, they look to me to have been mostly implemented already for Drupal 8, so can they be removed from the list either way? :)

japerry’s picture

Localize is super close to a launch, just have 2 or so issues that need to be closed.

webchick’s picture

Since we are within sight of 10 D8 release blockers (if we don't find any more, this could potentially happen this week!) wanted to revisit this list.

Done!

Partially done.

Not started.

While good progress has been made on the D7 port of localize, nothing has yet been done on the two big Localization-related blockers for D8:

Needs research.

I'll update the issue summary with this once I get confirmation from a couple of other committers that this is accurate.

Note that there are other issues also tracked above (Composer, contrib support for DrupalCI, etc.) that are not in this list, because AFAIK they do not directly block a Drupal 8.0.0 release.

YesCT’s picture

Under needs research,
there was something mentioned in https://groups.drupal.org/node/475163#comment-1115173 about #2273481: Contrib PSR-4 PHPUnit tests are not picked up by PIFR
"Contrib testing for Drupal8 is very nearly ready for deployment."

catch’s picture

Although these are really great features, I'm not sure why they'd be release blockers... Their absence has never blocked releases before.

They never blocked release before, but we've never had a release with working sqlite or postgresql support before either. So the idea was to release either with those working, or without them altogether, but not to keep pretending we support postgres and sqlite when we don't. This is all working now though which is fantastic, and sqlite/postgres should be 100% pass.

David_Rothstein’s picture

Supposedly both PostgreSQL and SQLite work OK on Drupal 7 (no critical issues filed, at any rate) - not sure if that was the case when Drupal 7 was released. But yes, it's really nice to be able to have the testbot run on those.

Gábor Hojtsy’s picture

@webchick, you mixed up the issues:

While good progress has been made on the D7 port of localize, nothing has yet been done on the two big Localization-related blockers for D8:

Should be the other way around:

Also we can work around #2113957: Build server side version fallback system for translations at least for core (not for distros) with manual release instructions. @catch wanted to avoid that and revisit #2113957: Build server side version fallback system for translations later in the cycle "if that was the only one left".

catch’s picture

I'm a bit confused on what the issue with core translations is.

I thought we'd improved it with #1914070: Improve version fallback for install language. and that should work now we update the VERSION constant in core and do release nodes?

The impression I had was that it was only an issue for 8.0.x-dev snapshots or git checkouts, which feels like it should be solved by either git_deploy or l.d.o - I don't think the right translation fallback for dev snapshots is release blocking (and also don't think an extra release step is worth it as a workaround either).

If we have an issue for tagged releases then that obviously is release blocking but could use a reminder of the exact bug and what the core workaround would be.

Gábor Hojtsy’s picture

@catch: It is true that beta releases started coming out with a version constant fitting the release with beta10 and later. This makes install_get_localization_release() kick in adding 8.0.0-beta12, 8.0.0-alpha1 and 7.0 as a candidates for the upcoming beta13 for example. So if there was a beta12 translation, it will be downloaded fine. This may be unpredictable if two releases come out in quick succession, as we have seen with some betas before and may see with minor releases later on. Then the fallback is to a much earlier version because we don't have info on what was the latest earlier version (the server side fallback would know). So if beta14 comes out within say a day or beta13 for whatever reason, l.d.o is not yet caught up with all the .po files, so translation downloads will then fall back to alpha1 until beta13 and beta14 translations become available. (Beta12 is not checked in this case because the list of fallbacks was designed to get to a somewhat useful file at least sooner than later to not slow down the installer with too many HTTP requests).

The same effect is going to be even stronger right when Drupal 8.0.0 is released, because until translation files become available, it will attempt to first do an HTTP request for 8.0.0's translation, not find it then the next fallback 8.0.0-rc1 will be looked for (even if RC1 was months earlier and if there were possibly multiple string changes due to critical bugs).

I am fine if we know about these problems and we don't consider them release blocking. I understood that both the multiple HTTP requests required in the installer (slowing it down) and that old translations will be used due to unknown prior versions were the reason #2113957: Build server side version fallback system for translations was made a blocker. We can remove it from the blockers if these two are not release blocking problems.

timmillwood’s picture

Issue summary: View changes

Moved #1475510: Remove external dependencies from the core repo and let Composer manage the dependencies instead and #2352091: Create (and maintain) a subtree split of Drupal core to nice to have. These should not block any 8.x release, but really really nice, nice to haves as they make life a lot easier for those building Drupal sites using composer.

catch’s picture

@Gabor so from #50 I'm still not sure what the workaround in core is - that we'd manually add 8.0.0-rc3 as a fallback for 8.0.0? Where does that go if so? Or are you saying the current state of core fallback is as good as it gets now, and it's a question of whether the l.d.o work should block release or not?

So if beta14 comes out within say a day or beta13 for whatever reason, l.d.o is not yet caught up with all the .po files

What's the reason for the lag? Is it literally a 24 hour cron job or something else?

webchick’s picture

Localize is now on D7. :) view-source:http://localize.drupal.org/

Gábor Hojtsy’s picture

@catch: the reason for the lag is the three step process:

  1. l.d.o first needs to get to know about the release, that is parsed in from a dump from d.o
  2. then l.d.o needs to grab the tar.gz and extract/parse it for all the source strings
  3. then l.d.o needs to export the .po files for the strings found

All three need to happen in succession. The cronjob for the first two is the same and runs every 5 minutes. *However* the export for the first one is only refreshed daily or so on drupal.org. (I've only found the release history XML job in jenkins, which may be tied to the tsv we use for (1), but indirectly based on when l.d.o's jobs find new projects, it looks like daily). Then (3) also runs every 5 minutes. It gives precedence to projects based on usage (well, at least an outdated copy of usage from a year or so ago), so it will generate core stuff first and foremost if all goes well. So unless there are issues on l.d.o, the bottleneck looks more like d.o giving out new project info only daily.

As for knowing the last release on a lower level, that either needs to be hardcoded when release levels are bumped (alpha to beta, beta to rc, rc to release, 8.0.x to 8.1.x, 8.max.x to 9.0, etc) or a server side fallback needs to be provided. The good thing about the fallback at least is it only needs to be provided manually when a release level is jumped over, if we choose to handle it manually and not automate it.

webchick’s picture

Issue summary: View changes

We had a pow-wow today with most of the core committers (@xjm, @alexbronstein, @alexpott, @catch) as well as most of the DA tech team (@drumm, @basic, @isntall, @joshuami, @Mixologic, @hestenet, @japerry... hope I didn't forget anyone!) to go through this list.

Localize

DrupalCI

  1. We now have the environments we need, so closed out #1867192: Testbots need to run on 5.4, 5.5, 5.6 and 7 and #697220: Run tests on all supported database servers
  2. However, there are still quite a few D8 blockers from the DrupalCI side, which are enumerated at #2467925-80: [plan] Remove blockers to the "minimum viable" state of DrupalCI to ship Drupal 8 RC1. SQLite/PostgreSQL aren't passing, only a handful of tests are being run on PostgreSQL, a workaround for lack of MariaDB testing, and regularly updated (and transparent) PHP 7 testbots.
  3. There's another class of blockers at #2534132: Disable Legacy Testbots and use drupalCI as our testing infrastructure, but these block the shutting off of PIFT, not Drupal 8 specifically.

Updated issue summary to reflect what I believe is the current status. If I missed anything, please feel free to adjust.

catch’s picture

I just opened #2551309: Increase opcache.max_accelerated_files to have an explicit issue for the PIFR blocker to #2495411: Make simpletest fail a test when it detects pages that need more than 64MB.

That's not a hard blocker for release, mainly for that patch, but it would allow us to take memory checks off the performance checklist item, since we'd have memory constraints enforced in core.

Could be solved either by fixing the PIFR opcache configuration or switching it off.

timmillwood’s picture

Can we add #2315537: Install composer dependencies before running tests to any DrupalCI discussions?

webchick’s picture

Not in this issue, since it's a feature request and it doesn't block D8.

webchick’s picture

As an update...

* #2467925: [plan] Remove blockers to the "minimum viable" state of DrupalCI to ship Drupal 8 RC1 looks close. Might even be done (marked it RTBC just now). Needs confirmation from a PostgreSQL hero.
* #2113957: Build server side version fallback system for translations is RTBC. Gábor is back "in the office" tomorrow and should hopefully be able to confirm.
* Either #2273481: Contrib PSR-4 PHPUnit tests are not picked up by PIFR or #2538462: Contrib Testing Test discovery not working for d6/d7 is an 8.0.0 blocker, but wouldn't necessarily need to block RC. But one way or another, contrib needs to be able to run unit tests before 8.0.0 ships (and the sooner, the better, for module porting acceleration).
* By the same logic, we could shift the deadline of #1933988: Support for Drupal 8 shipped configuration translatables with external dependencies (in effect contrib) to an 8.0.0 blocker instead, since it likewise only screws contrib, not core.

So we may be able to remove this from the RC1 checklist soon-ish if those RTBC things end up actually RTBC.

webchick’s picture

Title: [meta] Drupal.org (websites/infra) blockers to a Drupal 8 release » [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1
Assigned: Unassigned » Gábor Hojtsy
Status: Active » Reviewed & tested by the community

Talked with most of the core committers yesterday—@xjm, @catch, @alexpott, @effulgentsia—and we agreed that we could split this up into two phases:

a) Things that block core == RC1 blocker.
b) Things that block contrib == 8.0.0 blocker.

In light of that, since #2467925: [plan] Remove blockers to the "minimum viable" state of DrupalCI to ship Drupal 8 RC1 was just marked Fixed, I believe that we might actually be done here. (We can re-purpose this issue from RC1 => 8.0.0 and move it to critical once RC1 is shipped.)

It would be good for Gábor to chime in here with his thoughts on whether or not that's true, but I believe:

1) Either #2273481: Contrib PSR-4 PHPUnit tests are not picked up by PIFR or #2538462: Contrib Testing Test discovery not working for d6/d7 are a blocker for 8.0.0 (I know the DA plans to turn off PIFT as soon as possible, so hopefully the second one).

2) Translations are inherently contrib, so #2113957: Build server side version fallback system for translations (which is RTBC anyway) and #1933988: Support for Drupal 8 shipped configuration translatables with external dependencies (in effect contrib) would block 8.0.0, not RC1.

Marking RTBC + assigning to Gábor to get some visibility on this question.

Gábor Hojtsy’s picture

Seeing #2113957: Build server side version fallback system for translations done looks good, however it does not help until #2113955: Rely on proper server side version fallback for translations is also done (or Drupal core is manually updated with last version numbers). While it does not block RC1, it would be sad to get it done as late as 8.0.0 if we want to get translators test with the latest translations using Drupal dev versions and not require them to use concrete tagged betas/RCs. If we are fine requiring translators to use tagged releases at all times to test their translations and/or manually keep updating last versions in core PHP files, then this does not block any release whatsoever.

#1933988: Support for Drupal 8 shipped configuration translatables with external dependencies (in effect contrib) only affects contrib as the title shows :)

Gábor Hojtsy’s picture

Assigned: Gábor Hojtsy » Unassigned
webchick’s picture

My layman's, not-a-translator opinion is that testing against latest dev will become important as Drupal 8.0.0 gets closer and closer to release, but that for the first little while during RC there'll be more than enough new/changed strings in D8 to keep translators busy, so tagged releases only is fine for the first few.

Happy to be told I'm wrong about that, though.

catch’s picture

Title: [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1 » [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1/8.0.0

Just updating the title since the issue summary has information relevant to post-RC i.e. #2559575: [meta] The Drupal 8.0.0 Release Checklist

catch’s picture

Issue summary: View changes
catch’s picture

Title: [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1/8.0.0 » [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1
Issue summary: View changes
webchick’s picture

Status: Reviewed & tested by the community » Fixed

Ok, no disagreements so I think we can call this good to go for RC1. YAY!

webchick’s picture

Assigned: Unassigned » Gábor Hojtsy
Status: Fixed » Needs review

Ah, hold on, I was a bit hasty there.

I think we still need Gábor to respond to #64, because I notice #2113957: Build server side version fallback system for translations got bumped back from RTBC. My read of #63 is that we either need both #1933988: Support for Drupal 8 shipped configuration translatables with external dependencies (in effect contrib) and #2113955: Rely on proper server side version fallback for translations in before RC1, or both of them kicked out of the RC1 checklist and moved to 8.0.0 blockers in #2559711: [meta] Drupal.org (websites/infra) blockers to Drupal 8.0.0, 8.1.0, etc. instead.

There was also some concern from @catch that without one/both? of those fixes, translators would get beta1 for the first 24 hours of rc1's release? Any way to mitigate that, such as manually returning "8.0.0-rc1" from api.drupal.org/api/function/install_get_localization_release/8 in the tagged release?

Basically, I think the bottom line: what of this must block RC1 (which I believe means: prevents translators from starting their D8 translations) vs. what of this can wait until after RC1 but must be done before 8.0.0 (which I believe means: prevents translators from finishing their D8 translations)?

hass’s picture

@webchick: What is the roadmap/deadline for fixing translatable strings? Do we have a tag?

Gábor Hojtsy’s picture

@webchick: yes, we can either manually hardcode the fallback list (add the last (or last two) beta release numbers so they are used as fallback for RC1) or do #2113955: Rely on proper server side version fallback for translations if the server side fallback is working well now. That should help core translatability. #1933988: Support for Drupal 8 shipped configuration translatables with external dependencies (in effect contrib) was already moved to #2559711: [meta] Drupal.org (websites/infra) blockers to Drupal 8.0.0, 8.1.0, etc..

webchick’s picture

I was still confused from that comment, so asked Gábor to tell me in very small words what is still needed here. :)

Basically:

- All of this revolves around the "little function that could," function install_get_localization_release(). This is the function that returns which Drupal version's translations to pull in when downloading translations during installation.

- For tagged releases (beta15, rc1, etc.) this function does what it's intended to do. There is a problem where .po files on localize.drupal.org lag about ~48 hours behind the creation of the release on Drupal.org. However, this is an existing problem, doesn't block RC1. (It could use some investigation by DA staff, however; summary: d.o generates a dump of their new releases every day. l.d.o tries to in fact parse it every fine minutes (LOL) but its only updated once daily on d.o.)

- For dev releases, this function will attempt to make a guess. Until both #2113957: Build server side version fallback system for translations and #2113955: Rely on proper server side version fallback for translations are done, it will have no choice but to try and do this in an extremely silly way, which is falling back to the first release of a given type (alpha1, beta1, rc1). See http://cgit.drupalcode.org/drupal/tree/core/includes/install.core.inc#n1466.

There are, however, at least two workarounds:

#1: Until those two d.o issues are fixed, as part of the checklist before tagging a release, manually hack the following lines in install_get_localization_release:

      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-rc1';
      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-beta1';

to instead be the current release and the one previous (to account for the 48 hour lag). So hypothetically in beta15 right now it would read:

      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-rc1';
      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-beta15';
      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-beta14';

then in RC1 (assuming we don't have a beta16) it would be:

      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-rc1';
      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-beta15';

then in RC2 it would be:

      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-rc2';
      $alternatives[] = $info['major'] . '.' . $info['minor'] . '.0-rc1';

...and so on. Which feels messy and error-prone to me, but hey, it's a workaround.

The second is putting some kind of disclaimer in a drupal_set_message() or whatever on localize.drupal.org that's like:

"Attention Drupal 8 translators: Note that you should only be translating against tagged releases of Drupal 8 (e.g. beta15, rc1), not dev releases, until these issues are fixed."

And then move those issues to 8.0.0 blockers vs. RC1 blockers.

That seems like a 30 second solution, but not sure if it's sufficient.

webchick’s picture

Status: Needs review » Fixed

Ok!

#2113957: Build server side version fallback system for translations is officially DONE! I tested and committed the core patch to remove the silly non-serverside language fallback system here: #2113955: Rely on proper server side version fallback for translations

I believe. We. Are. FINISHED! Thank you so much, DA!!!

webchick’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.