Problem/Motivation

Under PHP 8 installing Drupal results in errors because \Drupal\Core\Database\StatementInterface::fetch(), ::fetchAll() and ::setFetchMode() do not match their \PDOStatement::* counterparts.

Ref https://github.com/php/php-src/pull/6220

Proposed resolution

Introduce a PHP 7 compatible interface and trait for these three methods, as well as PHP 8 compatible interface and trait. Use the right one based on the PHP version identified.

interface Php7StatementInterface {
  public function fetch($mode = NULL, $cursor_orientation = NULL, $cursor_offset = NULL);
  public function fetchAll($mode = NULL, $column_index = NULL, $constructor_arguments = NULL);
  public function setFetchMode($mode, $a1 = NULL, $a2 = []);
}

interface Php8StatementInterface {
  public function fetch(int $mode = \PDO::FETCH_BOTH, int $cursor_orientation = \PDO::FETCH_ORI_NEXT, int $cursor_offset = 0);
  public function fetchAll(int $mode = \PDO::FETCH_BOTH, ...$args);
  public function setFetchMode(int $mode, ...$params);
}

Remaining tasks

Review.

User interface changes

None.

API changes

When using PHP 8, the three methods will have a different signature. Given that the interface is already based off of \PDOStatement which became more strict in PHP 8, we cannot go around this. The default values of the methods also changed from NULL to integer constants defined by PDO. Callers that wish to use some of the default values and specify some other values will encounter problems when running on PHP 8.

Data model changes

None.

Release notes snippet

The signatures of \Drupal\Core\Database\StatementInterface::fetch(), ::fetchAll() and ::setFetchMode() changed when running on PHP 8 due to the underlying changes in PHP's built in \PDOStatement signatures for the respective methods.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alexpott created an issue. See original summary.

alexpott’s picture

catch’s picture

se/StatementInterface.php
+++ b/core/lib/Drupal/Core/Database/StatementInterface.php
@@ -109,7 +109,7 @@ public function setFetchMode($mode, $a1 = NULL, $a2 = []);

@@ -109,7 +109,7 @@ public function setFetchMode($mode, $a1 = NULL, $a2 = []);
    * @return
    *   A result, formatted according to $mode.
    */
-  public function fetch($mode = NULL, $cursor_orientation = NULL, $cursor_offset = NULL);
+  public function fetch(int $mode = \PDO::FETCH_BOTH, int $cursor_orientation = \PDO::FETCH_ORI_NEXT, int $cursor_offset = 0);
 

Should we remove this method altogether so that it just inherits from PHP?

daffie’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Access/CsrfTokenGenerator.php
    @@ -86,8 +86,14 @@ public function validate($token, $value = '') {
    +    $value = $this->computeToken($seed, $value);
    ...
    -    return hash_equals($this->computeToken($seed, $value), $token);
    +    return hash_equals($value, $token);
    

    We can just remove these changes from the patch. The method computeToken() does not change the value of the variable $token.

  2. This change will be a BC for every contrib database driver that overrides the method fetch(). I am not if/how we can fix that.
daffie’s picture

+++ b/core/lib/Drupal/Core/Access/CsrfTokenGenerator.php
@@ -86,8 +86,14 @@ public function validate($token, $value = '') {
+    // PHP 8.0 strictly typehints for hash_equals. Maintain BC until we can
+    // enforce scalar typehints on this method.
+    if (!is_string($token)) {
+      return FALSE;
+    }

We should add testing for this change.

alexpott’s picture

Status: Needs work » Needs review
FileSize
748 bytes
2.05 KB

@daffie thanks for #5 That change was not meant to be here. It's part of #3156880: \Drupal\Core\Access\CsrfTokenGenerator::validate() - ensure $token is a string before calling hash_equals() - fwiw there is test coverage of this returning false in this case it is just that in PHP 8 we need to return early because there's a scalar typehint on the PHP method.

Re #2 this is an odd one. The docs from the interface say:

 * Child implementations should either extend PDOStatement:
 * @code
 * class Drupal\Core\Database\Driver\oracle\Statement extends PDOStatement implements Drupal\Core\Database\StatementInterface {}
 * @endcode
 * or define their own class. If defining their own class, they will also have
 * to implement either the Iterator or IteratorAggregate interface before
 * Drupal\Core\Database\StatementInterface:
 * @code
 * class Drupal\Core\Database\Driver\oracle\Statement implements Iterator, Drupal\Core\Database\StatementInterface {}
 * @endcode

If we remove this method then the second case of defining their own class is problematic. See \Drupal\Core\Database\StatementEmpty

#3. The BC concerns.

So this is a really tricky. Here are something we can do. Move this method into another interface where we can maintain 2 versions. Then implement two traits that implement the correct signature for each version and call a doFetch method. Then contrib can upgrade to this system in their own time to support PHP 8. Doing this has some disadvantages:

  • It's complex to maintain - lots of sleight of hand.
  • Happens on every request
  • For Drupal 9 the changes appear to be BC compatible so perhaps the easiest thing is the hard break - but that would mean getting this D9 as earlier as possible.

tldr; I think we have to do something awful with classes/interfaces/traits in order to make this work. Because it is too late for Drupal 9 and we also have tried to make it easy to support code across D8 and D9 and we want D8 to be compatible with PHP 8 so we're going to need to do it anyway.

Patch attached removes the unintended code. BC shenanigans is yet to be implemented.

alexpott’s picture

Here's an implementation will a complete BC layer for PHP 7 and will also work for Drupal 8 (i hope). Do this will mean that contrib drivers use the same methods to only make a minimal change when they choose to support PHP 8 and they won't break on sites using PHP 7.

As promised this is not a pretty solution but I'm not sure there's much else we can do.

alexpott’s picture

Doesn't apply to 8.9.x - conflict in StatementEmpty. Here's a patch for 8.9.x to show the BC layer works all the way back to 8.9.x / PHP7

Also testing on PHP 8 on #3156595-50: Make Drupal 9 installable on PHP8

alexpott’s picture

The last submitted patch, 7: 3156881-7.patch, failed testing. View results

The last submitted patch, 8: 3156881-8.9.x-8.patch, failed testing. View results

alexpott’s picture

So #9 didn't quite work on PHP 8 because of the change of default value from NULL to \PDO::FETCH_BOTH

Here's a fix for that.

hussainweb’s picture

Title: \Drupal\Core\Database\StatementInterface::fetch() needs scalar typehints for PHP 8 » Various methods of \Drupal\Core\Database\StatementInterface have changed their signature for PHP 8
FileSize
17.04 KB
23.45 KB

There are related changes in PDOStatement which were introduced in PHP 8.0.0beta3. These are the two commits which made the relevant changes:

I think it makes sense to include these changes here as well. Hence, renaming the issue for clarity.

hussainweb’s picture

I noticed a few issues during manual testing. It was funny how such a critical error was not visible except in dynamic routes.

Status: Needs review » Needs work

The last submitted patch, 14: 3156881-15.patch, failed testing. View results

hussainweb’s picture

Status: Needs work » Needs review
FileSize
1.08 KB
24.39 KB

Hopefully, this workaround should be fine to fix the problem.

alexpott’s picture

We shouldn't really be making other people's concrete class methods our interface. This mess is being caused by borrowing parts of PDO's API without proper encapsulation. Fortunately some of this appears to be completely dead code.

hussainweb’s picture

I agree. Case in point: The site installed and ran well when I forgot to use the new StatementTrait in the \Drupal\Core\Database\Statement class. I then realised I was missing it. I added it and created the patch in #13 (which failed manual testing).

I wonder if it would be better to refactor after making it work with PHP 8 or how much refactoring would even be possible in the current release cycle. I would at least want to move away from the current func_get_args() logic which now gets repeated in multiple places in the patch.

Gábor Hojtsy’s picture

+++ b/core/lib/Drupal/Core/Database/Statement.php
@@ -2,6 +2,13 @@
+if (PHP_VERSION_ID >= 80000) {
+  class_alias('\Drupal\Core\Database\Php8StatementTrait', '\Drupal\Core\Database\StatementTrait');
+}
+else {
+  class_alias('\Drupal\Core\Database\Php7StatementTrait', '\Drupal\Core\Database\StatementTrait');
+}
+

@@ -14,6 +21,7 @@
+  use StatementTrait;

Clever! I agree this is not easy to maintain per say, but we would need to do the same in Drupal 8 and also to keep BC in 9. We can get rid of it in Drupal 10 ;) Should we include our own deprecation here of the old interfaces that we exposed, targeting Drupal 10?

alexpott’s picture

Looking HEAD I don't understand why we have the code in \Drupal\Core\Database\Statement

 /**
   * {@inheritdoc}
   */
  public function setFetchMode($mode, $a1 = NULL, $a2 = []) {
    // Call \PDOStatement::setFetchMode to set fetch mode.
    // \PDOStatement is picky about the number of arguments in some cases so we
    // need to be pass the exact number of arguments we where given.
    switch (func_num_args()) {
      case 1:
        return parent::setFetchMode($mode);

      case 2:
        return parent::setFetchMode($mode, $a1);

      case 3:
      default:
        return parent::setFetchMode($mode, $a1, $a2);
    }
  }

  /**
   * {@inheritdoc}
   */
  public function fetchAll($mode = NULL, $column_index = NULL, $constructor_arguments = NULL) {
    // Call \PDOStatement::fetchAll to fetch all rows.
    // \PDOStatement is picky about the number of arguments in some cases so we
    // need to be pass the exact number of arguments we where given.
    switch (func_num_args()) {
      case 0:
        return parent::fetchAll();

      case 1:
        return parent::fetchAll($mode);

      case 2:
        return parent::fetchAll($mode, $column_index);

      case 3:
      default:
        return parent::fetchAll($mode, $column_index, $constructor_arguments);
    }
  }

Like is it even needed at all? Like there's no logic here and passing the exact same number of arguments you're passed is what's going to happen even if these overrides did not exist.

alexpott’s picture

Ah I know why we're doing #21 - because we have different defaults :( :( :( :(

alexpott’s picture

andypost’s picture

andypost’s picture

hussainweb’s picture

@alexpott, re #21 and #22, you're right. This seems a little unintuitive (especially the fact that this happens twice in case of \Drupal\Core\Database\Statement) but it's required. The comment explains why and not having this resulted in the bugs I mentioned where routes with dynamic parameters wouldn't work.

Sidenote about that issue: It's weird that a PDO behaviour would cause impact to dynamic parameters in routes but this happens because a PDO exception is swallowed in \Drupal\Core\Routing\RouteProvider::getRoutesByPath.

hussainweb’s picture

@Gábor Hojtsy, re: #20

We can get rid of it in Drupal 10 ;) Should we include our own deprecation here of the old interfaces that we exposed, targeting Drupal 10?

Removing this in Drupal 10 would mean that we would also have to remove PHP 7.4 support. We'll have to refactor the methods into new ones and then deprecate existing methods. But that would be tricky too as these are actually concrete methods in \PDOStatement and we extend that class.

Gábor Hojtsy’s picture

@hussainweb: PHP 7.4 is the last version series of PHP 7 and is EOL is a few months after the release of Drupal 10 (https://www.php.net/supported-versions.php). The current plan in #3118147: [meta] Set Drupal 10 platform and browser requirements six months before the release is to require PHP 8 in Drupal 10.x as soon as the branch opens for development. That would be too late to deprecate anything for Drupal 10. I don't know how to make sure we revisit this as a possible deprecation once the PHP requirement is decided but soon enough so contrib can act on it.

Gábor Hojtsy’s picture

Added a change record at https://www.drupal.org/node/3170913, the issue summary totally needs updating though. I am not sure I am the best person to do that. Also need reviews on the change record :D

andypost’s picture

Gábor Hojtsy’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Updated the issue summary as well (please verify). Also added a release notes snippet, as this kind of change cannot go without one. Will need a release notes tag once we know which version it lands in.

andypost’s picture

+++ b/core/lib/Drupal/Core/Database/Statement.php
@@ -2,6 +2,13 @@
+if (PHP_VERSION_ID >= 80000) {
+  class_alias('\Drupal\Core\Database\Php8StatementTrait', '\Drupal\Core\Database\StatementTrait');
...
+else {
+  class_alias('\Drupal\Core\Database\Php7StatementTrait', '\Drupal\Core\Database\StatementTrait');

+++ b/core/lib/Drupal/Core/Database/StatementEmpty.php
@@ -2,6 +2,13 @@
+if (PHP_VERSION_ID >= 80000) {
+  class_alias('\Drupal\Core\Database\Php8StatementTrait', '\Drupal\Core\Database\StatementTrait');
...
+else {
+  class_alias('\Drupal\Core\Database\Php7StatementTrait', '\Drupal\Core\Database\StatementTrait');

+++ b/core/lib/Drupal/Core/Database/StatementInterface.php
@@ -2,6 +2,13 @@
+if (PHP_VERSION_ID >= 80000) {
+  class_alias('\Drupal\Core\Database\Php8StatementInterface', '\Drupal\Core\Database\StatementInterfaceBase');
...
+else {
+  class_alias('\Drupal\Core\Database\Php7StatementInterface', '\Drupal\Core\Database\StatementInterfaceBase');

+++ b/core/lib/Drupal/Core/Database/StatementPrefetch.php
@@ -2,6 +2,13 @@
+  class_alias('\Drupal\Core\Database\Php8StatementTrait', '\Drupal\Core\Database\StatementPrefetchTrait');
...
+  class_alias('\Drupal\Core\Database\Php7StatementTrait', '\Drupal\Core\Database\StatementPrefetchTrait');

Why they are not using ::class - from PHP POV strings are slower and code should prevent using duplicated strings in the same file

alexpott’s picture

@andypost class aliases tend to use the fully qualified class name because:

  • It only works for the first argument. So not doing so introduces a mental dissonance
  • Having both sides as full qualified makes it easier to spot exactly what is happening - class aliases are one layer of complexity and use statements are another layer.

See how vendor is using class_alias()

catch’s picture

I think we can deprecate for Drupal 10 now on the assumption we require PHP 8, and we always have the option of updating the deprecation to Drupal 11 for whatever reason later.

alexpott’s picture

I keep on looking at this issue and feel that we should be addressing the root cause. It feels dangerous that \Drupal\Core\Database\StatementInterface::fetchAll(),\Drupal\Core\Database\StatementInterface::setFetchMode() and \Drupal\Core\Database\StatementInterface::fetch() exist. These are not our methods. These are from \PDOStatement and their signature is owned by PHP and not by Drupal. The reason we have this interface is so that we can have alternate implementations like \Drupal\Core\Database\StatementPrefetch and \Drupal\Core\Database\StatementEmpty

Looking for prior art... looks like Doctrine is going to have the same issues... https://github.com/doctrine/dbal/blob/2.11.x/lib/Doctrine/DBAL/Driver/Re... - ah I see the 3.0.x branch will be PHP 8 compatible. Glancing at that code - they've moved away from copying PHP's code to their interface - https://github.com/doctrine/dbal/blob/3.0.x/src/Driver/Result.php and they've properly decoupled there Statement from PDO's statement - see https://github.com/doctrine/dbal/blob/3.0.x/src/Driver/PDO/Statement.php

Side note this blog post is interesting.

mondrake’s picture

+1 on #35. I filed #3174662: Encapsulate \PDOStatement instead of extending from it to (potentially) address the root cause.

andypost’s picture

catch’s picture

Status: Needs review » Postponed

#3174662: Encapsulate \PDOStatement instead of extending from it is looking very viable, so going to mark this postponed - we can mark duplicate once that issue lands, or re-open here if there's a nasty surprise.

andypost’s picture

One more change in PDO parameters https://github.com/php/php-src/pull/6272

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

mondrake’s picture

Status: Postponed » Closed (won't fix)

The alternative approach in #3174662: Encapsulate \PDOStatement instead of extending from it was committed.

mikemadison’s picture

FYI this cropped up for me today, and I beat my head on it for an hour before I realized what happened (and thought I would share here). We updated an environment to PHP 8.0 and Drupal 9.3.x. Except, that the deployment of 9.3.x silently failed and we were actually still running Drupal 8.9.x on a PHP 8.0 server (which obviously isn't going to work). So, if you are hitting this, definitely take the 30 seconds required to 100% confirm that you are running Drupal 9.1.x or newer (which actually supports PHP 8.0)!