Problem/Motivation
Passing an entity ID between 2147483648 and 9223372036854776832 can cause an error 500 on Postgres. Higher or lower IDs return a 404 as expected.
I wasn't able to reproduce this when testing with sqlite or mysql.
Steps to reproduce
Visit any of the following error URLs to validate a Postgres DB backend.
/node/9223372036854776833(404 shown, correct behaviour)/node/9223372036854776832(500 error, incorrect)/node/2147483648(500 error, incorrect)/node/2147483647(404, correct)/taxonomy/term/99999999999999999(500 error, incorrect)/user/99999999999999999(500 error, incorrect)
Expected behaviour is a 404 for missing node.
Incorrect behaviour (IMO) is an error 500.
database=# select version(); PostgreSQL 12.9 on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
AFAICT Postgres 64-bit is matched here with 64-bit webserver.
I'm using lando/drupal-contributions to test locally:
# MySQL passes SIMPLETEST_DB=mysql://drupal9:drupal9@database/drupal9 php /app/web/vendor/bin/phpunit -c /app/phpunit.xml web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest # SQLite passes SIMPLETEST_DB=sqlite://tmp/test.sqlite php /app/web/vendor/bin/phpunit -c /app/phpunit.xml web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest # Postgres fails SIMPLETEST_DB=pgsql://postgres:postgres@postgres:5432/database php /app/web/vendor/bin/phpunit -c /app/phpunit.xml web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest
Child of #2564307: [meta] Remaining Drupal 10/11 PostgreSQL issues.
Proposed resolution
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #21 | postgres_entity_key_limit-3327520-21.patch | 1.21 KB | chi |
| #16 | 3327520-16-testonly.patch | 1.03 KB | andypost |
Issue fork drupal-3327520
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
xurizaemonComment #4
xurizaemonOutput:
PHPUnit 8.5.31 by Sebastian Bergmann and contributors. Testing web/core/tests/Drupal/FunctionalTests/Routing .EE.EE 6 / 6 (100%) Time: 1.01 minutes, Memory: 8.00 MB There were 4 errors: 1) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #1 ('/node/9223372036854776832') Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected. /app/web/vendor/behat/mink/src/WebAssert.php:794 /app/web/vendor/behat/mink/src/WebAssert.php:130 /app/web/core/tests/Drupal/Tests/WebAssert.php:818 /app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33 /app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703 2) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #2 ('/node/2147483648') Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected. /app/web/vendor/behat/mink/src/WebAssert.php:794 /app/web/vendor/behat/mink/src/WebAssert.php:130 /app/web/core/tests/Drupal/Tests/WebAssert.php:818 /app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33 /app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703 3) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #4 ('/taxonomy/term/99999999999999999') Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected. /app/web/vendor/behat/mink/src/WebAssert.php:794 /app/web/vendor/behat/mink/src/WebAssert.php:130 /app/web/core/tests/Drupal/Tests/WebAssert.php:818 /app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33 /app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703 4) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #5 ('/user/99999999999999999') Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected. /app/web/vendor/behat/mink/src/WebAssert.php:794 /app/web/vendor/behat/mink/src/WebAssert.php:130 /app/web/core/tests/Drupal/Tests/WebAssert.php:818 /app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33 /app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703 ERRORS! Tests: 6, Assertions: 18, Errors: 4.Comment #5
xurizaemonComment #6
xurizaemonComment #7
xurizaemonQueueing tests for cross-db validation of the issue.
Comment #8
andypostI think it should be kernel test to catch where id is wrongly converted or passed
Comment #9
xurizaemonThanks Andy, appreciate that input - I had a feeling this wasn't going to be the right place for it, but wanted to at least provide a test which reproduces the issue. Will take a look at converting the test when I get a chance!
Comment #10
andypostIn logs I see following
Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value "9223372036854775807" is out of range for type integer LINE 4: WHERE "base"."uid" IN ('9223372036854775807') ^: SELECT "base"."uid" AS "uid", "base"."uuid" AS "uuid", "base"."langcode" AS "langcode" FROM "users" "base" WHERE "base"."uid" IN (:db_condition_placeholder_0); Array ( [:db_condition_placeholder_0] => 9223372036854775807 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage() (line 422 of /var/www/html/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).Comment #11
andypostIt looks like the error is the same but input IDs are different 9223372036854776831, 9223372036854776832
Comment #12
andypostAnd the node table
Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value "9223372036854775807" is out of range for type integer LINE 5: WHERE "base"."nid" IN ('9223372036854775807') ^: SELECT "revision"."vid" AS "vid", "revision"."langcode" AS "langcode", "revision"."revision_uid" AS "revision_uid", "revision"."revision_timestamp" AS "revision_timestamp", "revision"."revision_log" AS "revision_log", "revision"."revision_default" AS "revision_default", "base"."nid" AS "nid", "base"."type" AS "type", "base"."uuid" AS "uuid", CASE "base"."vid" WHEN "revision"."vid" THEN 1 ELSE 0 END AS "isDefaultRevision" FROM "node" "base" INNER JOIN "node_revision" "revision" ON "revision"."vid" = "base"."vid" WHERE "base"."nid" IN (:db_condition_placeholder_0); Array ( [:db_condition_placeholder_0] => 9223372036854775807 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage() (line 422 of /var/www/html/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).Comment #13
andypostSomehow ID for entity using "integer" type but revision table using "bigInt"
taxonomy
and node
Comment #14
andypostI think it major as could lead to data loss
Comment #15
andypostThat's because ID is passed to DB as strings
Comment #16
andypostHere's smaller test
Comment #17
daffie commentedAccording to the PostgreSQL documentation the maximum value for an integer value is 2147483647. That PostgreSQL throws an error with
SQLSTATE[22003]: Numeric value out of rangeis for me the right response. That MySQL, MariaDB and SQLite are not doing the same is to me a bit of a disappointment.If we want to fix this, then what should we do? We have 2 options:
1. Change the database driver for PostgreSQL and make it respond like the other ones. With a try and catch block and then in the catch part test if we have called the database with an integer that was too big and then somehow calculate what the right response should be. To me this is not doable.
2. Change the other database drivers to make it respnd like PostgreSQL. This is something we can do, only it will make those drivers do more more and therefor slower. To me that is not worth it.
It would be better if all by Drupal core supported databases would work in the same way, only they do not. We have 2 options to fix this problem and for me both of them do not result in an exceptable result. Therefor I will close this issue as "won't fix". If somebody has a solution that does not result in bigger problem then it solves, please reopen this issue.
Comment #18
andypostComment #19
andypostThat's actually a duplicate of #3081144: Database primary keys can exceed maximum integer storeable (has actually occurred, in watchdog)
Comment #20
chi commented@andypost, how could this duplicate #3081144: Database primary keys can exceed maximum integer storeable (has actually occurred, in watchdog)? The problem described here still exists. Even though #3081144: Database primary keys can exceed maximum integer storeable (has actually occurred, in watchdog) is marked as fixed.
Comment #21
chi commented@daffie, this issue is specifically about content entity IDs. So we can fix on entity storage level.
The patch sets upper limit for allowed entity IDs.
The only concern here is possible BC break. MySQL unsigned int has two times bigger range. So that, theoretically there could be Drupal sites that have entity IDs bigger than 2147483647.