Problem/Motivation

Passing an entity ID between 2147483648 and 9223372036854776832 can cause an error 500 on Postgres. Higher or lower IDs return a 404 as expected.

I wasn't able to reproduce this when testing with sqlite or mysql.

Steps to reproduce

Visit any of the following error URLs to validate a Postgres DB backend.

  • /node/9223372036854776833 (404 shown, correct behaviour)
  • /node/9223372036854776832 (500 error, incorrect)
  • /node/2147483648 (500 error, incorrect)
  • /node/2147483647 (404, correct)
  • /taxonomy/term/99999999999999999 (500 error, incorrect)
  • /user/99999999999999999 (500 error, incorrect)

Expected behaviour is a 404 for missing node.

Incorrect behaviour (IMO) is an error 500.

database=# select version();
 PostgreSQL 12.9 on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit

AFAICT Postgres 64-bit is matched here with 64-bit webserver.

I'm using lando/drupal-contributions to test locally:

# MySQL passes
SIMPLETEST_DB=mysql://drupal9:drupal9@database/drupal9 php /app/web/vendor/bin/phpunit -c /app/phpunit.xml  web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest

# SQLite passes
SIMPLETEST_DB=sqlite://tmp/test.sqlite php /app/web/vendor/bin/phpunit -c /app/phpunit.xml  web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest

# Postgres fails
SIMPLETEST_DB=pgsql://postgres:postgres@postgres:5432/database php /app/web/vendor/bin/phpunit -c /app/phpunit.xml  web/core/tests/Drupal/FunctionalTests/Routing --filter EntityIdValidationTest

Child of #2564307: [meta] Remaining Drupal 10/11 PostgreSQL issues.

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-3327520

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

xurizaemon created an issue. See original summary.

xurizaemon’s picture

Issue summary: View changes

xurizaemon’s picture

Issue summary: View changes

Output:

PHPUnit 8.5.31 by Sebastian Bergmann and contributors.

Testing web/core/tests/Drupal/FunctionalTests/Routing
.EE.EE                                                              6 / 6 (100%)

Time: 1.01 minutes, Memory: 8.00 MB

There were 4 errors:

1) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #1 ('/node/9223372036854776832')
Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected.

/app/web/vendor/behat/mink/src/WebAssert.php:794
/app/web/vendor/behat/mink/src/WebAssert.php:130
/app/web/core/tests/Drupal/Tests/WebAssert.php:818
/app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33
/app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703

2) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #2 ('/node/2147483648')
Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected.

/app/web/vendor/behat/mink/src/WebAssert.php:794
/app/web/vendor/behat/mink/src/WebAssert.php:130
/app/web/core/tests/Drupal/Tests/WebAssert.php:818
/app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33
/app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703

3) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #4 ('/taxonomy/term/99999999999999999')
Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected.

/app/web/vendor/behat/mink/src/WebAssert.php:794
/app/web/vendor/behat/mink/src/WebAssert.php:130
/app/web/core/tests/Drupal/Tests/WebAssert.php:818
/app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33
/app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703

4) Drupal\FunctionalTests\Routing\EntityIdValidationTest::testLargeEntityIds with data set #5 ('/user/99999999999999999')
Behat\Mink\Exception\ExpectationException: Current response status code is 500, but 404 expected.

/app/web/vendor/behat/mink/src/WebAssert.php:794
/app/web/vendor/behat/mink/src/WebAssert.php:130
/app/web/core/tests/Drupal/Tests/WebAssert.php:818
/app/web/core/tests/Drupal/FunctionalTests/Routing/EntityIdValidationTest.php:33
/app/web/vendor/phpunit/phpunit/src/Framework/TestResult.php:703

ERRORS!
Tests: 6, Assertions: 18, Errors: 4.
xurizaemon’s picture

Title: Postgres error 500 on large entity IDs » Postgres error 500 on paths when supplied entity ID greater than 2147483648 and less than 9223372036854776832
xurizaemon’s picture

Title: Postgres error 500 on paths when supplied entity ID greater than 2147483648 and less than 9223372036854776832 » Postgres error 500 on paths when supplied entity ID gte 2147483647 and lte 9223372036854776832
xurizaemon’s picture

Issue summary: View changes

Queueing tests for cross-db validation of the issue.

andypost’s picture

I think it should be kernel test to catch where id is wrongly converted or passed

xurizaemon’s picture

Thanks Andy, appreciate that input - I had a feeling this wasn't going to be the right place for it, but wanted to at least provide a test which reproduces the issue. Will take a look at converting the test when I get a chance!

andypost’s picture

In logs I see following

Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value "9223372036854775807" is out of range for type integer LINE 4: WHERE "base"."uid" IN ('9223372036854775807') ^: SELECT "base"."uid" AS "uid", "base"."uuid" AS "uuid", "base"."langcode" AS "langcode" FROM "users" "base" WHERE "base"."uid" IN (:db_condition_placeholder_0); Array ( [:db_condition_placeholder_0] => 9223372036854775807 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage() (line 422 of /var/www/html/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

andypost’s picture

It looks like the error is the same but input IDs are different 9223372036854776831, 9223372036854776832

andypost’s picture

And the node table
Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22003]: Numeric value out of range: 7 ERROR: value "9223372036854775807" is out of range for type integer LINE 5: WHERE "base"."nid" IN ('9223372036854775807') ^: SELECT "revision"."vid" AS "vid", "revision"."langcode" AS "langcode", "revision"."revision_uid" AS "revision_uid", "revision"."revision_timestamp" AS "revision_timestamp", "revision"."revision_log" AS "revision_log", "revision"."revision_default" AS "revision_default", "base"."nid" AS "nid", "base"."type" AS "type", "base"."uuid" AS "uuid", CASE "base"."vid" WHEN "revision"."vid" THEN 1 ELSE 0 END AS "isDefaultRevision" FROM "node" "base" INNER JOIN "node_revision" "revision" ON "revision"."vid" = "base"."vid" WHERE "base"."nid" IN (:db_condition_placeholder_0); Array ( [:db_condition_placeholder_0] => 9223372036854775807 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage() (line 422 of /var/www/html/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

andypost’s picture

Somehow ID for entity using "integer" type but revision table using "bigInt"

db=# \d users
uid	integer		not null	nextval('users_uid_seq'::regclass)
...
db=# \d users_data
uid	bigint		not null	0
...

taxonomy

db=# \d taxonomy_term_data
tid	integer		not null	nextval('taxonomy_term_data_tid_seq'::regclass)
revision_id	bigint			
...
db=# \d taxonomy_term_field_data
tid	bigint		not null	
revision_id	bigint		not null	
...

and node

db=# \d node
nid	integer		not null	nextval('node_nid_seq'::regclass)
vid	bigint			
...
db=# \d node_field_data
nid	bigint		not null	
vid	bigint		not null	
...
andypost’s picture

Priority: Normal » Major

I think it major as could lead to data loss

andypost’s picture

That's because ID is passed to DB as strings

db=# select * from users where uid='9223372036854776831';
ERROR:  value "9223372036854776831" is out of range for type integer
LINE 1: select * from users where uid='9223372036854776831';
                                      ^
db=# select * from users where uid=9223372036854776831;
...
db=# select * from users_field_data where uid='9223372036854776831';
ERROR:  value "9223372036854776831" is out of range for type bigint
LINE 1: select * from users_field_data where uid='922337203685477683...
                                                 ^
db=# select * from users_field_data where uid=9223372036854776831;
andypost’s picture

Status: Active » Needs review
StatusFileSize
new1.03 KB

Here's smaller test

daffie’s picture

Status: Needs review » Closed (won't fix)

According to the PostgreSQL documentation the maximum value for an integer value is 2147483647. That PostgreSQL throws an error with SQLSTATE[22003]: Numeric value out of range is for me the right response. That MySQL, MariaDB and SQLite are not doing the same is to me a bit of a disappointment.
If we want to fix this, then what should we do? We have 2 options:
1. Change the database driver for PostgreSQL and make it respond like the other ones. With a try and catch block and then in the catch part test if we have called the database with an integer that was too big and then somehow calculate what the right response should be. To me this is not doable.
2. Change the other database drivers to make it respnd like PostgreSQL. This is something we can do, only it will make those drivers do more more and therefor slower. To me that is not worth it.

It would be better if all by Drupal core supported databases would work in the same way, only they do not. We have 2 options to fix this problem and for me both of them do not result in an exceptable result. Therefor I will close this issue as "won't fix". If somebody has a solution that does not result in bigger problem then it solves, please reopen this issue.

andypost’s picture

andypost’s picture

Status: Closed (won't fix) » Closed (duplicate)
chi’s picture

chi’s picture

Status: Closed (duplicate) » Needs review
StatusFileSize
new1.21 KB

@daffie, this issue is specifically about content entity IDs. So we can fix on entity storage level.

The patch sets upper limit for allowed entity IDs.

The only concern here is possible BC break. MySQL unsigned int has two times bigger range. So that, theoretically there could be Drupal sites that have entity IDs bigger than 2147483647.

Status: Needs review » Needs work

The last submitted patch, 21: postgres_entity_key_limit-3327520-21.patch, failed testing. View results

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.