Problem/Motivation

Drupal works only on PostgreSQL when it is installed on the default schema called "public". Drupal should also work on other schemas.

Proposed resolution

Change the hardcoded schema name "public" to one set by the connection options from settings.php.
To test that it all works a number of tests need to be added. The drupal testbot has for PostgreSQL an extra schema called "testing_fake". This schema was added for this issue.
What operation the test should do:

  1. The test should start by creating a cloned database connection with the schema set to "testing_fake";
  2. Create a table in the new schema;
  3. Add, change, and delete a primary key;
  4. Add, change and delete a unique key;
  5. Add, change and delete a non unique key;
  6. Add, change and delete a column/field;
  7. Do a table search;
  8. Rename a table;
  9. Insert data in the table;
  10. Update the data in the table;
  11. Merge the data in the table;
  12. Upsert data in the table;
  13. Delete data from the table;
  14. Truncate the data in the table;
  15. Drop the table;

All operations should be tested that do what they should. For instance when you test renaming a table, test that there is no table with the old name and there is a table with the new name.

Remaining tasks

Write the patch with the test.
Review the patch.
Commit the patch.

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

TBD

The original summary

I've tried to install Drupal 7 on 2 separate systems using a PostgreSQL 9.0 database backend, and both times it has failed at the same point.

The error message in the PostgreSQL logs both times is:

ERROR: null value in column "rid" violates not-null constraint
STATEMENT: INSERT INTO role_permission (rid, permission, module)
VALUES (NULL, 'administer blocks', 'block')

I followed the instructions in INSTALL.pgsql.txt, and everything else
installs up to this point.

The actual error message returned on the installation page is:

An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: http://bison:8089/install.php?profile=standard&locale=en&id=1&op=do
StatusText: OK
ResponseText:
Home | Drupal
@import url("http://bison:8089/modules/system/system.theme.css?0");
@import url("http://bison:8089/modules/system/system.messages.css?0");
@import url("http://bison:8089/modules/system/system.menus.css?0");
@import url("http://bison:8089/modules/system/system.base.css?0");
@import url("http://bison:8089/modules/comment/comment.css?0");
@import url("http://bison:8089/modules/field/theme/field.css?0");
@import url("http://bison:8089/modules/node/node.css?0");
@import url("http://bison:8089/modules/search/search.css?0");
@import url("http://bison:8089/modules/user/user.css?0");
@import url("http://bison:8089/modules/system/system.admin.css?0");
@import url("http://bison:8089/modules/system/system.maintenance.css?0");
@import url("http://bison:8089/themes/seven/reset.css?0");
@import url("http://bison:8089/themes/seven/style.css?0");
Home
Installation tasksChoose profile(done)Choose language(done)Verify
requirements(done)Set up database(done)Install
profile(active)Configure siteFinished
SQLSTATE[25P02]: In failed sql transaction: 7 ERROR: current
transaction is aborted, commands ignored until end of transaction
block

Steps to recreate:

Gentoo x64 (2.6.31 kernel)
Apache 2.2.15
PHP 5.2.14
PostgreSQL 9.0.3

1) Run: wget http://ftp.drupal.org/files/projects/drupal-7.0.tar.gz
2) Extract using: tar xvf drupal-7.0.tar.gz
3) Create destination directroy: mkdir /var/www/drupal-7.0
4) Copy files: mv drupal-7.0/* drupal-7.0/.htaccess /var/www/drupal-7.0
3) chgrp -R apache /var/www/drupal-7.0
4) Run: createuser --pwprompt --encrypted --no-createrole --no-createdb drupal
5) Run: createdb --encoding=UTF8 --owner=drupal drupal
6) Run: psql -U postgres -c "CREATE SCHEMA drupal AUTHORIZATION drupal;" drupal
7) Change to drupal dir: cd /var/www/drupal-7.0
8) Change permissions on default sites dir: chmod a+w sites/default
9) Create settings file: cp sites/default/default.settings.php sites/default/settings.php
10) Update settings permissions: chmod a+w sites/default/settings.php
11) Update sites/default/settings.php file as per INSTALL.pgsql.txt with schema name, by adding this line: $db_prefix = 'drupal';
12) Navigate to webpage and follow installation.
13) Error appears as described above.

Screenshots of installation attached.

Installation appears to work fine when using a schema named anything but "drupal", or not using a schema at all.

CommentFileSizeAuthor
#176 interdiff_173-176.txt4.77 KBArantxio
#176 1060476-176.patch25.42 KBArantxio
#173 interdiff_171-173.txt663 bytesravi.shankar
#173 1060476-173.patch24.68 KBravi.shankar
#171 interdiff_167-171.txt3.33 KBravi.shankar
#171 1060476-171.patch24.69 KBravi.shankar
#170 1060476-nr-bot.txt86 bytesneeds-review-queue-bot
#167 interdiff_164_167.txt896 bytesArantxio
#167 1060476-167.patch24.7 KBArantxio
#164 interdiff_161_164.txt3.11 KBArantxio
#164 1060476-164.patch24.71 KBArantxio
#161 1060476-161.patch24.75 KBArantxio
#159 interdiff_157-159.txt1.64 KBArantxio
#159 1060476-159.patch24.75 KBArantxio
#158 1060476-with-if-on-schema-158.patch24.19 KBArantxio
#158 1060476-without-if-on-schema-158.patch24.16 KBArantxio
#157 interdiff_156-157.txt610 bytesArantxio
#157 1060476-157.patch24.21 KBArantxio
#156 interdiff_155-156.txt5.84 KBArantxio
#156 1060476-156.patch24.28 KBArantxio
#156 interdiff_152-155.txt1.37 KBArantxio
#155 1060476-155.patch24.24 KBArantxio
#152 interdiff_150-152.txt31.42 KBArantxio
#152 1060476-152.patch24.77 KBArantxio
#150 interdiff_149-150.txt581 bytesArantxio
#150 1060476-150.patch29.37 KBArantxio
#149 interdiff_148-149.txt5.08 KBArantxio
#149 1060476-149.patch29.37 KBArantxio
#148 interdiff_147-148.txt1.32 KBArantxio
#148 1060476-148.patch29.57 KBArantxio
#147 interdiff_142-147.txt8.07 KBArantxio
#147 1060476-147.patch29.84 KBArantxio
#146 interdiff_142-145.txt23.06 KBasad_ahmed
#146 1060476_145.patch19.77 KBasad_ahmed
#142 1060476-142.patch31.51 KBArantxio
#141 interdiff_135-141.txt731 bytesjordan.jamous
#141 1060476-141.patch14.95 KBjordan.jamous
#57 drupal-1060476-57.patch6.3 KBRenrhaf
#57 diff-55-bis-57.txt3.51 KBRenrhaf
#55 drupal-1060476-55-bis.patch2.96 KBRenrhaf
#55 drupal-1060476-55.patch2.18 KBRenrhaf
#52 drupal-1060476-52.patch2.18 KBRoSk0
#50 drupal-1060476-50.patch2.21 KBRoSk0
#44 1060476-44.patch1.17 KBozden_meren
#38 interdiff-1060476-37-38.txt1.28 KBdaffie
#38 1060476-38.patch2.14 KBdaffie
#37 1060476-37.patch2.14 KBdaffie
#37 interdiff-1060476-34-37.txt1.89 KBdaffie
#34 1060476-pgsql_public_schema-34.patch744 bytesjaredsmith
#31 1060476-pgsql_public_schema-31.patch742 bytesstefan.r
#29 1060476-pgsql_public_schema-29.patch725 bytesstefan.r
#25 sandboxdiff.txt568.64 KBericsol
#2 watchdog.txt41.2 KBdark_ixion
7.png23.23 KBdark_ixion
6.png5.15 KBdark_ixion
5.png5.07 KBdark_ixion
4.png3.69 KBdark_ixion
3.png13.78 KBdark_ixion
2.png5.76 KBdark_ixion
1.png13.22 KBdark_ixion
#58 drupal-1060476-58.patch6.9 KBRenrhaf
#58 interdiff-57-58.txt812 bytesRenrhaf
#60 drupal-1060476-60.patch6.93 KBRoSk0
#62 drupal-1060476-62.patch9.14 KBRoSk0
#62 drupal-1060476-60-62-interdiff.txt3.55 KBRoSk0
#64 drupal-1060476-64.patch9.2 KBRoSk0
#64 drupal-1060476-60-64-interdiff.txt3.61 KBRoSk0
#66 drupal-1060476-66.patch9.35 KBRoSk0
#66 drupal-1060476-60-66-interdiff.txt3.6 KBRoSk0
#68 drupal-1060476-68.patch10.17 KBRoSk0
#68 drupal-1060476-66-68-interdiff.txt832 bytesRoSk0
#70 drupal-1060476-70.patch12.96 KBRoSk0
#70 drupal-1060476-68-70-interdiff.txt7.52 KBRoSk0
#72 drupal-1060476-72.patch14.39 KBRoSk0
#72 drupal-1060476-70-72-interdiff.txt1.43 KBRoSk0
#73 drupal-1060476-73.patch15.21 KBRoSk0
#73 drupal-1060476-72-73-interdiff.txt3.15 KBRoSk0
#76 drupal-1060476-76-test-only.patch3.32 KBRoSk0
#76 drupal-1060476-76.patch16.28 KBRoSk0
#76 drupal-1060476-73-76-interdiff.txt2.61 KBRoSk0
#78 drupal-1060476-78-test-only.patch2.76 KBRoSk0
#78 drupal-1060476-78.patch15.72 KBRoSk0
#78 drupal-1060476-76-78-interdiff.txt3.63 KBRoSk0
#85 drupal-1060476-85.patch17.14 KBRoSk0
#85 drupal-1060476-85-test-only.patch4.19 KBRoSk0
#85 drupal-1060476-78-85-interdiff.txt2.22 KBRoSk0
#102 drupal-1060476-102.patch16.94 KBRenrhaf
#105 1060476-105.patch16.94 KBravi.shankar
#113 1060476-113.patch15.91 KBridhimaabrol24
#120 Schema.php_.rej_.txt3.4 KBintrafusion
#123 1060476-123.patch15.63 KBvsujeetkumar
#124 1060476-124.patch15.61 KBvsujeetkumar
#124 interdiff_123-124.txt1.92 KBvsujeetkumar
#125 diff-105-113.txt5.96 KBquietone
#125 diff-113-124.txt4.35 KBquietone
#128 1060476-128.patch13.69 KBmogtofu33
#128 interdiff_124-128.txt5.49 KBmogtofu33
#132 1060476-132.patch14.33 KBintrafusion
#135 1060476-135.patch14.32 KBintrafusion
#137 1060476-137.patch13.8 KBtostinni
#138 1060476-138.patch13.92 KBtostinni
#140 1060476-140.patch14.96 KBjordan.jamous
#140 interdiff_135-140.txt731 bytesjordan.jamous

Issue fork drupal-1060476

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

David_Rothstein’s picture

Thanks for the detailed bug report!

It looks like this is happening when the administration role gets created, via this code:

  // Create a default role for site administrators, with all available permissions assigned.
  $admin_role = new stdClass();
  $admin_role->name = 'administrator';
  $admin_role->weight = 2;
  user_role_save($admin_role);
  user_role_grant_permissions($admin_role->rid, array_keys(module_invoke_all('permission')));

Somehow the role ID is winding up NULL, so the issue must occur in user_role_save(), before it.

If you look in the {watchdog} table in Drupal's database after this occurs, what kind of messages appear at the end? Are there any relevant error messages?

dark_ixion’s picture

FileSize
41.2 KB

I've checked what appears in the watchdog table as you suggested, and there appears to be some issues in the last few rows, although what's specifically wrong I don't know. Output attached.

David_Rothstein’s picture

Component: install system » postgresql database

Hm, I am not really sure how to intrepet that. Normally the "variables" column of that database table would contain something human-readable, but in your case it doesn't seem to....

I'm afraid I don't know a whole lot about PostgreSQL myself, but perhaps moving it to a different place in the issue queue will get some other people's eyes on it?

Anonymous’s picture

I got exactly the same error during an installation. Debian Squeeze, Apache 2.2.16, php 5.3.3-7 and PostgreSQL 8.4.7.

I know it was possible because I already did an installation on a PostgreSQL server without any problems. I figured out that if you are installing Drupal in a different database schema as the default public, you got this problem. Installing in public works like a charm...

hshana’s picture

Anybody have a work around? This worked fine in 6...

Josh Waihi’s picture

Priority: Normal » Minor
Status: Active » Reviewed & tested by the community

Ok, this makes sense to me now. I'm happy with the commit. Its kinda minor since its more a PostgreSQL specific issue than anything to do with Drupal.

David_Rothstein’s picture

Priority: Minor » Normal
Status: Reviewed & tested by the community » Active

Er, there is no patch here :) Maybe you posted that in the wrong issue?

Josh Waihi’s picture

LOL, I fully did, apologies!

dark_ixion’s picture

To answer the question as to why the variables column is returning hex code, that's because I'm using PostgreSQL 9.0 which, by default, uses hex output for bytea columns. If I change this to "escape", it shows normal text.

agentile’s picture

Same error presented itself when trying to install Drupal in drupal schema.

Environment:
PHP 5.3.6
PostgreSQL 8.4.6
Apache 2.2.11
Solaris 5.11 snv_130

Would be nice for this to be addressed as it breaks Drupal installation and forces installation into a public schema, which depending on the use case of Drupal may not be ideal. Thanks.

Damien Tournoud’s picture

Most likely, the ID generator doesn't play nice with schemas.

From looking at the code, it seems that we are reusing the sequence generator from the {sequences}.value serial column. This is an horrible hack, and I bet that the sequence doesn't have the name we expect when schemas are being used.

Josh Waihi’s picture

I just installed Drupal 7 fine on PHP 5.3.2-1ubuntu4.7, PostgreSQL 8.4.7 and Apache 2.2.14 under a PostgreSQL schema called "drupal". Thats the standard build on Ubuntu 10.04 LTS.

I didn't test the user workflow however. I filled out the $databases array manually and set the prefix key to "drupal.".

Can someone else try that and see if it still doesn't work for them?

dark_ixion’s picture

Josh, what PostgreSQL user name are you using for the database? This can make a difference since the default schema search path is $user,public meaning if there is a schema named the same as the user, it gets searched first without the need of a prefix.

tbarreno’s picture

Hi,


It really looks like a schema/search patch issue.


I've a Drupal 7 default installation and recently created a test site. The default configuration relies in a database schema called 'drupal' (with the same username: 'drupal'). The test site has a new schema and username ('drupaltest'). Every time I tried to install Drupal test site I got the same response (posted by dark_ixion).


The solution was specify the schema with "prefix":

$databases = array (
  'default' =>
  array (
    'default' =>
    array (
      'database' => 'webd1',
      'username' => 'drupaltest',
      'password' => '*********',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'pgsql',
      'prefix' => 'drupaltb.', // Before was empty
    ),
  ),
);

Then, the installation was successfully.


Best Wishes

Josh Waihi’s picture

This seems related to #956942: Use current postgresql schema instead of hardcoded "public". . We should be settings the search path to the correct schema on connection. I recall we were looking at this but were waiting for prefixes per connection to get in first.

See other related issues: http://drupal.org/project/issues/drupal?text=search_path&status=All

dark_ixion’s picture

Are you sure we should be setting the search path at all? First of all the prefix should take care of any problems with things not being found in the search path. And also, the case I'm reporting here is where both the user name and schema name is 'drupal', and since $user appears before 'public' in the schema search path, it should already be in there and therefore not a problem.

But from what I think you're suggesting, the prefix functionality isn't working correctly anyway, so maybe that's making it fail somewhere.

But excluding the prefix anyway, I tried something to rely on it using a schema named after the user, and I get the same error on the web installation interface, and the same error in the PostgreSQL log:

2011-05-28 18:01:45 BST ERROR:  null value in column "rid" violates not-null constraint
2011-05-28 18:01:45 BST STATEMENT:  INSERT INTO role_permission (rid, permission, module) VALUES (NULL, 'administer blocks', 'block')
2011-05-28 18:01:45 BST ERROR:  current transaction is aborted, commands ignored until end of transaction block
2011-05-28 18:01:45 BST STATEMENT:  SELECT 1 AS expression
        FROM 
        role_permission role_permission
        WHERE ( (rid IS NULL ) AND (permission = 'administer blocks') ) FOR UPDATE
dark_ixion’s picture

I think I know roughly in which area the problem lies:

In includes/database/pgsql/schema.inc there's this bit of code:

  public function queryTableInformation($table) {
    // Generate a key to reference this table's information on.
    $key = $this->connection->prefixTables('{' . $table . '}');
    if (!strpos($key, '.')) {
      $key = 'public.' . $key;
    }

That hard-coded public schema value makes its way into the real output. If, using my setup, I replace that with 'drupal.', installation is successful. Obviously that's not the solution, but it does mean that $key is not being assigned the schema-qualified table name.

dark_ixion’s picture

I've also just done a quick check of an installation with a schema name different from the user name (which, as I've previously reported, works fine), and none of the tables are in the schema specified by $db_prefix in my settings file. It is therefore assumed that the INSTALL.pgsql.txt file is misleading in that setting the $db_prefix variable will result in it being used for the schema name. This is clearly not the case.

Drupal really shouldn't assume there is a public schema. It is actually bad practise to keep the public schema around due to security reasons. Ideally, if no schema is specified, it shouldn't prefix the table at all and just follow the schema search path for that user. This would mean that if there was a user named the same as the schema, all tables would automatically go in there by default (assuming the user's search_path setting hadn't been changed from the default of "$user",public). If such a schema didn't exist, they'd go into public anyway.

dark_ixion’s picture

Oh, I should have mentioned that PostgreSQL has a current_schema function which returns the name of the first valid schema in the connection's search path. If that could be run and used to write into the prefix field in the settings file, it should always work.

Josh Waihi’s picture

Title: Drupal 7 installation fails on PostgreSQL with "drupal" schema » Drupal 8 installation fails on PostgreSQL with "drupal" schema
Version: 7.0 » 8.x-dev

To me, this issue lies in how Drupal uses prefixes rather than schemas which is a reflection of how MySQL treats schemas as databases which gives MySQL users no concept of a schema.

What that means is, a connection prefix should be renamed to a connection "schema", giving more reason to support dot syntax within a prefix. With that in place, the search_path can be set per connection based on the schema's outlined default schema if any, otherwise the search_path can be left alone.

With that in place, prefixing tables and keys would become much simpler. However, when schema details were addressed in D7, they were done before prefixes per connections were implemented

catch’s picture

Title: Drupal 8 installation fails on PostgreSQL with "drupal" schema » Multiple issues when PostgreSQL is used with non-public schema
Issue tags: +Needs backport to D7
mva.name’s picture

Hey guys! As far as I see, it is silence here for about a year. And all isses still in 8.x/7.15.
Nobody wants to fix this?

By the way, I have one more issue related to this topic:
When trying to install d7.15 in non-public schema (when another copy is already installed there) Drupal says "Drupal could not be correctly setup with the existing database. Revise any errors."
as far as I looken in code — error happens when drupal trying to create functions in global namespace. And it seems, that PgSQL server (9.1.4) doesn't allow to do that.

P.S. How can I help to speed up issue fixing? I'm very interested in fixing it asap.
//wbr,
mva

Josh Waihi’s picture

Hey mva.name! By all means, please jump in and fix the issue! I have literally been stuck on a project for the last 18 months using a MySQL database (ewww) and since my effort is focused there, I haven't been able to put in time to fix these PostgreSQL issues. More than happy to review any patches though!

ericsol’s picture

Version: 8.x-dev » 7.21
FileSize
568.64 KB

Jumping in to see if I can shed some light on this.

This thread is tagged version 8.x-dev. From the timing of the issue and the description I would assume its not. Since I can reproduce the issue in 7.21 I've changed the thread accordingly.

I've made separate installs with minimal profile and language = English with PostgreSQL 9.2:

  1. install sandbox4 in its own database with schema public
  2. install sandbox4 in schema sandbox4 in database drupal

I visually compared the two databases and saw no differences (same number of tables, same records in system, menu_router and so on).

In the next step I've made a dump of both schema's:

pg_dump -U sandbox4 --schema= > sandbox4<-db>.psql

Next step was to find/replaced all instances of "Schema: public" and "public." with "Schema: sandbox4" and "sandbox." in sandbox4-db.psql to rule out all differences in the dump caused by the name of the schema.

After that I compared the two dumps: diff sandbox4.psql sandbox4-db.psql > sandboxdiff.txt. I've attached the file for further inspection.

My initial finding is there are no meaningful differences between the dumps; the binary data being shown is probably an artefact of diff.

All this leads me to the assumption we might be looking in the wrong direction: this is not an installation issue but some other bug. The most likely cause is somewhere in menu_router, since paths like /admin produce WSOD and my error log points to line 2127 in system.module:

  if (!isset($item['mlid'])) {
    $item += db_query("SELECT mlid, menu_name FROM {menu_links} ml WHERE ml.router_path = :path AND module = 'system'", array(':path' => $item['path']))->fetchAssoc();
  }

Any thoughts or comments?

ericsol’s picture

Status: Active » Closed (fixed)

To follow up on my post:

All of the reported problems disappeared when I changed the prefix from .

. to . I was experimenting with having user-related tables in one Drupal installation to share users across multiple sites. I now have all of this working through the use of the following database declaration in settings.php:
$databases['default']['default'] = array (
      'database' => 'drupal',
      'username' => 'sandbox',
      'password' => 'sandbox',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'pgsql',
      'prefix' => array(
     	'default'   => 'sandbox.',
     	'users'     => 'profiles.',
     	'users'     => 'profiles.',
     	'users_roles'     => 'profiles.',
     	'sessions'  => 'profiles.',
     	'role'      => 'profiles.',
     	'authmap'   => 'profiles.',
   		)
	);
I can now have multiple sites in one PostgreSQL database with shared user credentials! Marking this as closed fixed.
Anonymous’s picture

I've had the same error with mixed-case schema name where postgres would mangle it to become all lowercase. Making it lowercase in the Drupal settings fixed it.

JimmyAx’s picture

Version: 7.21 » 8.x-dev
Status: Closed (fixed) » Active

Why is this issue closed? I'm having issues with my PostgreSQL Drupal site migrated from MySQL. Other sites migrated from MySQL using the schema "public" are not having issues.

One issue I'm having is my tables not dropping when I uninstall modules. I basically tracked it down to db_table_exists() which ends up in DatabaseSchema::getPrefixInfo() that is using the hardcoded value defaultSchema being set to exactly "public". This can not be changed in any way but hacking core.

stefan.r’s picture

Component: postgresql database » postgresql db driver
Status: Active » Needs review
FileSize
725 bytes

Status: Needs review » Needs work

The last submitted patch, 29: 1060476-pgsql_public_schema-29.patch, failed testing.

stefan.r’s picture

Status: Needs work » Needs review
FileSize
742 bytes
bzrudi71’s picture

Patch fails with:
Fatal error: Call to undefined method Drupal\Core\Database\Schema::construct() in /var/www/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php on line 36

If I remember right, construct() moved over to Connection.php?

Berdir’s picture

Status: Needs review » Needs work

It means that it should be parent::__construct() :)

jaredsmith’s picture

I tried changing the patch to call parent::__construct(), but that caused other problems. I'll post the patch here just for reference, but I really don't expect it to solve the problem.

bzrudi71’s picture

Yep, I think there are more hardcoded 'public' vars around and this could help but will not finally make Drupal work outside public schema. Also I don't think that this is required any more to pass tests, at least for the remaining once other than migrate - but that is thing of it's own to fix ;-) Thanks anyway @jaredsmith for the patch.

daffie’s picture

Status: Needs work » Needs review
daffie’s picture

FileSize
1.89 KB
2.14 KB

Changed the hardcoded 'public' vars for the class variable $defaultSchema.

daffie’s picture

FileSize
2.14 KB
1.28 KB

Oeps. The class variables go first!

bzrudi71’s picture

Status: Needs review » Needs work

Sorry, back to needs work ;-) I just installed D8 outside public schema, it works with, or without the patch. However, I did just a quick test run of the Database test group and there are still fails and exceptions with patch applied.

catch queued 38: 1060476-38.patch for re-testing.

The last submitted patch, 38: 1060476-38.patch, failed testing.

BORANBURCIN’s picture

Hi,

I am using Drupal 8.0.2. I cannot install on non-public schema. Have you guys installed?

just_A_boy’s picture

@BORANBURCIN Is this still being the case.. Same problem here on 8.0.3

ozden_meren’s picture

Hi,

Same problem with drupal-8.0.5. I tried to create a patch file, but I don't have much experience about patch files. I hope it's in the correct format.

andypost’s picture

Status: Needs work » Needs review

Patch looks right, let's see what testing will tell

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

daffie’s picture

Version: 8.1.x-dev » 8.2.x-dev
Status: Needs review » Needs work
Issue tags: +Needs tests

@ozden_meren: Your patch looks good. Thanks.

I think that we need to combine the patches from #38 and #44. We also need some tests to make sure that the problems are fixed and stay fixed.

David_Rothstein’s picture

Version: 8.2.x-dev » 8.1.x-dev

Not sure why this can't be fixed in 8.1.x?

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

RoSk0’s picture

Status: Needs review » Needs work

The last submitted patch, 50: drupal-1060476-50.patch, failed testing.

RoSk0’s picture

daffie’s picture

Status: Needs review » Needs work

The patch looks good. I do have some remarks:

  1. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -46,6 +46,15 @@ class Schema extends DatabaseSchema {
    +    $this->defaultSchema = $prefix_info['schema'];
    

    Can we default to "public" if $prefix_info['schema'] is empty.

  2. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -101,9 +110,9 @@ public function queryTableInformation($table) {
    -    else {
    +    elseif (strpos($table, 'db_temporary_')) {
    

    Why is this change necessary?

  3. The method Connection:: getFullQualifiedTableName() has a hard coded ‘public’.

  4. We will also need some tests to make sure that all drupal works with a non-public schema.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Renrhaf’s picture

Hello, I had the same issue trying to install a D8 8.3-x with a PostGreSQL schema different from the "public" one.
Using the patch in #52 works great, but due to short array notation change, this patch can't be applied automatically anymore.
So here the patch rerolled on D8 8.4-x. and some answers :

1. Default to "public" is already happening because of protected $defaultSchema = 'public'; in Schema.php
2. Don't really know but there is no "else" statement anymore and it seems a bit strange.
3. I tried to do something in the "drupal-1060476-55-bis.patch". The getFullQualifiedTableName method seems to be called in migrations only. It can be called and tested like that \Drupal::service('database')->getFullQualifiedTableName('TABLE');
4. D8 8.4-x fresh install works properly.

Renrhaf’s picture

Status: Needs work » Needs review
Renrhaf’s picture

I faced some issues with the command "drush entity-updates", on field deletions, and more particularly on index renaming.
The new patch includes a fix for this too.

Renrhaf’s picture

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

RoSk0’s picture

Assigned: Unassigned » RoSk0

I think I finally figured it out, updated patch to follow soon...

RoSk0’s picture

Changes:

  1. Updated Connection::getFullQualifiedTableName() method for PostgreSQL.
  2. Made Schema::getPrefixInfo() method public because it has crucial functionality which is not accessible from Connection.
  3. Made Sql::getQualifiedMapTableName() use not fully qualified table name because PostgreSQL doesn't support cross DB joins. Map table is joinable only in same DB per SqlBase::mapJoinable()
  4. Minor code style/docs fixes.

Status: Needs review » Needs work

The last submitted patch, 62: drupal-1060476-62.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 64: drupal-1060476-64.patch, failed testing. View results

RoSk0’s picture

Status: Needs review » Needs work

The last submitted patch, 66: drupal-1060476-66.patch, failed testing. View results

RoSk0’s picture

RoSk0’s picture

Assigned: Unassigned » RoSk0

Still on it, looks like we overlooked identifier name quoting in some places...

RoSk0’s picture

Slightly reworked version.

Main changes from previous:

* Removed strpos() detection of schema where possible, replaced with Schema::getPrefixInfo()
* Used Connection::escapeTable() for identifier quoting

Status: Needs review » Needs work

The last submitted patch, 70: drupal-1060476-70.patch, failed testing. View results

RoSk0’s picture

RoSk0’s picture

Added small performance improvement and made unit test a bit more explicit.

AnaSwin’s picture

Thanks a lot for the last patch (drupal-1060476-73.patch)!
I applied it on my website (Drupal 4.0 - PostGreSQL 9.4.7 - PHP 5.6). No regression seen yet.
Thanks for your work.

daffie’s picture

The patch looks good. But what I am missing is tests to make sure that the problem is fixed and to make sure that future patches do break the support for non-public schema's. Kerneltest will be diffecult to do with the current testbot. So my suggestion will be to create unittests. Let the PostgreSQL driver calculate the query string and test if that will work for different schema's (public, temporary and others). Do that for SELECT, UPDATE, DELETE, INSERT and the different ALTER queries. Create the test with provider methods. See: https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writ....

+++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
@@ -45,6 +45,19 @@ class Schema extends DatabaseSchema {
+    // Retrieve the max identifier length which is usually 63 characters
+    // but can be altered before PostgreSQL is compiled so we need to check.
+    $this->maxIdentifierLength = $this->connection->query("SHOW max_identifier_length")->fetchField();

Please, create a separate issue for this.

RoSk0’s picture

Thanks for your review @daffie.

I don't think that writing tests for different query types will help. They all using Connection::prefixTables so I tried to test this. Lets see what testbot will say.

Removed max_identifier_length retrieval change as requested and added prefixTable test.

daffie’s picture

Status: Needs review » Needs work

@RoSk0: Lets look at this in a different way. Lets say that this patch fixes all non default schema problems for PostgreSQL. What parts of this patch can later be changed back without the testbot generating any fails. The only security we have is the testbot. So we need tests for ALL the changes that make drupal core work on non default schema's.

RoSk0’s picture

Reverted testComputedConstraintName changes related to #2921665: PostgreSQL: Optimize getting maxIdentifierLength.

@daffie Thanks for the suggestion, but I'm struggling to figure it out.
There is a major issue with testing this patch, at least in my understanding, major changes are in
Schema::queryTableInformation() and Schema::renameTable(). In Schema::queryTableInformation() we fixing logic and to test the outcome we need to actually talk to DB. In Schema::renameTable() we adding schema name to queries and same thing with testing only possible by real DB queries.

So in my opinion we only can test it indirectly by testing Connection::getFullQualifiedTableName() which has hard-coded
"public" schema at the moment and producing "drupal8-contrib.public.drupal.cache" instead of "drupal8-contrib.drupal.cache".

I'm open to suggestions on how to improve test coverage for this.

RoSk0’s picture

Status: Needs work » Needs review

Wake up testbots

RoSk0’s picture

RoSk0’s picture

Status: Needs review » Needs work

The last submitted patch, 78: drupal-1060476-78.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

RoSk0’s picture

Assigned: Unassigned » RoSk0

Ok, so my assumption that testComputedConstraintName will work was wrong, fixing...

RoSk0’s picture

RoSk0’s picture

Status: Needs review » Needs work

The last submitted patch, 85: drupal-1060476-85-test-only.patch, failed testing. View results

RoSk0’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests

Setting back to "Needs review" as tests passed, removing "Needs test" tag as tests were added.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Renrhaf’s picture

I confirm that the patch works properly.
Patching Drupal 8.5.3 and using a non-public schema (multisite with one schema per site) on PostGreSQL 9.5 works with #85.
Without the patch, the installation fails when using prefixes like "siteA.", "siteB.", etc.
Thanks RoSK0 !

Status: Needs review » Needs work

The last submitted patch, 85: drupal-1060476-85-test-only.patch, failed testing. View results

daffie’s picture

The patch looks good and @Renrhaf has manual tested the patch with a non-public schema. That is great. I do have 2 minor remarks about this patch:

  1. The testbot is failing Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest.php for MySQL. This is wrong and needs to be fixed. See: https://www.drupal.org/pift-ci-job/805984.
  2. You are making a lot of changes to the test Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest. I find it difficult to see what changes are necessary for this patch and what changes are for making the test more pretty. Could you remove are the make the test more pretty changes.

@RoSkO: Great work on this patch!

We do not have a subsystem maintainer any more after @crell left. So changing it to the framework maintainer.

mradcliffe’s picture

I think that test failure is using the test-only patch and not the real patch in #85, @daffie.

diff --git a/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
index 6bdd51ebf0..77ccbf659e 100644
--- a/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
+++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
@@ -265,7 +265,23 @@ public function messageTableName() {
    *   The fully qualified map table name.
    */
   public function getQualifiedMapTableName() {
-    return $this->getDatabase()->getFullQualifiedTableName($this->mapTableName);
+    $qualifiedMapTableName = NULL;
+    $connection = $this->getDatabase();
+
+    switch ($connection->databaseType()) {
+      case 'pgsql':
+        // PostgreSQL doesn't support cross DB merges. So only use schema and
+        // prefixed table name.
+        $info = $connection->schema()->getPrefixInfo($this->mapTableName);
+        $qualifiedMapTableName = $info['schema'] . '.' . $info['table'];
+        break;
+
+      default:
+        $qualifiedMapTableName = $connection->getFullQualifiedTableName($this->mapTableName);
+        break;
+    }
+
+    return $qualifiedMapTableName;
   }

I think that this may get rejected on further review as it's been frowned upon to add in driver-specific options in core _not_ in the driver itself. I think it would be possible to work around this by adding an optional parameter to getFullQualifiedTableName?

diff --git a/core/tests/Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest.php b/core/tests/Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest.php
@@ -44,23 +44,34 @@ protected function setUp() {
     $statement = $this->getMock('\Drupal\Core\Database\StatementInterface');
     $statement->expects($this->any())
       ->method('fetchField')
-      ->willReturn($max_identifier_length);
+      ->will($this->onConsecutiveCalls($max_identifier_length, 1));

I don't think this is related to the patch, @daffie. I think this and some of the changes in PostgresqlSchemaTest are because there are additional calls to fetchField for checking indexes. It is because the old code should pass because the query that checks for indexes is truthy so it "works".

daffie’s picture

The testbot fixed my first remark from comment #92. For my second remark, I did take the time to have another good look at the test and I understand the changes now, but I do have a new point about the test:

+++ b/core/tests/Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest.php
@@ -44,23 +44,34 @@ protected function setUp() {
+    $this->connection->expects($this->any())
+      ->method('escapeTable')
+      ->willReturnMap([
+        ['user_field_data____pkey' , '"user_field_data____pkey"'],
+        ['user_field_data__name__key', '"user_field_data__name__key"'],
+        ['drupal_BGGYAXgbqlAF1rMOyFTdZGj9zIMXZtSvEjMAKZ9wGIk_key', '"drupal_BGGYAXgbqlAF1rMOyFTdZGj9zIMXZtSvEjMAKZ9wGIk_key"'],
+      ]);

I have a problem with this part. The entire idea of testing with a provider method is that you can add extra testcases to the provider method,
without changing the main test method. Only by adding the above codeblock, that is no longer possible. I do not think we can remove this codeblock from the test method. Therefor my suggestion would be to change this test to a kernel test. There you have a normal database connection and there is no need mocking used classes.

kattekrab’s picture

Looks like this might be the bug that bit me today.

daffie’s picture

Looks like this might be the bug that bit me today.

@kattekrab: Would you be able to test the patch from comment #85 and see if it fixes your problem. User @Renrhaf already confirmed that the patch worked for him. The only work on this patch is a test. The rest of this patch should work.

beram’s picture

Hi,

I confirm that the patch from comment #85 is working on a non-public schema with Drupal 8.5.3.
Tested on multiple environments.

mradcliffe’s picture

I agree that a kernel test would be useful. I confirmed in drupalci_environments that the database user is the database owner, however this may not always be the case for local development / testing environments. So if we're okay with possibly failing test runs on non-drupal.org systems, then that would make sense.

The current unit test could be modified to use the data provider because the test arguments are the same as what we're mocking, @daffie.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

RoSk0’s picture

Status: Needs work » Needs review

Back to "Needs review".

I don't see any benefit in conversion to Kernel test unless we actually will test on non-public schema, but as far as I can tell we can't specify what schema to use when testing on PostgreSQL.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Renrhaf’s picture

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

ravi.shankar’s picture

Status: Needs work » Needs review
FileSize
16.94 KB

Here I have re-rolled the patch.

daffie’s picture

Issue tags: -Needs reroll

@ravi.shankar: Thanks for the reroll.

daffie’s picture

Status: Needs review » Postponed
Related issues: +#3104007: Postgresql 12 DrupalCI environment needed for core testing

To be able to test a patch for this issue a new testbot with a non-public schema is needed. Postponing untill there is such a testbot. See: #3104007: Postgresql 12 DrupalCI environment needed for core testing.

mradcliffe’s picture

+++ b/core/tests/Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlConnectionTest.php
@@ -116,4 +116,65 @@ public function testEscapeField($expected, $name) {
+    $pgsql_connection   = new Connection($this->mockPdo, $connection_options);
...
+    $pgsql_connection   = new Connection($this->mockPdo, $connection_options);

There's an extra space on the left-hand side of the assignment operator.

ressa’s picture

I got this before patching:

$ drush st
In Connection.php line 695:
  SQLSTATE[42602]: Invalid name: 7 ERROR:  invalid name syntax
  LINE 6: AND pg_attribute.attrelid = '.my_postgres_db.cache_config'::regcl...
                                        ^: SELECT pg_attribute.attname AS column_name, format_type(pg_attribute.atttypid, pg_attribute.atttypmod) AS data_type, pg_get_expr(pg_attrdef.adbin, pg_attribute.attrelid) AS column_default
  FROM pg_attribute
...
In Statement.php line 59:

  SQLSTATE[42602]: Invalid name: 7 ERROR:  invalid name syntax
  LINE 6: AND pg_attribute.attrelid = '.my_postgres_db.cache_config'::regcl...

Applying the patch 1060476-105.patch fixes it.

Also, if I try to define the prefix with a traling dot during installation (in Lando) like this drush site:install --db-url=pgsql://postgres:@database/drupal8 --db-prefix=my_postgres_db. -y I get an error:

In install.core.inc line 1182:
To start over, you must empty your existing database and copy <em>default.settings.php</em> over.

So I have to run drush site:install with --db-prefix=my_postgres_db and manually add the dot to the prefix in settings.php afterwards, like this:

$databases['default']['default'] = array (
  'database' => 'drupal8',
  'username' => 'postgres',
  'password' => '',
  'prefix' => 'my_postgres_db.',
  'host' => 'database',
  'port' => '',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\pgsql',
  'driver' => 'pgsql',
);
daffie’s picture

Version: 8.9.x-dev » 9.1.x-dev
Status: Postponed » Needs work

We have now testbots for PostgreSQL 10 and 12. Both have an extra schema called: testing_fake. Because all PostgreSQL testbots must pass the added testing, I move this issue to 9.1.

ressa’s picture

To my luck, the table in the database-dump I was importing has since changed from my_postgres_db to the generic public, which made defining prefix with 'prefix' => 'my_postgres_db.', no longer necessary.

daffie’s picture

Issue tags: +Bug Smash Initiative
ridhimaabrol24’s picture

Status: Needs review » Needs work

The last submitted patch, 113: 1060476-113.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

RoSk0’s picture

In reply to #109: I believe your issue is related to bug in Drush described here https://github.com/drush-ops/drush/issues/1523 with fix here https://github.com/drush-ops/drush/pull/2567 .

ressa’s picture

Thanks for working on supporting non-public schema for PostgreSQL, and sharing the Github issue @RoSk0! I hope it gets committed at some point.

RoSk0’s picture

Still needs work to address:

  • #108
  • convert test to functional and use real DB as per #94 (see also #107 & #110)

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

intrafusion’s picture

FileSize
3.4 KB

#113 doesn't apply cleanly to 9.1.8, lib/Drupal/Core/Database/Driver/pgsql/Schema.php.rej is attached but I can't see any reason why this doesn't apply

daffie’s picture

Hi @intrafusion, that you for the reroll!. At the moment is your patch not working. A patch file should have the extension ".patch". Also when you are working with multiple patches it helps when the naming of the patch has the issue ID in it followed by its comment ID. See: https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa.... With which Drupal core branch are you working?

intrafusion’s picture

@daffie, it's not my patch, it's the patch from #113

I posted the rejected file that was generated when I tried to apply the patch. I've compared the line by line changes and cannot understand why the patch doesn't apply correctly

vsujeetkumar’s picture

Status: Needs work » Needs review
FileSize
15.63 KB

Re-roll patch given for 9.3.x.

vsujeetkumar’s picture

Fixing 'custom command fails' Issues.

quietone’s picture

Status: Needs review » Needs work
Issue tags: +Needs issue summary update
FileSize
5.96 KB
4.35 KB

Reviewing Bug Smash issues. I don't know db drivers well so can not comment on the actual fix or the tests here. I did not try to the steps to reproduce because they are for Drupal 7. The Issue Summary does not provide a list of tasks for a proposed resolution. So, even if I knew more about db drivers it is not possible to know if this patch is implementing the recommended fix. Tagging for issue summary update.

There were several rerolls since #105, none of which provided a diff. Those diffs are added.
@ridhimaabrol24 and @vsujeetkumar It is good practice to always add a interdiff or diff. It makes the task of the reviewer much easier.

  1. +++ b/core/lib/Drupal/Core/Database/Driver/mysql/Schema.php
    @@ -44,10 +44,10 @@ class Schema extends DatabaseSchema {
    +   * @return array
    

    Out of scope change

  2. +++ b/core/lib/Drupal/Core/Database/Driver/mysql/Schema.php
    @@ -44,10 +44,10 @@ class Schema extends DatabaseSchema {
    +  public function getPrefixInfo($table = 'default', $add_prefix = TRUE) {
    

    Why this change? Why does mysql have to change?

  3. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -202,14 +212,12 @@ protected function getTempNamespaceName() {
    +   * @param string $table
    

    Out of scope

  4. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -461,7 +469,7 @@ public function getFieldTypeMap() {
    +    ];
    

    Out of scope

  5. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -727,7 +741,12 @@ public function indexExists($table, $name) {
    +    $schema = $info['schema'];
    +    $table_name = $info['table'];
    +    return (bool) $this->connection->query("SELECT 1 FROM pg_indexes WHERE schemaname = :schema AND tablename = :table AND indexname = :index", [':schema' => $schema, ':table' => $table_name, ':index' => $index_name])->fetchField();
    

    Line too lone, put the array on multiple lines. And we can avoid the two variables if we use $info['schema'] and $iinfo['table'] in the query string.

  6. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -850,7 +869,13 @@ public function dropIndex($table, $name) {
    +    // Add the schema if not using the default one.
    

    When there is an 'if' in a comment there is usually one in the code as well. What happens here when the default schema is used? It is not clear to me.

  7. +++ b/core/lib/Drupal/Core/Database/Schema.php
    @@ -81,7 +81,7 @@ public function nextPlaceholder() {
    +  public function getPrefixInfo($table = 'default', $add_prefix = TRUE) {
    

    As above why is this needed?

  8. +++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
    @@ -279,7 +279,23 @@ public function messageTableName() {
    +    $qualifiedMapTableName = NULL;
    

    Not needed, the switch has a default case where it is set.

daffie’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Updated the IS.

@quietone: Thank you for the feedback!

daffie’s picture

Issue summary: View changes
mogtofu33’s picture

Updated patch for last 9.3.x (Test class didn't apply).
Including quietone remarks.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

intrafusion’s picture

In relation to #7 in #1060476-125: Multiple issues when PostgreSQL is used with non-public schema by not making getPrefixInfo public I get the following error when using migrations:

Error: Call to protected method Drupal\Core\Database\Schema::getPrefixInfo() from context 'Drupal\migrate\Plugin\migrate\id_map\Sql' in Drupal\migrate\Plugin\migrate\id_map\Sql->getQualifiedMapTableName() (line 288 of core/modules/migrate/src/Plugin/migrate/id_map/Sql.php) #0 modules/contrib/migrate_tools/src/MigrateExecutable.php(128): Drupal\migrate\Plugin\migrate\id_map\Sql->getQualifiedMapTableName()

Changing this function from protected to public fixes this error and allows the migrations to complete

intrafusion’s picture

Patch attached which adds back #7 from #125, no idea how to create an interdiff

Also re-rolled for 9.3 as it was no longer applying cleanly, no idea about 9.4 yet

daffie’s picture

The tests from the proposed solution still needs to be added.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

tostinni’s picture

Last patch wasn't applying for 9.4 due to this file that has been moved core/tests/Drupal/Tests/Core/Database/Driver/pgsql/PostgresqlSchemaTest.php.

Updating patch

tostinni’s picture

Fix incorrect patch uploaded for 9.4

daffie’s picture

@tostinni: Great to see that you are working on this issue. I shall help by reviewing your work. You can contact me on drupal.slack.com in the #contribute channel. My username there is the same as here.

The first goal is to get this fixed in 10.1.x. Later we maybe able to backport this to earlier Drupal versions.
The testbot for PostgreSQL has a special schema to make this issue testable and the schema name is testing_fake.
The issue summery has a list of tests that are needed. It is a long list, only they are all very simple test. It is less work than it looks.

I have added the tag "Needs framework manager review" 4 years ago and I added it, because there was no subsystem maintainer for the Database API. As I am now that subsystem maintainer and I added the tag, makes me free to remove it. Not adding back the tag "Needs subsystem maintainer review", because as that subsystem maintainer, I think this issue should be fixed.

jordan.jamous’s picture

Thanks all. I am getting this error with #138.

PHP Fatal error:  Access level to Drupal\mysql\Driver\Database\mysql\Schema::getPrefixInfo() must be public (as in class Drupal\Core\Database\Schema) in web/core/modules/mysql/src/Driver/Database/mysql/Schema.php on line 50

Updating the patch for 9.4.x

Arantxio’s picture

I created a new patch based on the patch from #141 which should be working on 10.1.x dev. This patch contains most or all test in the proposed resolution and is made in collaboration with @daffie.

Arantxio’s picture

Status: Needs work » Needs review
daffie’s picture

Status: Needs review » Needs work

First quick review. The patch look good!

  1. +++ b/core/lib/Drupal/Core/Database/Connection.php
    @@ -774,6 +781,7 @@ public function query($query, array $args = [], $options = []) {
    +
    

    This added line can be removed.

  2. +++ b/core/lib/Drupal/Core/Database/Schema.php
    @@ -81,7 +81,7 @@ public function nextPlaceholder() {
    -  protected function getPrefixInfo($table = 'default', $add_prefix = TRUE) {
    +  public function getPrefixInfo($table = 'default', $add_prefix = TRUE) {
    

    This change is not necessary any more. We have a new function called getPrefix() we can use instead.

  3. +++ b/core/modules/migrate/src/Plugin/migrate/id_map/Sql.php
    @@ -298,7 +298,22 @@ public function messageTableName() {
    -    return $this->getDatabase()->getFullQualifiedTableName($this->mapTableName);
    +    $connection = $this->getDatabase();
    +
    +    switch ($connection->databaseType()) {
    +      case 'psql':
    +        // PostgreSQL doesn't support cross DB merges. So only use schema and
    +        // prefixed table name.
    +        $info => $connection->schema()->getPrefixInfo($this->mapTableName);
    +        $qualifiedMapTableName = $info['schema'] . '.' . $info['table'];
    +        break;
    +
    +      default:
    +        $qualifiedMapTableName = $connection->getFullQualifiedTableName($this->mapTableName);
    +        break;
    +    }
    +
    +    return $qualifiedMapTableName;
    

    This change can be removed. I am not sure if it is necessary. Therefor remove it for now.

  4. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -73,6 +73,10 @@ class Connection extends DatabaseConnection implements SupportsTemporaryTablesIn
    +    // We need to set the connectionOptions before the parent,
    +    // because setPrefix needs this.
    

    Let the comment run until 80 characters.

  5. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -309,12 +314,11 @@ public function nextId($existing = 0) {
    -    $prefix = $this->getPrefix();
    +    $info = $this->schema()->getPrefixInfo($table);
    

    Lets use getPrefix() instead of getPrefixInfo(). Please undo this change.

  6. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -309,12 +314,11 @@ public function nextId($existing = 0) {
    +    return $options['database'] . '.' . $info['schema'] . '.' . $info['table'];
    

    Lets get the schema from the connection $options.

  7. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -50,6 +50,24 @@ class Schema extends DatabaseSchema {
    +
    +
    +    $this->defaultSchema = $connectionInfo['schema'] ?? 'public';
    

    Please remove one of the empty lines and add a comment with: "If the schema is not set in the connection options than schema defaults to public".

  8. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -89,12 +107,12 @@ protected function ensureIdentifiersLength($table_identifier_part, $column_ident
    -      $saveIdentifier = '"drupal_' . $this->hashBase64($identifierName) . '_' . $tag . '"';
    +      $saveIdentifier = 'drupal_' . $this->hashBase64($identifierName) . '_' . $tag;
    

    This change is unrelated to what we are doing in this patch (out of scope). There for please remove this change.

  9. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -135,6 +155,8 @@ public function queryTableInformation($table) {
    +
    +
    

    These 2 empty lines can be removed.

Arantxio’s picture

Created a new patch because the other one failed, and added the comments from @daffie in comment #144.

asad_ahmed’s picture

Worked on points 1, 3, 5, 7, 8 and 9 of #144. Please review and thanks

Arantxio’s picture

Forgot to add :ignore to cSpell for the ignored words.

Arantxio’s picture

Created a new patch for the failing tests, also assigned to myself as I will be working on this as much as possible with @daffie.

Arantxio’s picture

A new patch to try and fix the failing tests.

Arantxio’s picture

I had a double semicolon on one line, which caused the phpcs test to fail.

daffie’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    // Test the method for a non existing extension.
    

    Can we change this to: "Create a connection to the non-public schema."

  2. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $this->connection = Database::getConnection('testing_fake', 'default');
    

    Can we rename the variable to $this->testingFakeConnection

  3. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $this->connection = Database::getConnection('default', 'default');
    

    This line can be removed.

  4. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    Database::removeConnection('testing_fake');
    

    Can we, before we remove the connection assert that all tables have been dropped. "$this->assertEmpty($this->testingFakeConnection->schema()->findTables('%'));"

  5. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +  protected function createSampleData(): void {
    +    $this->connection->insert('faking_table')
    +      ->fields(
    +        [
    +          'id' => '1',
    +          'test_field' => '123',
    +        ]
    +      )->execute();
    +
    +    $this->connection->insert('faking_table')
    +      ->fields(
    +        [
    +          'id' => '2',
    +          'test_field' => '456',
    +        ]
    +      )->execute();
    +
    +    $this->connection->insert('faking_table')
    +      ->fields(
    +        [
    +          'id' => '3',
    +          'test_field' => '789',
    +        ]
    +      )->execute();
    +  }
    

    You can move the code from this method the setUp(). It is not much code and it removes a method.

  6. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    + * Tests schema API for the PostgreSQL driver.
    

    Can we change this to "Tests schema API for non-public schema for the PostgreSQL driver"

  7. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +  public function testPgsqlExtensionExists(): void {
    

    Can we rename the method to "testExtensionExists()". The pgsql part is not necessary. We are already in the Drupal\pgsql namespace. The same for the other method names.

  8. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    + * @group Database
    + *
    +class NonPublicSchemaTest extends DriverSpecificKernelTestBase {
    

    Can we add the line " * @coversDefaultClass \Drupal\pgsql\Driver\Database\pgsql\Schema"

  9. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +   * @covers \Drupal\Core\Database\Driver\pgsql\Schema::extensionExists
    

    Can we change this line to "* @covers ::extensionExists". We have already stated add the top of the class which class we are testing with @coversDefaultClass. For more info: https://phpunit.readthedocs.io/en/9.5/annotations.html#coversdefaultclass

  10. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $this->assertIsInt($result);
    

    This line can be removed.

  11. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $this->assertGreaterThanOrEqual(2, $result, 'The result of the upsert operation should report that at least two rows were affected.');
    

    Can we change the assertion to: "$this->assertSame()".

  12. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $oldTableExists = $this->connection->schema()->tableExists('faking_table');
    +    $newTableExists = $this->connection->schema()->tableExists('new_faking_table');
    ...
    +    $this->assertFalse($oldTableExists);
    +    $this->assertTrue($newTableExists);
    ...
    +    $newTableExists = $this->connection->schema()->tableExists('new_faking_table');
    ...
    +    $this->assertFalse($newTableExists);
    
    +++ b/core/modules/pgsql/tests/src/Unit/SchemaTest.php
    @@ -51,20 +51,32 @@ public function testComputedConstraintName($table_name, $name, $expected) {
           ->method('fetchField')
    

    Please do not use variable names. Use $this->assertFalse($this->connection->schema()->tableExists('some_table'));

  13. +++ b/core/modules/pgsql/tests/src/Kernel/pgsql/NonPublicSchemaTest.php
    @@ -0,0 +1,369 @@
    +    $results = $this->connection->query("SELECT * FROM pg_indexes WHERE schemaname = 'testing_fake'")->fetchAll();
    +
    +    $this->assertCount(0, $results);
    

    Please do this on a single line.

Arantxio’s picture

Status: Needs work » Needs review
FileSize
24.77 KB
31.42 KB

Changed getPrefixInfo back to protected, removed unnecessary changes. improved some coding standards.
Adjusted the new test with the review points from @daffie in comment #151.

daffie’s picture

  1. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -549,37 +554,45 @@ public function renameTable($table, $new_name) {
    +      $index_short_name = $index->indexname;
    

    Can we do with the variable $index_short_name and use $index->indexname instead?

  2. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -549,37 +554,45 @@ public function renameTable($table, $new_name) {
    +      // Append the schema to the index name.
    +      $index_short_name = $index->indexname;
    +      if ($this->defaultSchema != 'public') {
    +        $index_full_name = $this->defaultSchema . '.' . $index_short_name;
    +        $index_full_name = $this->connection->escapeTable($index_full_name);
    +      }
    

    Why is this change necessary? Does the schema name really need to be added here?

  3. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -835,7 +855,7 @@ public function dropIndex($table, $name) {
    -    $this->connection->query('DROP INDEX ' . $this->ensureIdentifiersLength($table, $name, 'idx'));
    +    $this->connection->query('DROP INDEX ' . $this->defaultSchema . '.' . $this->ensureIdentifiersLength($table, $name, 'idx'));
    

    Is this change really necessary? It feels wrong to me to add the schema name to an index. I am also worried that it will break existing Drupal sites.

daffie’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -549,37 +554,45 @@ public function renameTable($table, $new_name) {
    -      $this->connection->query('ALTER INDEX "' . $index->indexname . '" RENAME TO ' . $this->ensureIdentifiersLength($new_name, $index_name, $index_type) . '');
    

    Can we change this line to: $this->connection->query('ALTER INDEX "' . $this->defaultSchema . '"."' . $index->indexname . '" RENAME TO ' . $this->ensureIdentifiersLength($new_name, $index_name, $index_type) . '');.
    In this way we always add the schema name. The part with $index full name can than be removed and we can use $index->indexname instead of the variable $index_short_name.

  2. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -549,37 +554,45 @@ public function renameTable($table, $new_name) {
    -    $prefixInfo = $this->getPrefixInfo($new_name);
    +    $newPrefixInfo = $this->getPrefixInfo($new_name);
    
    @@ -601,7 +614,7 @@ public function renameTable($table, $new_name) {
    -    $this->connection->query('ALTER TABLE {' . $table . '} RENAME TO ' . $prefixInfo['table']);
    +    $this->connection->query('ALTER TABLE {' . $table . '} RENAME TO ' . $newPrefixInfo['table']);
    

    These changes are not necessary.

Arantxio’s picture

I've added the changes from comment #154. We still have to check why 1 test is failing in: "Drupal\Tests\pgsql\Kernel\pgsql\SchemaTest", or rather why the last insert in the function "testChangePrimaryKeyToSerial()" is failing in "tests/Drupal/KernelTests/Core/Database/DriverSpecificSchemaTestBase.php"

Arantxio’s picture

This new patch does not fix the issue, but it has some improvements for better clarity of the code.

Also added both interdiff's as I've forgotten to add it to the past patch.

Arantxio’s picture

FileSize
24.21 KB
610 bytes

Forgot to remove a commented line for the issue I've been working on.

Arantxio’s picture

Some extra test to see what fails when we change the key to get the table information

Arantxio’s picture

Created a new patch that seems to fix the last issue.
Made the diff with #157, as the previous patches were to check where it would fail.

Arantxio’s picture

Status: Needs work » Needs review
Arantxio’s picture

FileSize
24.75 KB

One space too many on a if statement.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

All code changes look good to me.
There is testing been added.
After @arantxio talked to @catch on Slack to create a followup for the installer part and the settings.php part.
To me it is the best solution to create CR in the followup.
The followup is #3328215: Add option to change the PostGreSQL Schema during installation.
For me is it RTBC.

larowlan’s picture

Status: Reviewed & tested by the community » Needs work

Thanks for resurrecting this one folks, here's some observations, disclaimer I haven't used postgres for 15 years, but I think the observations are generic

  1. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -84,6 +88,26 @@ public function __construct(\PDO $connection, array $connection_options) {
    +    // Add the schema name of it is not set to public, else it will use the
    

    I think 'of it' should be 'if it'

    And probably 'else' should be 'otherwise'

  2. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -84,6 +88,26 @@ public function __construct(\PDO $connection, array $connection_options) {
    +    if (isset($this->connectionOptions['schema']) && ($this->connectionOptions['schema'] != 'public')) {
    

    nit: !==

  3. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -309,12 +333,11 @@ public function nextId($existing = 0) {
    +    // <database>.<schema>.<table>.
    +    return $options['database'] . '.' . $schema . '.' . $this->getPrefix() . $table;
    

    personal preference, but I think this would be easier to read with sprintf so its clear the . is part of the string and not used for string concatenation

  4. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -50,6 +50,17 @@ class Schema extends DatabaseSchema {
    +    // If the schema is not set in the connection options than schema defaults
    

    than => then

  5. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -50,6 +50,17 @@ class Schema extends DatabaseSchema {
    +    $this->defaultSchema = $connection->getConnectionOptions()['schema'] ?? 'public';
    

    There's a few places this gets used in raw SQL (i.e. via string concatenation), should we sanitize it here or throw an exception if it contains special characters?

  6. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -116,16 +127,21 @@ protected function ensureIdentifiersLength($table_identifier_part, $column_ident
    +      $key = $quoted_key = $this->getTempNamespaceName() . '.' . $prefixed_table;
    

    should the quoted key be quoted or can we guarantee that the temporary namespace name is safe?

  7. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -116,16 +127,21 @@ protected function ensureIdentifiersLength($table_identifier_part, $column_ident
    +    elseif ($this->defaultSchema != 'public') {
    +      $key = $this->defaultSchema . '.' . $prefixed_table;
    +      $quoted_key = '"' . $this->defaultSchema . '"."' . $prefixed_table . '"';
         }
         else {
    -      $key = $this->getTempNamespaceName() . '.' . $key;
    +      $key = $this->defaultSchema . '.' . $prefixed_table;
    +      $quoted_key = '"' . $prefixed_table . '"';
    

    previously we were querying with `public.$key` but now, in the 'public' case we're querying with '"$key"' i.e without the schema?

    If we revert to the old logic, we can combine the elseif and else clauses, and in fact we can probably do that first, and then only have the if for the temporary schema (i.e no need for the else/elseif)

  8. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -572,10 +583,10 @@ public function renameTable($table, $new_name) {
    +      $this->connection->query('ALTER INDEX "' . $this->defaultSchema . '"."' . $index->indexname . '" RENAME TO ' . $this->ensureIdentifiersLength($new_name, $index_name, $index_type) . '');
    
    @@ -588,7 +599,7 @@ public function renameTable($table, $new_name) {
    +        $old_sequence = $this->connection->query("SELECT pg_get_serial_sequence('" . $this->defaultSchema . '.' . $table_name . "', '" . $field . "')")->fetchField();
    
    @@ -835,7 +853,7 @@ public function dropIndex($table, $name) {
    +    $this->connection->query('DROP INDEX ' . $this->defaultSchema . '.' . $this->ensureIdentifiersLength($table, $name, 'idx'));
    

    Here we're using $this->defaultSchema in raw SQL - should it be parameterized?

  9. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -711,7 +722,14 @@ public function indexExists($table, $name) {
    +    // Get the schema and tablename for the old table.
    +    $sql_params = [
    +      ':schema' => $this->defaultSchema,
    +      ':table' => $this->connection->getPrefix() . $table,
    +      ':index' => $index_name,
    +    ];
    +    return (bool) $this->connection->query("SELECT 1 FROM pg_indexes WHERE schemaname = :schema AND tablename = :table AND indexname = :index", $sql_params)->fetchField();
    

    because we use parameters here

Arantxio’s picture

@larowlan thank you for your comment, hereby is a new patch which includes some of the comments made:
163.1 - done
163.2 - done
163.3 - I have not changed this as it currently represents how it was, and I think this is currently out of scope. If we really want this I think we should create a follow up issue which addresses multiple of these type of lines.
163.4 - done
163.5 - I have added the sanitization like we do in "connections->escapeDatabase" but without adding te quotes. I have added this to the schema within the Connections class and the Schema class for pgsql.
163.6 - This is almost exactly how it used to be, the only thing that changed was the name of the parameter. So I won't change this. If we want to change this we could also make a follow up issue for this.
163.7 - I have changed it a little bit back to how it was with some of the changes we have made so it should work.
163.8 - I have not changed this, because when I make it parameterized it adds its own quotes to the query string. which made it fail.

Arantxio’s picture

Status: Needs work » Needs review
poker10’s picture

A minor thing - can we remove the empty string concatenation at the end of the query?

@@ -572,10 +581,10 @@ public function renameTable($table, $new_name) {
+      $this->connection->query('ALTER INDEX "' . $this->defaultSchema . '"."' . $index->indexname . '" RENAME TO ' . $this->ensureIdentifiersLength($new_name, $index_name, $index_type) . '');

It was left here by this commit. If we are touching this query, probably this can be also cleaned-up (as it seems it is not needed anymore).

Arantxio’s picture

Added the suggestion from @poker10, thank you for noticing.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

All points of @larowlan have been fixed or answered.
Back to RTBC.

needs-review-queue-bot’s picture

Status: Reviewed & tested by the community » Needs work
FileSize
86 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

ravi.shankar’s picture

Status: Needs work » Needs review
FileSize
24.69 KB
3.33 KB

Added reroll of patch #167 on Drupal 10.1.x.

daffie’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

Reroll failed.

ravi.shankar’s picture

Fixed Drupal CS issue of patch #171.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC.

larowlan’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Connection.php
    @@ -73,6 +73,10 @@ class Connection extends DatabaseConnection implements SupportsTemporaryTablesIn
    +    $this->connectionOptions = $connection_options;
    

    Should we do the sanitizing of the schema name here (the preg_replace), we're doing it twice (once in ::setPrefix and again in Schema::__construct).

    Doing it here would also save any caller of ::getConnectionOptions having to do the same sanitization

  2. +++ b/core/modules/pgsql/src/Driver/Database/pgsql/Schema.php
    @@ -116,16 +128,18 @@ protected function ensureIdentifiersLength($table_identifier_part, $column_ident
    -    if (!str_contains($key, '.') && !str_contains($table, 'db_temporary_')) {
    -      $key = 'public.' . $key;
    +    if (strpos($table, 'db_temporary_') !== FALSE) {
    

    this looks like a bad merge, we moved to using str_contains/str_starts_with/str_ends_with in 10.1.x

Arantxio’s picture

@larowlan I've adjusted the code according to your suggestions and also some changes due to the latest commits.

Arantxio’s picture

Status: Needs work » Needs review
daffie’s picture

Status: Needs review » Reviewed & tested by the community

The points of @larowlan have been addressed.
Back to RTBC.

larowlan’s picture

Issue credits

  • larowlan committed f4fcd7b5 on 10.1.x
    Issue #1060476 by Arantxio, RoSk0, Renrhaf, ravi.shankar, mogtofu33,...
larowlan’s picture

Title: Multiple issues when PostgreSQL is used with non-public schema » [needs backport] Multiple issues when PostgreSQL is used with non-public schema
Version: 10.1.x-dev » 10.0.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed to 10.1.x

I think this is eligible for backport, but will ask for a second opinion

Setting to patch (To be ported) in the meantime

catch’s picture

Version: 10.0.x-dev » 10.1.x-dev
Status: Patch (to be ported) » Fixed

I also think this is eligible but we're now approaching the rc of 10.1 and the last bugfix release of 10.0.x so might be easier to leave it in 10.1.x. It will mostly benefit new installs.

intrafusion’s picture

Version: 10.1.x-dev » 9.5.x-dev
Status: Fixed » Patch (to be ported)

If it's going to be ported, I need for 9.5.x as this is issue is holding me back on 9.4.x

catch’s picture

@intrafusion can you explain how it's preventing you from updating from 9.4.x to 9.5.x, and also are you able to apply the patch?

intrafusion’s picture

@catch the last working patch for me is #1060476-138: Multiple issues when PostgreSQL is used with non-public schema and none tagged with 9.5.x, etc. apply cleanly. I haven't had an opportunity to review why

catch’s picture

Status: Patch (to be ported) » Fixed

@intrafusion so do you mean you have the patch applied to 9.4.x, and will need to re-apply the patch if it's not backported to 9.5.x? In that case I would suggest trying to work from https://www.drupal.org/project/drupal/issues/1060476#comment-14995059 to create a new 9.5-compatible patch, or that version might even apply without changes. Moving back to fixed.

intrafusion’s picture

Status: Fixed » Patch (to be ported)

@catch but it's not fixed, the patch still needs to be ported

catch’s picture

Title: [needs backport] Multiple issues when PostgreSQL is used with non-public schema » Multiple issues when PostgreSQL is used with non-public schema
Version: 9.5.x-dev » 10.1.x-dev
Status: Patch (to be ported) » Fixed

@intrafusion it's fixed in Drupal 10.1.x, and the change won't be committed to 10.0.x, or 9.5.x - therefore from the point of view of core development the issue should be 'fixed'.

You are welcome to check if the latest patch applies to 10.0.x and 9.5.x, and to post a backport of that patch to this issue for other people to use, but none of these require changing the issue status.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

quietone’s picture

Published change record