This issue is part of #2157455: [Meta] Make Drupal 8 work with PostgreSQL or remove support from core before release.

Problem/Motivation

The migration tests fail with a fatal error on PostgreSQL because the database driver cannot create the database tables from the database dump. Drupal Schema data types like "varchar" and "character" have explicit "length" keys on table definitions, but these are missing from the Drupal 6 database dumps.

This obscures further failures behind the fatal error exception.

Proposed resolution

Fix table creation during migrate tests so that database dumps without explicit length key from Drupal 6 are created. This will allow the migration tests to run better and expose actual fails instead of fatal errors in follow-up tasks such as #2477853: PostgreSQL: Add support for reserved field/column names and #2477945: PostgreSQL: Fix migrate_drupal\Tests\d6\MigrateUserProfileFieldTest.

The database dump script from Drupal 6 could be fixed in a follow-up issue, but is no longer as major an issue because of the change in approach committed in #2469623: Process for creating migration source DBs for automated tests.

Remaining tasks

  • Write patch
  • Patch review
  • Manual test / Testbot test run

User interface changes

None.

API changes

None.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because database driver should deal with specification from Drupal 6
Issue priority Major because blocks migration tests for PostgreSQL users only
Unfrozen changes Unfrozen because Migrate code
Prioritized changes Prioritized because PostgreSQL prior to rc1

Comments

bzrudi71’s picture

Issue summary: View changes
Status: Active » Needs review
StatusFileSize
new1.5 KB

Straight re-roll of @mradcliffes patch from #2356985: PostgreSQL: Fix tests in migrate_drupal test group.

bzrudi71’s picture

Wonder if there is room for improvements here or should we go as is. @mradcliffe, do you have something in mind?

erik.erskine’s picture

  1. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -43,6 +43,13 @@ class Schema extends DatabaseSchema {
    +   * A list of field types that support NOT NULL or NULL modifier.
    +   *
    +   * @var array
    +   */
    +  protected $modifier_supported = array('varchar', 'character', 'int', 'real', 'smallint', 'bigint', 'double precision', 'numeric');
    +
    +  /**
    
    @@ -291,7 +298,7 @@ protected function createFieldSql($name, $spec) {
    -    if (isset($spec['not null'])) {
    +    if (isset($spec['not null']) && in_array($spec['pgsql_type'], $this->modifier_supported)) {
    

    Is this bit needed? The tests don't seem to be failing because of null/not null, and declaring a column of type text not null is valid.

  2. +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
    @@ -280,7 +287,7 @@ protected function createFieldSql($name, $spec) {
    -    if (in_array($spec['pgsql_type'], array('varchar', 'character', 'text')) && isset($spec['length'])) {
    +    if (in_array($spec['pgsql_type'], array('varchar', 'character')) && isset($spec['length'])) {
    

    This looks good, because text may not have a length limit according to http://www.postgresql.org/docs/9.1/static/datatype-character.html#DATATYPE-CHARACTER-TABLE

erik.erskine’s picture

StatusFileSize
new1.08 KB

Revised the patch in #1 so that it only removes the length modifier for text.

erik.erskine’s picture

StatusFileSize
new709 bytes
new1.08 KB

Oops, trying again:
Revised the patch in #1 so that it only removes the length modifier for text.

mradcliffe’s picture

Component: postgresql db driver » migration system

I was talking with migrate folks, and I think we would need to fix the database script in d6 and d7 as well. Also moving this to migration system component to get more visibility. We still have the issue relationship back to the parent and tag for PostgreSQL.

bzrudi71’s picture

@mradcliffe wonder how to move forward here. What is the current plan here, fix it in PG driver space with patch from this issue, or fix it in the migrate database scripts?

mradcliffe’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Sorry for the confusing comment. I think I meant to write that database dump script should be fixed when dealing with issues like #2477857: PostgreSQL: Fix not null constraint exceptions in migrate_drupal tests or #2477853: PostgreSQL: Add support for reserved field/column names. I wanted to add component as migration system. #246962: controlling what shows in "recent posts' menu item seems to have fixed the process for creating database dumps so that we are not reliant on the database scripts, and are just using a SQL file to do so, right?

I think @ingaro's patch is fine, and all that's left to do is a confirmation of test results for migration tests.

bzrudi71’s picture

Status: Needs review » Reviewed & tested by the community

Okay thanks @mradcliffe! I tried the patch in #5 while working on druplaci switch some hours ago and it works as expected. RTBC

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 8.0.x. Thanks!

  • webchick committed f74f967 on 8.0.x
    Issue #2477845 by ingaro, bzrudi71: PostgreSQL: Fix broken migrate table...
bzrudi71’s picture

Yay! Small patch, big impact: -64 exceptions! Thanks all.
Next step, decide about #2477853: PostgreSQL: Add support for reserved field/column names.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.