We're updating paragraphs from 8.x-1.0-rc4 to 8.x-1.0. We use Jenkins and a project.make.yml. Unfortunately when Jenkins gets to drush updatedb, we get:

Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S22]:          [error]
Column not found: 1054 Unknown column 'data.parent_id' in 'field
list': SELECT revision.*, data.parent_id AS parent_id,
data.parent_type AS parent_type, data.parent_field_name AS
parent_field_name
FROM 
{paragraphs_item_revision_field_data} revision
LEFT OUTER JOIN {paragraphs_item_field_data} data ON (revision.id =
data.id and revision.langcode = data.langcode)
WHERE  (revision.id IN  (:db_condition_placeholder_0)) AND
(revision.revision_id IN  (:db_condition_placeholder_1)) 
ORDER BY revision.id ASC; Array
(
    [:db_condition_placeholder_0] => 211
    [:db_condition_placeholder_1] => 11047
)
 in
Drupal\Core\Entity\Sql\SqlContentEntityStorage->loadFromSharedTables()
(line 554 of
/private/var/www/sites/mskcc_deploy/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column     [error]
'data.parent_id' in 'field list' in
/private/var/www/sites/mskcc_deploy/core/lib/Drupal/Core/Database/Statement.php:59
Stack trace:
#0
...

So our database doesn't have the column data.parent_id. drush updatedb works fine when we run it after Jenkins installs 8.x-1.0-rc4, we manually drush dl 8.x-1.0 and manually drush updatedb at that point. But we can't upgrade paragraphs like that because it would be impossible to QA our paragraphs pages and we could end up with conflicts.

Has anyone else come across this issue? If so, how did you resolve it? We have the same problem updating from 8.x-1.0-rc5 to 8.x-1.0.

I'll add more information about the schema after I rebuild again and review the database after a manual upgrade.

Thanks.

CommentFileSizeAuthor
#14 database-updated-using-methods.png20.88 KBesod
#14 database-updates.png48.84 KBesod
#6 drupal-paragraphs.png403.91 KBesod
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

esod created an issue. See original summary.

miro_dietiker’s picture

miro_dietiker
Location Switzerland
CreditAttribution: miro_dietiker at MD Systems GmbH commented

I guess you simply forgot to update ERR module. There are lots of related issues.

johnchque’s picture

johnchque
Primary language Spanish
Location Bolivia
CreditAttribution: johnchque at MD Systems GmbH commented
Status: Active » Postponed (maintainer needs more info)

Seems a problem with an outdated ERR.
Please report back if updating it fixes the problem.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented

We have been running ERR 8.x-1.0.

# Entity Reference Revisions 8.x-1.0 - 2016-07-28
  entity_reference_revisions:
    subdir: contrib
    version: 1.0

I just tried the development release of ERR, 8.x-1.x-dev.

  # Entity Reference Revisions 8.x-1.x-dev - 2016-11-18
  entity_reference_revisions:
    subdir: contrib
    version: 1.x-dev

Unfortunately, I'm getting the same error.

miro_dietiker’s picture

miro_dietiker
Location Switzerland
CreditAttribution: miro_dietiker at MD Systems GmbH commented

Just as a cross reference:
Did you try installing the mentioned versions manually on a vanilla Drupal.. and then update both ERR and Paragraphs to the latest dev versions?

Is it possible that you either started pre-beta (no upgrade path guarantee) or updated earlier at some point to some dev so that update methods are no more executed?

In fact column parent_id was introduced by paragraphs_update_8002 that was added 2 months post 8.x-1.0-rc4 by commit ebba36d.
So if the states were correct, this method would need to be executed.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented
FileSize
drupal-paragraphs.png403.91 KB

I install vanilla Drupal and drush updb works, as it has been all along. These are the steps I take.

$ cd /
$ drush updb
No database updates required                                                              [success]
$ cd modules/
/var/www/sites/d8/modules
$ drush dl entity_reference_revisions
Project entity_reference_revisions (8.x-1.0) downloaded to /private/var/www/sites/d8//modules/entity_reference_revisions.                                                                        [success]
$ drush dl paragraphs-1.0-rc4
Project paragraphs (8.x-1.0-rc4) downloaded to /private/var/www/sites/d8//modules/paragraphs.                                                                                                    [success]
Project paragraphs contains 3 modules: paragraphs_type_permissions, paragraphs_demo, paragraphs.
$ drush en -y paragraphs
The following extensions will be enabled: paragraphs, entity_reference_revisions
Do you really want to continue? (y/n): y
entity_reference_revisions was enabled successfully.                                                                                                                                             [ok]
paragraphs was enabled successfully.                                                                                                                                                             [ok]
paragraphs defines the following permissions: administer paragraphs types
$ drush updb
No database updates required                                                                                                                                                                     [success]
$ drush dl entity_reference_revisions-1.x-dev
Install location /private/var/www/sites/d8//modules/entity_reference_revisions already exists. Do you want to overwrite it? (y/n): y
Project entity_reference_revisions (8.x-1.x-dev) downloaded to /private/var/www/sites/d8//modules/entity_reference_revisions.                                                                    [success]
$ drush dl paragraphs-1.x-dev
Install location /private/var/www/sites/d8//modules/paragraphs already exists. Do you want to overwrite it? (y/n): y
Project paragraphs (8.x-1.x-dev) downloaded to /private/var/www/sites/d8//modules/paragraphs.                                                                                                    [success]
Project paragraphs contains 3 modules: paragraphs_type_permissions, paragraphs_demo, paragraphs.
$ drush updb
The following updates are pending:

paragraphs module :
  8001 -   Add status field.
  8002 -   Add parent ID, parent type and parent field name fields.
  8003 -   Placeholder for the previous 8003 update.
  8004 -   Truncate the content_translation_status columns.
  8006 -   Remove revision_timestamp, changed fields, add content_translation_changed.
  8007 -   Ensure that existing paragraphs are published.
  8008 -   Ensure that the parent indexes are added to the paragraphs entity.
  8009 -   Set the weight to 11 to override content_translation's hook_module_implements_alter implementation

paragraphs module :
  Set the parent id, type and field name to the already created paragraphs.   @param $sandbox

Do you wish to run all pending updates? (y/n): y
Performing paragraphs_update_8001                                                                                                                                                                [ok]
Performing paragraphs_update_8002                                                                                                                                                                [ok]
Performing paragraphs_update_8003                                                                                                                                                                [ok]
Performing paragraphs_update_8004                                                                                                                                                                [ok]
Performing paragraphs_update_8006                                                                                                                                                                [ok]
Performing paragraphs_update_8007                                                                                                                                                                [ok]
Performing paragraphs_update_8008                                                                                                                                                                [ok]
Performing paragraphs_update_8009                                                                                                                                                                [ok]
Cache rebuild complete.                                                                                                                                                                          [ok]
Post updating paragraphs                                                                                                                                                                         [ok]
Cache rebuild complete.                                                                                                                                                                          [ok]
Finished performing updates.                                                                                                                                                                     [ok]

My vanilla system looks like this:

drupal-paragraphs.png

I'm thinking of writing an update hook to for the paragraphs updates that will run before drush updb in jenkins.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented

Interesting. I went through the history of our project.make.yml and we started with paragraphs 8.x-1.0-rc4

# Paragraphs 8.x-1.0-rc4 - 2015-Nov-17
  paragraphs:
    version: 1.0-rc4

It could be 2015-Nov-17 was typed in wrong. I see commit ebba36d is from April, 2016.

miro_dietiker’s picture

miro_dietiker
Location Switzerland
CreditAttribution: miro_dietiker at MD Systems GmbH commented

We don't type these release dates. They are autogenereated by d.o on release push.

git show 8.x-1.0-rc4

commit 740874bacb56a6a90683e7818c06ba77a638125c
Author: versantus <versantus@1243392.no-reply.drupal.org>
Date:   Tue Nov 17 14:21:21 2015 +0100

However, i see between rc4 and rc5 that there were some update bugs prior to rc5 release.
If you switched to -dev after rc4 and before rc5, that could likely have caused the problem.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented

We're on rc4, so we couldn't have switched to -dev before rc5. Oh well. Thanks for git show 8.x-1.0-rc4. That will save me some time.

I'm thinking of using a drush_hook_pre_COMMAND() to add the missing fields. Like this:

  if (!db_field_exists('paragraphs_item_field_data', 'status')) {
    db_query("ALTER TABLE {paragraphs_item_field_data}
    ADD status tinyint(4) DEFAULT NULL AFTER default_langcode,
    ADD parent_id varchar(255) CHARACTER SET ascii DEFAULT NULL AFTER status,
    ADD parent_type varchar(32) CHARACTER SET ascii DEFAULT NULL AFTER parent_id,
    ADD parent_field_name varchar(32) CHARACTER SET ascii DEFAULT NULL AFTER parent_type;
  ");
  }
  if (!db_field_exists('paragraphs_item_revision_field_data', 'status')) {
    db_query("ALTER TABLE {paragraphs_item_revision_field_data}
     ADD status tinyint(4) DEFAULT NULL AFTER default_langcode;
  ");
  }

I know db_field_exists() and db_query() are deprecated functions. My concern would be if I'm overlooking an index by going directly to MySQL instead of using PDO or Doctrine methods.

miro_dietiker’s picture

miro_dietiker
Location Switzerland
CreditAttribution: miro_dietiker at MD Systems GmbH commented

I would check on a clone of that system if it helps if you manually run paragraphs_update_8002() prior to update.
This would also be of interest:

# drush core-cli
>>> drupal_get_installed_schema_version('paragraphs');
=> "8009"

So i'm locally at 8009 with latest dev.

This is really strange, i have requested that our team is retesting to install an vanilla rc4 and check update to rc5 and then also latest dev to make sure this process works regular.

johnchque’s picture

johnchque
Primary language Spanish
Location Bolivia
CreditAttribution: johnchque at MD Systems GmbH commented

I just tested with rc4 to rc5 and demo module, the paragraphs seem to be properly updated, I also tested with the latest dev and I don't see any problem when updating. I'm locally at 8009 too.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented

Thanks for your help. I'll have a look at drupal_get_installed_schema_version().

The module always updates from rc4 to 1.0 when we run drush updb from the command line. The error occurs during our Jenkins build.

esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented

I wrote a tiny php script, whichSchema.php, in the drupal root on a dev server that was built with rc4.

<?php
$whichSchema = drupal_get_installed_schema_version('paragraphs');
echo $whichSchema . "\n";

Run the script...

$ drush php-script whichSchema.php
8000

For completeness sake ...

$ drush pm-info paragraphs
 Extension        :  paragraphs
 Project          :  paragraphs
 Type             :  module
 Title            :  Paragraphs
 Description      :  Implements the Paragraphs entity types, Entity Reference selection type and Entity Reference field widget.
 Version          :  8.x-1.0-rc4
 Date             :  2015-11-17
 Package          :  Paragraphs
 Core             :  8.x
 PHP              :  5.5.9
 Status           :  enabled
 Path             :  modules/contrib/paragraphs
 Schema version   :  8000
 Requires         :  entity_reference_revisions
 Required by      :  msk_card, msk_timeline, paragraphs_demo, paragraphs_type_permissions
 Permissions      :  administer paragraphs types
 Configure        :  None

On a different dev server, one whose Drupal was built with the drush_hook_pre_COMMAND()

$ drush php-script whichSchema.php
8008

$ drush pm-info paragraphs
 Extension        :  paragraphs
 Project          :  paragraphs
 Type             :  module
 Title            :  Paragraphs
 Description      :  Enables the creation of paragraphs entities. To safely uninstall, go first to Structure > Paragraphs types > Settings.
 Version          :  8.x-1.0
 Date             :  2016-07-28
 Package          :  Paragraphs
 Core             :  8.x
 PHP              :  5.5.9
 Status           :  enabled
 Path             :  modules/contrib/paragraphs
 Schema version   :  8008
 Requires         :  entity_reference_revisions
 Required by      :  msk_card, msk_timeline, paragraphs_demo, paragraphs_type_permissions
 Permissions      :  administer paragraphs types
 Configure        :  None
esod’s picture

esod CreditAttribution: esod at Memorial Sloan Kettering Cancer Center commented
FileSize
database-updates.png48.84 KB
database-updated-using-methods.png20.88 KB

I'm on the right track with the drush_hook_pre_COMMAND(). Adding the new fields with db_query("ALTER TABLE... worked fine on local and dev environments. However when we moved the drush_hook_pre_COMMAND() to our load balanced staging system we started seeing Database updates - Out of date and Entity/field definitions - Mismatched entity and/or field definitions errors.

database-updates.png

I copied the storage_definition code from paragraphs.install to the drush_hook_pre_COMMAND(). The Post updating paragraphs are now running and the error is gone.

database-updated-using-methods.png

Although you might think the field creation methods could be run twice by Jenkins since the same code is in two places, the install hooks are not run twice and the fields are not created twice. Here are the table structures after the build:

$ drush sqlc

MySQL [my_database]> DESCRIBE paragraphs_item_field_data;
+-------------------+------------------+------+-----+---------+-------+
| Field             | Type             | Null | Key | Default | Extra |
+-------------------+------------------+------+-----+---------+-------+
| id                | int(10) unsigned | NO   | PRI | NULL    |       |
| revision_id       | int(10) unsigned | NO   | MUL | NULL    |       |
| type              | varchar(32)      | NO   | MUL | NULL    |       |
| langcode          | varchar(12)      | NO   | PRI | NULL    |       |
| uid               | int(10) unsigned | YES  | MUL | NULL    |       |
| created           | int(11)          | YES  |     | NULL    |       |
| default_langcode  | tinyint(4)       | NO   |     | NULL    |       |
| status            | tinyint(4)       | YES  |     | NULL    |       |
| parent_id         | varchar(255)     | YES  |     | NULL    |       |
| parent_type       | varchar(32)      | YES  | MUL | NULL    |       |
| parent_field_name | varchar(32)      | YES  |     | NULL    |       |
+-------------------+------------------+------+-----+---------+-------+
11 rows in set (0.00 sec)

MySQL [my_database]> DESCRIBE paragraphs_item_revision_field_data;
+------------------+------------------+------+-----+---------+-------+
| Field            | Type             | Null | Key | Default | Extra |
+------------------+------------------+------+-----+---------+-------+
| id               | int(10) unsigned | NO   | MUL | NULL    |       |
| revision_id      | int(10) unsigned | NO   | PRI | NULL    |       |
| langcode         | varchar(12)      | NO   | PRI | NULL    |       |
| uid              | int(10) unsigned | YES  | MUL | NULL    |       |
| created          | int(11)          | YES  |     | NULL    |       |
| default_langcode | tinyint(4)       | NO   |     | NULL    |       |
| status           | tinyint(4)       | YES  |     | NULL    |       |
+------------------+------------------+------+-----+---------+-------+
7 rows in set (0.00 sec)


Anyway, I assume others are using Jenkins to deploy Drupal 8 and need to update paragraphs. This is how we did it.

Thank you very much for the code and the great module.