Problem/Motivation

I tried running a sample migration with a vanilla D6 site and got the following issue:

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'd6-migrate.content_node_field' doesn't exist: SELECT cnf.field_name AS field_name, cnf.type AS type, cnf.global_settings AS global_settings, cnf.required AS required, cnf.multiple AS multiple, cnf.db_storage AS db_storage, cnf.module AS module, cnf.db_columns AS db_columns, cnf.active AS active, cnf.locked AS locked, cnfi.widget_type AS widget_type, cnfi.widget_settings AS widget_settings, map.sourceid1 AS migrate_map_sourceid1, map.source_row_status AS migrate_map_source_row_status FROM {content_node_field} cnf INNER JOIN {content_node_field_instance} cnfi ON cnfi.field_name = cnf.field_name LEFT OUTER JOIN migrate.migrate_map_d6_field map ON cnf.field_name = map.sourceid1 WHERE ( (map.sourceid1 IS NULL ) OR (map.source_row_status = :db_condition_placeholder_0) ) ORDER BY field_name ASC; Array ( [:db_condition_placeholder_0] => 1 )

This is because cck's content module is not enabled, thus the content_node_field table is unavailable. Also, this shuts down the migration process entirely and there's no coming back from such error.

Proposed resolution

Make it so the content_node_field table shouldn't have to be available when running the migration. Gracefully proceed to other available migrations in any case.

Remaining tasks

Write patch. Add test.

User interface changes

None.

API changes

None.

Data model changes

TBD.

CommentFileSizeAuthor
#18 2561697-18-FAIL.patch5.37 KBphenaproxima
#18 2561697-18.patch5.72 KBphenaproxima
#16 interdiff-2561697-12-16.txt496 bytesphenaproxima
#16 2561697-16.patch4.16 KBphenaproxima
#12 interdiff-2561697-5-12.txt1.29 KBphenaproxima
#12 2561697-12.patch3.5 KBphenaproxima
#5 2561697-5.patch2.05 KBphenaproxima
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

anavarre created an issue. See original summary.

anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented
Issue summary: View changes
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Murphysboro, IL, USA
CreditAttribution: mikeryan at Acquia commented
Project: Migrate Upgrade » Drupal core
Version: 8.x-1.x-dev » 8.0.x-dev
Component: Code » migration system
Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented
Priority: Normal » Major
Issue tags: +Migrate critical

After talking with mikeryan on IRC, adding "Migrate critical" tag as 'content' isn't a core D6 module.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Active » Needs review
FileSize
2561697-5.patch2.05 KB

Does this help?

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Issue tags: +Needs tests
anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented

Applied the patch but unfortunately it didn't seem to help.

$ apply 2561697-5.patch 
Checking patch core/modules/migrate/src/Plugin/migrate/builder/BuilderBase.php...
Checking patch core/modules/node/src/Plugin/migrate/builder/d6/Node.php...
Applied patch core/modules/migrate/src/Plugin/migrate/builder/BuilderBase.php cleanly.
Applied patch core/modules/node/src/Plugin/migrate/builder/d6/Node.php cleanly.

$ drush @head.local migrate-upgrade --legacy-db-url=mysql://root:root@localhost:3306/d6 --legacy-root=http://d6.local
    exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'd6.content_node_field'      [error]
doesn't exist' in /var/www/html/head/core/lib/Drupal/Core/Database/Statement.php:64
(snipped)

Enabled the content module on the source D6 site:

$ drush @head.local migrate-upgrade --legacy-db-url=mysql://root:root@localhost:3306/d6 --legacy-root=http://d6.local
    Upgrading d6_date_formats
Upgrading d6_dblog_settings
Upgrading d6_file_settings
(snipped)

And it worked immediately.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

Dang. Well, it was a shot in the dark...

@anavarre, can you provide a backtrace of the PDOException? That would help tremendously in figuring out what's wrong here. I suspect the fix is simple, but I need to know where to look :)

Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented 
[vagrant@vagrant-multi1 vagrant-multi1.tag1consulting.com]$ drush migrate-upgrade --configure-only --legacy-db-url='mysql://old:old@localhost/old' exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or[error] view not found: 1146 Table 'old.content_node_field' doesn't exist' in ../core/lib/Drupal/Core/Database/Statement.php:64

Stack trace:
#0 ../core/lib/Drupal/Core/Database/Statement.php(64): PDOStatement->execute(Array)
#1 ../core/lib/Drupal/Core/Database/Connection.php(597): Drupal\Core\Database\Statement->execute(Array, Array)
#2 ../core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(73): Drupal\Core\Database\Connection->query('SELECT cnf.fiel...', Array, Array)
#3 ../core/lib/Drupal/Core/Database/Query/Select.php(481): Drupal\Core\Database\Driver\mysql\Connection->query('SELECT cnf.fiel...', Array, Array)
#4 ../core/modules/migrate/src/Plugin/migrate/source/SqlBase.php(179): Drupal\Core\Database\Query\Select->execute()
#5 ../core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(218): Drupal\migrate\Plugin\migrate\source\SqlBase->initializeIterator()
#6 ../core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(261): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->getIterator()
#7 ../core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php(32): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#8 ../core/modules/migrate/src/MigrationBuilder.php(53): Drupal\migrate_drupal\Plugin\migrate\builder\d6\CckMigration->buildMigrations(Array)
#9 ../modules/migrate_upgrade/src/MigrationCreationTrait.php(67): Drupal\migrate\MigrationBuilder->createMigrations(Array)
#10 ../modules/migrate_upgrade/src/MigrateUpgradeDrushRunner.php(39): Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->createMigrations(Array, NULL)
#11 ../modules/migrate_upgrade/migrate_upgrade.drush.inc(40): Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->configure()
#12 [internal function]: drush_migrate_upgrade()
#13 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(359): call_user_func_array('drush_migrate_u...', Array)
#14 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(210): _drush_invoke_hooks(Array, Array)
#15 [internal function]: drush_command()
#16 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(178): call_user_func_array('drush_command', Array)
#17 /home/vagrant/.composer/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(62): drush_dispatch(Array)
#18 /home/vagrant/.composer/vendor/drush/drush/drush.php(70): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#19 /home/vagrant/.composer/vendor/drush/drush/drush.php(11): drush_main()
#20 {main}

Next exception 'Drupal\Core\Database\DatabaseExceptionWrapper' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'old.content_node_field' doesn't exist: SELECT cnf.field_name AS field_name, cnf.type AS type, cnf.global_settings AS global_settings, cnf.required AS required, cnf.multiple AS multiple, cnf.db_storage AS db_storage, cnf.module AS module, cnf.db_columns AS db_columns, cnf.active AS active, cnf.locked AS locked, cnfi.widget_type AS
widget_type, cnfi.widget_settings AS widget_settings FROM  {content_node_field} cnf INNER JOIN {content_node_field_instance} cnfi ON cnfi.field_name = cnf.field_name ORDER BY field_name ASC; Array ( ) ' in ../core/lib/Drupal/Core/Database/Connection.php:658

Stack trace:
#0 ../core/lib/Drupal/Core/Database/Connection.php(622): Drupal\Core\Database\Connection->handleQueryException(Object(PDOException), 'SELECT cnf.fiel...', Array, Array)
#1 ../core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(73): Drupal\Core\Database\Connection->query('SELECT cnf.fiel...', Array, Array)
#2 ../core/lib/Drupal/Core/Database/Query/Select.php(481): Drupal\Core\Database\Driver\mysql\Connection->query('SELECT cnf.fiel...', Array, Array)
#3 ../core/modules/migrate/src/Plugin/migrate/source/SqlBase.php(179): Drupal\Core\Database\Query\Select->execute()
#4 ../core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(218): Drupal\migrate\Plugin\migrate\source\SqlBase->initializeIterator()
#5 ../core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(261): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->getIterator()
#6 ../core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php(32): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#7 ../core/modules/migrate/src/MigrationBuilder.php(53): Drupal\migrate_drupal\Plugin\migrate\builder\d6\CckMigration->buildMigrations(Array)
#8 ../modules/migrate_upgrade/src/MigrationCreationTrait.php(67): Drupal\migrate\MigrationBuilder->createMigrations(Array)
#9 ../modules/migrate_upgrade/src/MigrateUpgradeDrushRunner.php(39): Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->createMigrations(Array, NULL)
#10 ../modules/migrate_upgrade/migrate_upgrade.drush.inc(40): Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->configure()
#11 [internal function]: drush_migrate_upgrade()
#12 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(359): call_user_func_array('drush_migrate_u...', Array)
#13 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(210): _drush_invoke_hooks(Array, Array)
#14 [internal function]: drush_command()
#15 /home/vagrant/.composer/vendor/drush/drush/includes/command.inc(178): call_user_func_array('drush_command', Array)
#16 /home/vagrant/.composer/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(62): drush_dispatch(Array)
#17 /home/vagrant/.composer/vendor/drush/drush/drush.php(70): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#18 /home/vagrant/.composer/vendor/drush/drush/drush.php(11): drush_main()
#19 {main}
phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

#8 ../core/modules/migrate/src/MigrationBuilder.php(53): Drupal\migrate_drupal\Plugin\migrate\builder\d6\CckMigration->buildMigrations(Array)

Oooh! I was really close. New patch comin' atcha today.

anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented

Thanks for the stack trace, @Jeremy - You beat me to it. Will test the patch tomorrow if it's available. Thanks.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
FileSize
2561697-12.patch3.5 KB
interdiff-2561697-5-12.txt1.29 KB

Give this one a try -- I doubt you will get the exception now.

anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented

Sorry, still failing, but note that it's now yelling because content_node_field_instance doesn't exist.

$ apply 2561697-12.patch 
Checking patch core/modules/migrate/src/Plugin/migrate/builder/BuilderBase.php...
Checking patch core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php...
Checking patch core/modules/node/src/Plugin/migrate/builder/d6/Node.php...
Applied patch core/modules/migrate/src/Plugin/migrate/builder/BuilderBase.php cleanly.
Applied patch core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php cleanly.
Applied patch core/modules/node/src/Plugin/migrate/builder/d6/Node.php cleanly.

$ drush @head.local migrate-upgrade --legacy-db-url=mysql://root:root@localhost:3306/d6 --legacy-root=http://d6.local
    exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table                              [error]
'd6.content_node_field_instance' doesn't exist' in
/var/www/html/head/core/lib/Drupal/Core/Database/Statement.php:64
Stack trace:
#0 /var/www/html/head/core/lib/Drupal/Core/Database/Statement.php(64): PDOStatement->execute(Array)
#1 /var/www/html/head/core/lib/Drupal/Core/Database/Connection.php(597):
Drupal\Core\Database\Statement->execute(Array, Array)
#2 /var/www/html/head/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(73):
Drupal\Core\Database\Connection->query('SELECT cnfi.fie...', Array, Array)
#3 /var/www/html/head/core/lib/Drupal/Core/Database/Query/Select.php(481):
Drupal\Core\Database\Driver\mysql\Connection->query('SELECT cnfi.fie...', Array, Array)
#4 /var/www/html/head/core/modules/field/src/Plugin/migrate/source/d6/FieldInstancePerFormDisplay.php(26):
Drupal\Core\Database\Query\Select->execute()
#5 /var/www/html/head/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(218):
Drupal\field\Plugin\migrate\source\d6\FieldInstancePerFormDisplay->initializeIterator()
#6 /var/www/html/head/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(261):
Drupal\migrate\Plugin\migrate\source\SourcePluginBase->getIterator()
#7 /var/www/html/head/core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php(46):
Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#8 /var/www/html/head/core/modules/migrate/src/MigrationBuilder.php(53):
Drupal\migrate_drupal\Plugin\migrate\builder\d6\CckMigration->buildMigrations(Array)
#9 /home/anavarre/Sites/git/drupal/migrate_upgrade/src/MigrationCreationTrait.php(67):
Drupal\migrate\MigrationBuilder->createMigrations(Array)
#10 /home/anavarre/Sites/git/drupal/migrate_upgrade/src/MigrateUpgradeDrushRunner.php(39):
Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->createMigrations(Array, 'http://d6.local')
#11 /home/anavarre/Sites/git/drupal/migrate_upgrade/migrate_upgrade.drush.inc(40):
Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->configure()
#12 [internal function]: drush_migrate_upgrade()
#13 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(364): call_user_func_array('drush_migrate_u...',
Array)
#14 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(215): _drush_invoke_hooks(Array, Array)
#15 [internal function]: drush_command()
#16 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(183): call_user_func_array('drush_command', Array)
#17 /home/anavarre/Sites/composer/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(62): drush_dispatch(Array)
#18 /home/anavarre/Sites/composer/vendor/drush/drush/drush.php(70): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#19 /home/anavarre/Sites/composer/vendor/drush/drush/drush.php(11): drush_main()
#20 {main}

Next exception 'Drupal\Core\Database\DatabaseExceptionWrapper' with message 'SQLSTATE[42S02]: Base table or view not found:
1146 Table 'd6.content_node_field_instance' doesn't exist: SELECT cnfi.field_name AS field_name, cnfi.type_name AS type_name,
cnfi.weight AS weight, cnfi.label AS label, cnfi.widget_type AS widget_type, cnfi.widget_settings AS widget_settings,
cnfi.display_settings AS display_settings, cnfi.description AS description, cnfi.widget_module AS widget_module,
cnfi.widget_active AS widget_active, cnf.type AS type, cnf.module AS module
FROM 
{content_node_field_instance} cnfi
INNER JOIN {content_node_field} cnf ON cnfi.field_name = cnf.field_name
ORDER BY weight ASC; Array
(
)
' in /var/www/html/head/core/lib/Drupal/Core/Database/Connection.php:658
Stack trace:
#0 /var/www/html/head/core/lib/Drupal/Core/Database/Connection.php(622):
Drupal\Core\Database\Connection->handleQueryException(Object(PDOException), 'SELECT cnfi.fie...', Array, Array)
#1 /var/www/html/head/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(73):
Drupal\Core\Database\Connection->query('SELECT cnfi.fie...', Array, Array)
#2 /var/www/html/head/core/lib/Drupal/Core/Database/Query/Select.php(481):
Drupal\Core\Database\Driver\mysql\Connection->query('SELECT cnfi.fie...', Array, Array)
#3 /var/www/html/head/core/modules/field/src/Plugin/migrate/source/d6/FieldInstancePerFormDisplay.php(26):
Drupal\Core\Database\Query\Select->execute()
#4 /var/www/html/head/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(218):
Drupal\field\Plugin\migrate\source\d6\FieldInstancePerFormDisplay->initializeIterator()
#5 /var/www/html/head/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(261):
Drupal\migrate\Plugin\migrate\source\SourcePluginBase->getIterator()
#6 /var/www/html/head/core/modules/migrate_drupal/src/Plugin/migrate/builder/d6/CckMigration.php(46):
Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind()
#7 /var/www/html/head/core/modules/migrate/src/MigrationBuilder.php(53):
Drupal\migrate_drupal\Plugin\migrate\builder\d6\CckMigration->buildMigrations(Array)
#8 /home/anavarre/Sites/git/drupal/migrate_upgrade/src/MigrationCreationTrait.php(67):
Drupal\migrate\MigrationBuilder->createMigrations(Array)
#9 /home/anavarre/Sites/git/drupal/migrate_upgrade/src/MigrateUpgradeDrushRunner.php(39):
Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->createMigrations(Array, 'http://d6.local')
#10 /home/anavarre/Sites/git/drupal/migrate_upgrade/migrate_upgrade.drush.inc(40):
Drupal\migrate_upgrade\MigrateUpgradeDrushRunner->configure()
#11 [internal function]: drush_migrate_upgrade()
#12 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(364): call_user_func_array('drush_migrate_u...',
Array)
#13 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(215): _drush_invoke_hooks(Array, Array)
#14 [internal function]: drush_command()
#15 /home/anavarre/Sites/composer/vendor/drush/drush/includes/command.inc(183): call_user_func_array('drush_command', Array)
#16 /home/anavarre/Sites/composer/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(62): drush_dispatch(Array)
#17 /home/anavarre/Sites/composer/vendor/drush/drush/drush.php(70): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#18 /home/anavarre/Sites/composer/vendor/drush/drush/drush.php(11): drush_main()
#19 {main}
anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented
Status: Needs review » Needs work
phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

According to my debugging, the only reason you would be getting this error is if both of the following conditions exist in your D6 database:

1) The content module is listed in the system table and its status is 1;
2) The content_node_field and/or content_node_field_instance table(s) do not exist.

Otherwise, I don't see why it should be freaking out. The code responsible is pretty simple...

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs work » Needs review
FileSize
2561697-16.patch4.16 KB
interdiff-2561697-12-16.txt496 bytes

Debugged with @anavarre and found the source of the problem. I will be shocked, scandalized and chagrined if this does not fix the exception.

anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented

I can confirm #16 fixes the issue. Splendid!

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Issue tags: -Needs tests
FileSize
2561697-18.patch5.72 KB
2561697-18-FAIL.patch5.37 KB

Now with a test! Let's get this done.

Status: Needs review » Needs work

The last submitted patch, 18: 2561697-18-FAIL.patch, failed testing.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs work » Needs review

Fail patches fail. It's what they do.

anavarre’s picture

anavarre
Primary language French
Location 🇪🇺
CreditAttribution: anavarre at Acquia commented
Status: Needs review » Reviewed & tested by the community

And, passing now!

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 18: 2561697-18-FAIL.patch, failed testing.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs work » Reviewed & tested by the community

A fail patch failed testing, you say? Whatever shall we do? </sarcasm>

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
CreditAttribution: webchick at Acquia commented
Status: Reviewed & tested by the community » Fixed

Great work on this one!

Committed and pushed to 8.0.x. Thanks!

  • webchick committed 7797b14 on 8.0.x 
    Issue #2561697 by phenaproxima, anavarre, Jeremy: Migration should not...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.