D7 Migration failing on d6_filter_format [#2868762]

Hi, I'm trying to run a D7 to D8 migration following the instructions here to create a custom module (using Drupal 8.3.0 core): http://cgit.drupalcode.org/migrate_upgrade/tree/README.txt

I have not altered any migrations yet. When I run the migrations it files on d6_filter_format, and I seem to have other tables missing, for example imagecache, which I had not installed in D7.

Here is the output of drush migrate-status:

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.imagecache_preset' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{imagecache_preset} icp) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.filter_formats' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{filter_formats} f) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.boxes' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{boxes} b) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.files' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{files} f) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.comments' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{comments} c
INNER JOIN {node} n ON c.nid = n.nid) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.content_node_field' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT DISTINCT cnf.field_name AS field_name, cnf.type AS type, cnf.global_settings AS global_settings, cnf.required AS required, cnf.multiple AS multiple, cnf.db_storage AS db_storage, cnf.module AS module, cnf.db_columns AS db_columns, cnf.active AS active, cnf.locked AS locked, 1 AS expression
FROM 
{content_node_field} cnf
INNER JOIN {content_node_field_instance} cnfi ON cnfi.field_name = cnf.field_name) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.content_node_field_instance' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{content_node_field_instance} cnfi
INNER JOIN {content_node_field} cnf ON cnf.field_name = cnfi.field_name) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.content_node_field_instance' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{content_node_field_instance} cnfi
INNER JOIN {content_node_field} cnf ON cnfi.field_name = cnf.field_name) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.content_node_field_instance' doesn't exist: SELECT cnfi.display_settings AS display_settings
FROM 
{content_node_field_instance} cnfi; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.content_node_field_instance' doesn't exist: SELECT cnfi.field_name AS field_name, cnfi.type_name AS type_name, cnfi.weight AS weight, cnfi.label AS label, cnfi.display_settings AS display_settings, cnfi.widget_settings AS widget_settings, cnf.type AS type, cnf.module AS module
FROM 
{content_node_field_instance} cnfi
INNER JOIN {content_node_field} cnf ON cnfi.field_name = cnf.field_name
ORDER BY cnfi.weight ASC; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.upload' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT DISTINCT u.nid AS nid, u.vid AS vid, n.type AS type, 1 AS expression
FROM 
{upload} u
INNER JOIN {node} n ON n.nid = u.nid AND n.vid = u.vid) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.vocabulary' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{vocabulary} v) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.vocabulary' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{vocabulary} v) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.vocabulary' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{vocabulary} v
INNER JOIN {vocabulary_node_types} nt ON v.vid = nt.vid) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.vocabulary' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{vocabulary} v
INNER JOIN {vocabulary_node_types} nt ON v.vid = nt.vid) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.vocabulary' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{vocabulary} v
INNER JOIN {vocabulary_node_types} nt ON v.vid = nt.vid) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.term_data' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT DISTINCT td.*, 1 AS expression
FROM 
{term_data} td) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.profile_field' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{profile_field} pf) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.profile_field' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{profile_field} pf) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.profile_field' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{profile_field} pf) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.profile_field' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT 1 AS expression
FROM 
{profile_field} pf) subquery; Array
(
)

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'cherished.profile_values' doesn't exist: SELECT COUNT(*) AS expression
FROM 
(SELECT DISTINCT pv.uid AS uid, 1 AS expression
FROM 
{profile_values} pv) subquery; Array
(
)

 Group: default                          Status  Total  Imported  Unprocessed  Last imported       
 contact_category                        Idle    2      2         0            2017-04-11 19:02:17 
 d6_date_formats                         Idle    3      3         0            2017-04-11 19:02:17 
 d6_dblog_settings                       Idle    0      1         -1           2017-04-11 19:02:17 
 d6_imagecache_presets                   Idle    N/A    0         N/A                              
 d6_menu                                 Idle    4      4         0            2017-04-11 19:02:17 
 d6_node_settings                        Idle    1      1         0            2017-04-11 19:02:17 
 d6_search_settings                      Idle    0      1         -1           2017-04-11 19:02:17 
 d6_system_cron                          Idle    1      1         0            2017-04-11 19:02:17 
 d6_system_date                          Idle    1      1         0            2017-04-11 19:02:17 
 d6_system_file                          Idle    0      0         -1           2017-04-11 19:02:17 
 d6_system_performance                   Idle    1      1         0            2017-04-11 19:02:17 
 d6_url_alias                            Idle    105    0         0            2017-04-11 19:02:17 
 d6_user_mail                            Idle    1      1         0            2017-04-11 19:02:17 
 d6_user_settings                        Idle    1      1         0            2017-04-11 19:02:17 
 d7_dblog_settings                       Idle    0      1         -1           2017-04-11 19:02:17 
 d7_filter_format                        Idle    4      4         0            2017-04-11 19:02:17 
 d7_filter_settings                      Idle    1      1         0            2017-04-11 19:02:17 
 d7_global_theme_settings                Idle    1      1         0            2017-04-11 19:02:17 
 d7_image_settings                       Idle    0      1         -1           2017-04-11 19:02:17 
 d7_image_styles                         Idle    0      0         0            2017-04-11 19:02:17 
 d7_node_settings                        Idle    1      1         0            2017-04-11 19:02:17 
 d7_search_settings                      Idle    0      0         -1           2017-04-11 19:02:17 
 d7_system_authorize                     Idle    0      1         -1           2017-04-11 19:02:17 
 d7_system_cron                          Idle    0      1         -1           2017-04-11 19:02:17 
 d7_system_date                          Idle    1      1         0            2017-04-11 19:02:17 
 d7_system_file                          Idle    1      0         0            2017-04-11 19:02:17 
 d7_system_mail                          Idle    0      0         -1           2017-04-11 19:02:17 
 d7_system_performance                   Idle    1      1         0            2017-04-11 19:02:17 
 d7_url_alias                            Idle    105    105       0            2017-04-11 19:02:17 
 d7_user_flood                           Idle    0      1         -1           2017-04-11 19:02:17 
 d7_user_mail                            Idle    1      1         0            2017-04-11 19:02:17 
 file_settings                           Idle    0      1         -1           2017-04-11 19:02:17 
 menu_settings                           Idle    0      1         -1           2017-04-11 19:02:18 
 search_page                             Idle    0      1         -1           2017-04-11 19:02:18 
 system_image                            Idle    0      1         -1           2017-04-11 19:02:18 
 system_image_gd                         Idle    0      1         -1           2017-04-11 19:02:18 
 system_logging                          Idle    1      1         0            2017-04-11 19:02:18 
 system_maintenance                      Idle    0      1         -1           2017-04-11 19:02:18 
 system_rss                              Idle    0      1         -1           2017-04-11 19:02:18 
 system_site                             Idle    1      1         0            2017-04-11 19:02:18 
 taxonomy_settings                       Idle    0      1         -1           2017-04-11 19:02:18 
 text_settings                           Idle    0      1         -1           2017-04-11 19:02:18 
 update_settings                         Idle    0      1         -1           2017-04-11 19:02:18 
 block_content_type                      Idle    1      1         0            2017-04-11 19:02:18 
 d6_menu_links                           Idle    34     30        0            2017-04-11 19:02:18 
 d7_contact_settings                     Idle    1      1         0            2017-04-11 19:02:18 
 block_content_body_field                Idle    1      1         0            2017-04-11 19:02:18 
 d6_contact_settings                     Idle    1      1         0            2017-04-11 19:02:18 
 d7_custom_block                         Idle    5      5         0            2017-04-11 19:02:18 
 d7_user_role                            Idle    4      4         0            2017-04-11 19:02:18 
 d7_block                                Idle    449    103       0            2017-04-11 19:02:18 
 d6_filter_format                        Idle    N/A    0         N/A                              
 d6_custom_block                         Idle    N/A    0         N/A                              
 d6_user_role                            Idle    4      0         4                                
 d6_block                                Idle    449    0         449                              
 d7_node_type                            Idle    4      4         0                                
 d7_comment_type                         Idle    4      0         4                                
 d7_comment_field                        Idle    4      0         4                                
 d6_comment_type                         Idle    1      0         1                                
 d6_comment_field                        Idle    1      0         1                                
 d6_node_type                            Idle    4      0         4                                
 d6_comment_field_instance               Idle    3      0         3                                
 d6_comment_entity_display               Idle    3      0         3                                
 d7_comment_field_instance               Idle    4      0         4                                
 d6_comment_entity_form_display_subject  Idle    1      0         1                                
 d7_comment_entity_display               Idle    4      0         4                                
 d7_field                                Idle    23     19        0                                
 d7_taxonomy_vocabulary                  Idle    2      2         0                                
 d7_field_instance                       Idle    37     33        0                                
 d7_file                                 Idle    515    0         0                                
 user_picture_field                      Idle    1      1         0                                
 user_picture_field_instance             Idle    1      1         0                                
 user_picture_entity_display             Idle    1      1         0                                
 user_picture_entity_form_display        Idle    1      1         0                                
 d7_user                                 Idle    3      3         0                                
 d7_node:article                         Idle    1      1         0                                
 d7_node:media_gallery                   Idle    1      1         0                                
 d7_node:page                            Idle    43     43        0                                
 d7_node:poll                            Idle    0      0         0                                
 d7_comment                              Idle    0      0         0                                
 d7_comment_entity_form_display_subject  Idle    4      0         4                                
 d6_comment_entity_form_display          Idle    3      0         3                                
 d6_file                                 Idle    N/A    0         N/A                              
 d6_user_picture_file                    Idle    0      0         0                                
 d6_user                                 Idle    3      0         3                                
 d6_comment                              Idle    N/A    0         N/A                              
 d7_comment_entity_form_display          Idle    4      0         4                                
 d6_field                                Idle    N/A    0         N/A                              
 d6_field_instance                       Idle    N/A    0         N/A                              
 d6_field_instance_widget_settings       Idle    N/A    0         N/A                              
 d7_field_instance_widget_settings       Idle    37     33        0                                
 d7_view_modes                           Idle    11     11        0                                
 d6_view_modes                           Idle    N/A    0         N/A                              
 d6_field_formatter_settings             Idle    N/A    0         N/A                              
 d7_field_formatter_settings             Idle    115    51        0                                
 d6_upload_field                         Idle    1      1         0                                
 d6_upload_field_instance                Idle    4      0         4                                
 d6_upload_entity_form_display           Idle    4      0         4                                
 d6_upload                               Idle    N/A    0         N/A                              
 d6_upload_entity_display                Idle    4      0         4                                
 d6_node_setting_status                  Idle    4      0         4                                
 d6_node_setting_sticky                  Idle    4      0         4                                
 d7_node_title_label                     Idle    4      4         0                                
 d7_node_revision:article                Idle    0      0         0                                
 d7_node_revision:media_gallery          Idle    0      0         0                                
 d7_node_revision:page                   Idle    0      0         0                                
 d7_node_revision:poll                   Idle    0      0         0                                
 d6_node_setting_promote                 Idle    4      0         4                                
 d7_shortcut_set                         Idle    1      1         0                                
 d7_shortcut_set_users                   Idle    0      0         0                                
 d7_menu                                 Idle    4      4         0                                
 d7_menu_links                           Idle    34     30        0                                
 d7_shortcut                             Idle    4      4         0                                
 d6_taxonomy_vocabulary                  Idle    N/A    0         N/A                              
 d6_vocabulary_field                     Idle    N/A    0         N/A                              
 d6_vocabulary_field_instance            Idle    N/A    0         N/A                              
 d6_vocabulary_entity_form_display       Idle    N/A    0         N/A                              
 d7_taxonomy_term:gallery_collections    Idle    1      1         0                                
 d7_taxonomy_term:tags                   Idle    54     54        0                                
 d6_vocabulary_entity_display            Idle    N/A    0         N/A                              
 d6_taxonomy_term                        Idle    N/A    0         N/A                              
 user_profile_field                      Idle    N/A    0         N/A                              
 user_profile_field_instance             Idle    N/A    0         N/A                              
 user_profile_entity_display             Idle    N/A    0         N/A                              
 user_profile_entity_form_display        Idle    N/A    0         N/A                              
 d6_profile_values                       Idle    N/A    0         N/A                              
 d6_user_contact_settings                Idle    3      0         3

And the output of drush migrate-import --all

Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'contact_category'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_date_formats'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_dblog_settings'
Migration d6_imagecache_presets did not meet the requirements. The module imagecache is  [error]
not enabled in the source site. source_provider: imagecache.
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_imagecache_presets'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd6_menu'      [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_node_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_search_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_system_cron'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_system_date'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_system_file'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_system_performance'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd6_url_alias' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd6_user_mail' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_user_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_dblog_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_filter_format'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_filter_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_global_theme_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_image_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_image_styles'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_node_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_search_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_authorize'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_cron'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_date'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_file'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_mail'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_system_performance'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd7_url_alias' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd7_user_flood'[status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd7_user_mail' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'file_settings'[status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'menu_settings'[status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'search_page'  [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'system_image' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'system_image_gd'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'system_logging'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'system_maintenance'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'system_rss'   [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'system_site'  [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'taxonomy_settings'

Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'text_settings'[status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'update_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'block_content_type'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd6_menu_links'[status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_contact_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'block_content_body_field'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd6_contact_settings'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with                [status]
'd7_custom_block'
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd7_user_role' [status]
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - done with 'd7_block'     [status]
Migration failed with source plugin exception: SQLSTATE[42S02]: Base table or view not   [error]
found: 1146 Table &#039;cherished.filter_formats&#039; doesn&#039;t exist: SELECT f.*,
map.sourceid1 AS migrate_map_sourceid1, map.source_row_status AS
migrate_map_source_row_status
FROM 
{filter_formats} f
LEFT OUTER JOIN drupal.cherished_migrate_map_d6_filter_format map ON format =
map.sourceid1
WHERE (map.sourceid1 IS NULL) OR (map.source_row_status = :db_condition_placeholder_0);
Array
(
    [:db_condition_placeholder_0] =&gt; 1
)

PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table                  [error]
'cherished.filter_formats' doesn't exist in
/html/core/lib/Drupal/Core/Database/Statement.php:59
Stack trace:
#0 /html/core/lib/Drupal/Core/Database/Statement.php(59):
PDOStatement->execute(Array)
#1 /html/core/lib/Drupal/Core/Database/Connection.php(624):
Drupal\Core\Database\Statement->execute(Array, Array)
#2 /html/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(87):
Drupal\Core\Database\Connection->query('SELECT COUNT(*)...', Array, Array)
#3 /html/core/lib/Drupal/Core/Database/Query/Select.php(508):
Drupal\Core\Database\Driver\mysql\Connection->query('SELECT COUNT(*)...', Array, Array)
#4 /html/core/modules/migrate/src/Plugin/migrate/source/SqlBase.php(321):
Drupal\Core\Database\Query\Select->execute()
#5 /html/core/modules/migrate/src/Plugin/Migration.php(515):
Drupal\migrate\Plugin\migrate\source\SqlBase->count()
#6 /html/core/modules/migrate/src/Plugin/Migration.php(443):
Drupal\migrate\Plugin\Migration->allRowsProcessed()
#7 /html/core/modules/migrate/src/MigrateExecutable.php(164):
Drupal\migrate\Plugin\Migration->checkRequirements()
#8 /vendor/drush/drush/includes/drush.inc(720):
Drupal\migrate\MigrateExecutable->import()
#9 /vendor/drush/drush/includes/drush.inc(711):
drush_call_user_func_array(Array, Array)
#10 /html/modules/migrate_tools/migrate_tools.drush.inc(280):
drush_op(Array)
#11 [internal function]:
_drush_migrate_tools_execute_migration(Object(Drupal\migrate\Plugin\Migration),
'd6_custom_block', Array)
#12 /html/modules/migrate_tools/migrate_tools.drush.inc(246):
array_walk(Array, '_drush_migrate_...', Array)
#13 /vendor/drush/drush/includes/command.inc(422):
drush_migrate_tools_migrate_import()
#14 /vendor/drush/drush/includes/command.inc(231):
_drush_invoke_hooks(Array, Array)
#15 /vendor/drush/drush/includes/command.inc(199): drush_command()
#16 /vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67):
drush_dispatch(Array)
#17 /vendor/drush/drush/includes/preflight.inc(66):
Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#18 /vendor/drush/drush/drush.php(12): drush_main()
#19 {main}

Next Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S02]: Base table or view
not found: 1146 Table 'cherished.filter_formats' doesn't exist: SELECT COUNT(*) AS
expression
FROM 
(SELECT 1 AS expression
FROM 
{filter_formats} f) subquery; Array
(
)
 in /html/core/lib/Drupal/Core/Database/Connection.php:685
Stack trace:
#0 /html/core/lib/Drupal/Core/Database/Connection.php(649):
Drupal\Core\Database\Connection->handleQueryException(Object(PDOException), 'SELECT
COUNT(*)...', Array, Array)
#1 /html/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(87):
Drupal\Core\Database\Connection->query('SELECT COUNT(*)...', Array, Array)
#2 /html/core/lib/Drupal/Core/Database/Query/Select.php(508):
Drupal\Core\Database\Driver\mysql\Connection->query('SELECT COUNT(*)...', Array, Array)
#3 /html/core/modules/migrate/src/Plugin/migrate/source/SqlBase.php(321):
Drupal\Core\Database\Query\Select->execute()
#4 /html/core/modules/migrate/src/Plugin/Migration.php(515):
Drupal\migrate\Plugin\migrate\source\SqlBase->count()
#5 /html/core/modules/migrate/src/Plugin/Migration.php(443):
Drupal\migrate\Plugin\Migration->allRowsProcessed()
#6 /html/core/modules/migrate/src/MigrateExecutable.php(164):
Drupal\migrate\Plugin\Migration->checkRequirements()
#7 /vendor/drush/drush/includes/drush.inc(720):
Drupal\migrate\MigrateExecutable->import()
#8 /vendor/drush/drush/includes/drush.inc(711):
drush_call_user_func_array(Array, Array)
#9 /html/modules/migrate_tools/migrate_tools.drush.inc(280):
drush_op(Array)
#10 [internal function]:
_drush_migrate_tools_execute_migration(Object(Drupal\migrate\Plugin\Migration),
'd6_custom_block', Array)
#11 /html/modules/migrate_tools/migrate_tools.drush.inc(246):
array_walk(Array, '_drush_migrate_...', Array)
#12 /vendor/drush/drush/includes/command.inc(422):
drush_migrate_tools_migrate_import()
#13 /vendor/drush/drush/includes/command.inc(231):
_drush_invoke_hooks(Array, Array)
#14 /vendor/drush/drush/includes/command.inc(199): drush_command()
#15 /vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67):
drush_dispatch(Array)
#16 /vendor/drush/drush/includes/preflight.inc(66):
Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#17 /vendor/drush/drush/drush.php(12): drush_main()
#18 {main}

I couldn't find a previous issue on this - hopeully this is not a duplicate. Do I need to install additional modules in D7?

Comment	File	Size	Author
#46	d7_migration_failing_on_d6_migrations-2868762-46.patch	1.09 KB	J-Lee
#43	d7_migration_failing_on_d6_migrations-2868762-43.patch	1.06 KB	webdrips
#42	d7_migration_failing_on_d6_migrations-2868762-42.patch	1.13 KB	webdrips
#41	d7_migration_failing_on_d6_migrations-2868762-41.patch	1.78 KB	webdrips
#40	d7_migration_failing_on_d6_migrations-2868762-40.patch	1.28 KB	webdrips
#37	d7_migration_failing_on_d6_migrations-2868762-37.patch	782 bytes	el1_1el
#16	d7_migration_failing_on_d6_migrations-2868762-16.patch	755 bytes	JamesOakley

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

11 April 2017 at 18:27

Matt B created an issue. See original summary.

Comment #2

mikeryan

he/him

English

Murphysboro, IL, USA

CreditAttribution: mikeryan as a volunteer and at Virtuoso Performance commented 10 May 2017 at 23:07

Title:	D& Migration failing on d6_filter_format	» D7 Migration failing on d6_filter_format
Status:	Active	» Postponed (maintainer needs more info)

Can you try again with the migrate_plus and migrate_tools 8.x-4.0-beta1 versions? I suspect the plugin manager issues fixed in those releases might be behind what you're seeing...

Comment #3

Matt B

English

CreditAttribution: Matt B commented 22 May 2017 at 19:44

Hi Mike, same issue with migrate_plus and migrate_tools 8.x-4.0-beta1

Comment #4

heddn

English

Nicaragua

CreditAttribution: heddn at MTech, LLC, Acro Commerce commented 28 August 2017 at 15:36

Status:	Postponed (maintainer needs more info)	» Closed (duplicate)
Related issues:		+#2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission

This is the same issue as in #2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission.

Comment #5

dalinian CreditAttribution: dalinian commented 8 November 2019 at 03:05

Can someone explain why the d6_filter_format is relevant when doing a Drupal 7-8 upgrade. I am still experiencing this problem and looking at all the proposed solutions in the various referenced issues, I have found no answers or workarounds. Anyone know how to resolve this yet?

[Exception]
d6_filter_format migration failed.

Comment #6

junaidpv

Kannur, Kerala

CreditAttribution: junaidpv at Detlus for Wickwood Marketing commented 6 May 2020 at 11:32

Me too having same question as dalinian.

Does d6_filter_format really relevant when migrating from D7 site?

Comment #7

ressa CreditAttribution: ressa at Ardea commented 6 May 2020 at 12:40

I also pondered over this ... ideally it should be possible to define if you are interested in migrations for either D6, D7 or both. It gives a lot of extra noise in the output.

Comment #8

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 8 May 2020 at 04:43

I too was very confused to see numerous 'd6_...' migration items appear in the list and then (not surprisingly) fail, in a D7->D8.8.5 migration.

For me it was more than just "noise" -- the migration fatally aborted.

I don't understand what tricks or triggers the migration process to thinking a D7-to-D8 migration has anything to do with D6 or that it's just a case of old d6 names not having been updated.

Anyway, I figured ignoring anything that starts with 'd6_..' was probably worth trying and so I patched the source code to do just that.
This did the trick for me, in that ALL errors disappeared.

You can skip 'd6_...' migration items by inserting this code, at line 713 of:
web/modules/contrib/migrate_tools/src/Commands/MigrateToolsCommands.php, function migrationsList

    // Strip out D6 junk.
    foreach ($matched_migrations as $id => $migration) {
      if (substr($id, 0, 3) == 'd6_') {
        $notice = dt('Migration @id skipped.', ['@id' => $id]);
        $this->logger()->notice($notice);
        unset($matched_migrations[$id]);
      }
    }

PS: I disagree with the assertion in comment #4: I do not believe this to be a duplicate of #2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission

Comment #9

ressa CreditAttribution: ressa at Ardea commented 8 May 2020 at 10:02

Version:	8.x-3.0-rc1	» 8.x-3.2
Status:	Closed (duplicate)	» Active

Thanks for sharing your experimentation @RdeBoer, I am reopening the issue based on your feedback.

Perhaps a setting somewhere ([x] Exclude Drupal 6 migrations ?) could toggle your Strip out D6 ... code? Since the vast majority of migrations are coming from Drupal 7 to Drupal 8, it might even be enabled by default.

Comment #10

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 8 May 2020 at 10:14

Thanks @ressa,

Yes something along those lines would definitely be an improvement.
I needed something quick as I have a deadline for a client. It wasn't intended as a complete solution.

Alternatively, the Migration suite of modules could become more intelligent about detecting D6 "remnants" in a D7 database.
Migrations are not my expertise and I did not have time to get to the bottom of this.

I've also just realised that my fix is actually in the Migration Tools module, not this module, Migrate Upgrade.

Anyway, I hope it helps those who landed here, like I did, after much frustration.

Comment #11

bisonbleu CreditAttribution: bisonbleu commented 10 September 2020 at 01:53

Same problem. Inserted code from #8 in migrate_tools-8.x-5.0:717 and it did the trick. Thanks @RdeBoer

    // Do not return any migrations which fail to meet requirements.
    /** @var \Drupal\migrate\Plugin\Migration $migration */
    foreach ($matched_migrations as $id => $migration) {
      if (substr($id, 0, 3) == 'd6_') {
        $notice = dt('Migration @id skipped.', ['@id' => $id]);
        $this->logger()->notice($notice);
        unset($matched_migrations[$id]);
      }
      try {
        ...

Comment #12

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 10 September 2020 at 11:39

@bisonbleu
Glad it worked for you!
RIk

Comment #13

mducharme CreditAttribution: mducharme as a volunteer commented 22 September 2020 at 14:09

#8 worked for me as well.

Comment #14

sjhuskey CreditAttribution: sjhuskey as a volunteer commented 9 October 2020 at 22:18

#8 fixed at least some of my problems. Thank you very much, @RdeBoer!

Comment #15

Alejo D

English

Buenos Aires

CreditAttribution: Alejo D commented 19 November 2020 at 14:39

Hey,

Isn't necessary apply that patch, you are able to use the --group flag to skip D6 migrations

drush migrate-status --group="migrate_drupal_7"

Thanks!

Comment #16

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 20 January 2021 at 19:23

Title:	D7 Migration failing on d6_filter_format	» D7 Migration failing on Drupal 6 migrations
Project:	Migrate Upgrade	» Drupal core
Version:	8.x-3.2	» 9.2.x-dev
Component:	Code	» migration system
Category:	Support request	» Bug report

File	Size
d7_migration_failing_on_d6_migrations-2868762-16.patch	755 bytes

I'm having the same problem.

Here's the code snippet from #11 turned into a patch, so I can include it in my cweagans composer patch list. The patch applies to the migrate_tools module.

But: I think the problem is actually in core D9 migrate code and not in the migrate_upgrade or migrate_tools module. I say that because if I tried to run a Drupal 7 to Drupal 9 upgrade in Drush, it fails unless I apply this patch to stop Drupal 6 migrations causing the process to fail. However, if I run the upgrade in the browser using Migrate UI, it applies a certain number of the migrations and then falls over with a 500 error. So I suspect the browser-based upgrade is hitting the same problems.

This can be fixed for Drush upgrades by patching migrate_tools as here. However the real problem is Drupal 6 migrations being queued for a site that was itself an upgrade from Drupal 6 to Drupal 7. I don't know the migrate codebase well enough to pinpoint it, but I suspect that there's a fix there somewhere that will then see Drush and browser upgrades run successfully.

Comment #17

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 20 January 2021 at 19:39

I should perhaps add that it wasn't d6_filter_format that tripped up my migration. I got an error that it couldn't find the "vocabulary" table, as part of migrating the Drupal 6 taxonomy system. Of course, the taxonomy_vocabulary table did exist and was migrated successfully, but the D6 vocabulary table had not existed since the D6->D7 upgrade.

Comment #18

quietone CreditAttribution: quietone as a volunteer commented 21 January 2021 at 02:29

Title:	D7 Migration failing on Drupal 6 migrations	» D7 Migration failing on d6_filter_format
Status:	Active	» Closed (duplicate)
Related issues:	-#2898028: filter_format_permission and d6_cck_file fails on unprefixed migration parameter for filter_format_permission	+#3142747: filter_formats doesn't exist Error

@JamesOakley, yes, those errors are an unfortunate side effect of how migration works.

There are two separate issue here.

1) SQL errors from drush ms
When running drush migrate-status all the migrations are discovered and an attempt is made to get a count of the source rows by accessing the source database. Of course, some of those will fail if the source database is for D6 and the migration is for D7 and similar if the source database is for D7 and the migration is for D6. These errors can be ignored or you can filter the output of the drush command.

2) Getting a 500 error upgrading from the UI
When running a migration from the UI, /upgrade, the user must select the source database version and only migrations tagged for that version will run. To solve this the first place to look is at the logs, it is likely from the last migration to run.

Now what to do,
This issue was originally scoped for the first issue. Let's keep it that way, so restoring the original title and closing as works as designed. This is also a duplicate of #3142747: filter_formats doesn't exist Error.

For the second problem, open a new issue to discuss that.

Thx.

Comment #19

quietone CreditAttribution: quietone as a volunteer commented 21 January 2021 at 02:30

Issue tags:

+Bug Smash Initiative

Comment #20

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 21 January 2021 at 09:41

Project:	Drupal core	» Migrate Upgrade
Version:	9.2.x-dev	» 8.x-3.x-dev
Component:	migration system	» Code
Related issues:		+#2909256: D7 to D8 Upgrade Looking for Non-Existing D6 Tables

Thanks for explaining.

Seeing as you set the issue title back to what it was, I've set the project back to the original Migrate Upgrade as well.

The issue you linked when you closed this as duplicate is itself set to "Closed (duplicate)" and points to this one. That's a little circular for comfort. I suspect the real original version of this issue is #2909256: D7 to D8 Upgrade Looking for Non-Existing D6 Tables, which I see that you @quietone have set to "Works as designed" and explained why.

Comment #21

brooke_heaton CreditAttribution: brooke_heaton as a volunteer commented 28 January 2021 at 04:15

I honestly think there needs to be a better way to exclude migration groups from the command. In my case I don't have JUST a "migrate_drupal_7" group because I'm also using the "commerce_migrate" module which uses the 'default' group for some reason. So there's no method for me to not see these errors. There really has to be a better way forward. I don't think this issue should be closed.

Comment #22

brooke_heaton CreditAttribution: brooke_heaton as a volunteer commented 28 January 2021 at 04:18

The solution in comment #8 works like a charm but is obviously hacky. I'd suggest an --ignore-groups flag for the migrate:status command.

Comment #23

junaidpv

Kannur, Kerala

CreditAttribution: junaidpv at Detlus for Wickwood Marketing commented 28 January 2021 at 10:35

Suggestion from comment #8 is not good as it will skip migration that designed for both D6 and D7. For example, we have migrations d6_d7_acl_node, d6_d7_acl_user and d6_d7_acl from the ACL module, which would get prevented by the patch.

I am using this command as suggested in comment #15, which will execute all D7 migrations.

drush migrate-status --group="migrate_drupal_7"

Comment #24

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 28 January 2021 at 10:43

Without that patch or one like it, drush migrate-upgrade never completes successfully for me (on a site that was once D6 before being upgraded to D7).

@junaidpv, you suggest running drush migrate-status --group="migrate_drupal_7". But that comes later. Is there also a switch for drush migrate-upgrade, when we are collecting the available migrations, the would also hide the D6 ones?

Comment #25

junaidpv

Kannur, Kerala

CreditAttribution: junaidpv at Detlus for Wickwood Marketing commented 28 January 2021 at 11:19

@JamesOakley drush migrate-upgrade does not have option choose a group.

One dirty hack I follows with some other migration is to delete the YAML files for the migrations which I am certain not make sense for my site.

So, deleting "core/modules/filter/migrations/d6_filter_format.yml" (temporarily) in D8 site also will help to avoid running that migration.

Comment #26

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 28 January 2021 at 11:21

Whatever the perfect solution may turn out to be, it is clear from all of the above comments that we have a real multi-faceted problem here.

I therefore second @brooke_heaton and @JamesOakley.
This issue should not be closed. Lots of people land here with a stuffed up migration looking for answers.
Just closing the issue doesn't help anyone.

Comment #27

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 28 January 2021 at 11:35

Status:

Closed (duplicate)

» Active

I've set it back to active, and leave it to someone else to explain why it's an exact duplicate of another issue if they think it should be re-closed.

If this is a case of "works as designed", then this becomes instead an active documentation issue, since the main documentation page does not at all explain the steps to follow with a site that was previously migrated from Drupal 6 to Drupal 7.

But I'm not sure it is "works as designed" given (certainly in my case) drush migrate-upgrade fails silently unless a hack like the ones in #8 or #25 is used to modify the migration code.

Comment #28

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 28 January 2021 at 11:49

Status:

Active

» Closed (duplicate)

On ya, @JamesOakley
👍

Comment #29

RdeBoer

Melbourne

CreditAttribution: RdeBoer as a volunteer commented 28 January 2021 at 11:54

Status:

Closed (duplicate)

» Active

Comment #30

Alejo D

English

Buenos Aires

CreditAttribution: Alejo D commented 28 January 2021 at 16:12

Maybe a good feature could be if the script is able to detect what kind of DB are you trying to migrate to D8/D9. Then you don't need to apply the group filter to prevent the D6 error.

Works with the group flag ? Yes.
What will do a new Dev without knowledge on this module, execute the script without flag and it will throw D6 error.

Else this module need to have a better Doc to understand how this works. On my case i had to do a lot of research to understand migration.

Comment #31

huzooka

Hungarian

Hungary 🇭🇺🇪🇺

CreditAttribution: huzooka at Cheppers for Acquia commented 5 May 2021 at 18:43

Is this #3123950: 'Field discovery failed for Drupal core version 7' errors if migration source database key is not 'migrate'?

Comment #32

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 5 May 2021 at 18:55

@huzooka, no because I was using the database key of "migrate" when I hit this problem.

Comment #33

Dan_i_am CreditAttribution: Dan_i_am commented 22 June 2021 at 04:49

I'm using drush 10.4 (the default installed for drupal 9 now) so these patches to Migrate Tools can't be used as that module is incompatible with this drush version. Also deleting "core/modules/filter/migrations/d6_filter_format.yml" from #25 hasn't worked for me. I'm completely blocked on this. It seems like a pretty serious issue to me.

Comment #34

huzooka

Hungarian

Hungary 🇭🇺🇪🇺

CreditAttribution: huzooka at Cheppers for Acquia commented 22 June 2021 at 06:23

You should execute/export only those migrations which are tagged with Drupal 7.

Comment #35

aleix CreditAttribution: aleix commented 9 July 2021 at 11:58

So on
drush migrate:import --tag="Drupal 7" seems to go ahead.

Comment #36

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 9 July 2021 at 18:45

@aleix

Not quite. See my post #24 above and the reply to it in #25. The problem appears before we get to drush migrate:import.

Comment #37

el1_1el CreditAttribution: el1_1el commented 17 February 2022 at 21:47

File	Size
d7_migration_failing_on_d6_migrations-2868762-37.patch	782 bytes

Adding new patch for 8.x-3.2. Thanks!

Comment #38

firfin CreditAttribution: firfin commented 6 December 2022 at 21:41

Status:

Active

» Closed (works as designed)

Use --tag not --group, problem solved.

Comment #39

JamesOakley

he/him/his

English

Kent, UK

CreditAttribution: JamesOakley as a volunteer commented 7 December 2022 at 09:46

Status:

Closed (works as designed)

» Active

@firfin, could you please explain a little more exactly by "use --tag"? Because the problem being discussed in this issue is one that happens when before people get to use --tag on migrate:import (as I explained to @aleix in #36).

What exact workflow are you suggesting, with what commands, such that the people are reporting failing migrations simply because they doing it wrong?