Today I used Migrate UI to migrate a d6 site to d8. See #2793947-14: D6 migration testing with i18n for more context on the migration and what was in my d6 site.

I got an error in my dblog:

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'poplarware_d6.profile_fields' doesn't exist: SELECT COUNT(*) AS expression FROM (SELECT 1 AS expression FROM @profile_fields pf) subquery; Array ( )

Then the next message in dblog is:

Operation on User profile field instance configuration failed

I did not have the d6 core Profile module enabled on my site, and I do not have the profile_fields table in my d6 database. I also looked in the variables table on my d6 site, and I don't see anything in there that looks like user profiles (there are two variables with "profile" in the name, but one is for the Backup and Migrate module, and the other is for the install profile).

I think that Migrate should not have tried to migrate these settings, since I wasn't using that module.

CommentFileSizeAuthor
#26 interdiff-24-26.txt546 bytesquietone
#26 2859315-26.patch5.12 KBquietone
#24 interdiff-18-24.txt387 bytesquietone
#24 2859315-24.patch4.91 KBquietone
#18 2859315-18.patch4.34 KBquietone
#18 interdiff-10-18.txt3.91 KBquietone
#10 2859315-10.patch1000 bytesheddn
#9 2859315-9.patch790 bytesheddn
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jhodgdon created an issue. See original summary.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented

Using the ck2 fixture in commerce_migrate_commerce, I'm able to reproduce this issue when migrating from D7 as well.

Migration failed with source plugin exception: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'ck2.profile_field' doesn't exist: SELECT pf.*, map.sourceid1 AS migrate_map_sourceid1, map.source_row_status AS migrate_map_source_row_status FROM @profile_field pf LEFT OUTER JOIN drupal_sandbox.migrate_map_user_profile_field map ON fid = map.sourceid1 WHERE (map.sourceid1 IS NULL) OR (map.source_row_status = :db_condition_placeholder_0); Array ( [:db_condition_placeholder_0] => 1 )

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Active » Needs review
Issue tags: +Needs tests

Here's a start at a fix. Needs tests.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Needs review » Postponed (maintainer needs more info)
Issue tags: -Needs tests

I figured out what the issue was with #3 and possibly with the initial problem. The profile module was enabled but for some reason the tables for profile weren't in the database. Is that what was the problem from the initial post too?

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

No, as stated in the original issue summary, the Profile module was not enabled in my D6 site.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Postponed (maintainer needs more info) » Needs work

Hmm, head has a test with profile in d6 not enabled, so that is confusing...

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

Well, let me check, I still have the database for the D6 site..

So I logged in to a local copy of that old site. I verified that on admin/modules, the Profile module is not checked. And when I went to the Uninstall page, it is not shown there, so if it was ever installed, it was completely uninstalled rather than just disabled.

Looking at the database itself, I do see an entry in the system table for the profile module, but the status is 0 and schema version is -1. This is similar to other modules like the core Blog module that I never installed on that site.

As far as tables go in that database, there is no table named profile* at all in the database.

So... I guess the question is, why does Migrate think I have a profile module? Maybe it is checking some other way?

Let me know if you need more information from my D6 site... I don't want to attach the whole database dump (for privacy/security reasons), but I can dump/attach some specific tables if that would help.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Needs work » Needs review
FileSize
2859315-9.patch790 bytes

Does this fix things?

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
FileSize
2859315-10.patch1000 bytes

Helps if I include the import too. Ignore #9.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

I tested the patch in #10 as follows:

a) Downloaded the 8.5.x zip file from drupal.org/project/drupal (didn't want to use Git, zip file is easier, don't have to do the composer stuff)
b) Applied the patch.
c) Installed Drupal using Minimal install profile (using UI).
d) Switched themes (Bartik/Seven) so I could stand to look at the site. :)
e) Went to admin/modules and installed the Migrate, Migrate Drupal, and Migrate Drupal UI modules.
f) Went to /upgrade and clicked Continue.
g) Filled in form. Said it was coming from D6 and filled in database credentials and files location.
h) Reviewed list of unavailable (mostly) and 9 available upgrade paths and started the upgrade.

I got messages saying:

Completed 48 upgrade tasks successfully
1 upgrade failed

Upgrade process not completed
Review the detailed upgrade log

Looking in the log, there were a number of error messages, but not anything about Profile module this time. So I think this patch fixed the problem, although I didn't try the migration without the patch to verify that the error still occurred without the patch.

Do you want me to file issues about the other errors I saw? There appear to be problems with:
- custom blocks -- "The "block_content" entity type does not exist."
- field config -- "Source ID field_topic: Attempt to create a field storage of unknown type list_string. (/opt/www/d8migrate/core/modules/field/src/Entity/FieldStorageConfig.php:329)" [in the source site, the "field_topic" is a plain text field with an allowed values list]
- field config -- "Source ID field_images: Attempt to create a field storage of unknown type image. (/opt/www/d8migrate/core/modules/field/src/Entity/FieldStorageConfig.php:329)" [this is an image field, using the contrib FileField/ImageField modules, with multiple values allowed; got similar problems with a filefield file]
- field formatters on those fields
- There were also problems with the User Picture field on users

I'm not sure if these are known problems... I also had some errors with translated taxonomy terms but this I know is an existing issue.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Needs review » Needs work
Issue tags: +Needs tests

These extra errors all came because you installed minimal but didn't enable the file field and block content modules. No need to file an issue for me. Now we need to add a test for this.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
CreditAttribution: jhodgdon commented

That makes sense, regarding Minimal install profile. I guess I should have read that first screen more carefully. It does say:

In general, enable all modules on this site that are enabled on the previous site. For example, if you have used the book module on the previous site then you must enable the book module on this site for that data to be available on this site.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Issue tags: +migrate-d6-d8, +migrate-d7-d8

Adding tags

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Assigned: Unassigned » quietone

Been working on a test for this, almost done.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs work » Needs review
FileSize
interdiff-10-18.txt3.91 KB
2859315-18.patch4.34 KB

Seems to me that the test for the presence of the profile table is really a requirements check, so it is moved to checkRequirements(). And tests added for d6 and d7 .

Status: Needs review » Needs work

The last submitted patch, 18: 2859315-18.patch, failed testing. View results

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented 
+++ b/core/modules/user/src/Plugin/migrate/source/ProfileField.php
@@ -105,4 +97,32 @@ public function getIds() {
+  protected function setTableNames() {
+    if (empty($this->fieldTable) || empty($this->valueTable)) {
+      if ($this->getModuleSchemaVersion('system') >= 7000) {

In our quest to segment out D6 at some point, and since we are moving things around already can we make this more flexible so the d6 version of this thing can do this easier?

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs work » Needs review

Yes and no. I'm leaning to no because that is solving a different problem than what is in the IS. Yes, I'd prefer a separate issue for that.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Assigned: quietone » Unassigned

I'm leaning to no because this is a bug fix and not adding a new migration which tends to have wider scope.
Made an issue to make static variables for the relevant source plugins. #3029662: Use a static constant for D6/D7 tables is source plugins

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Assigned: Unassigned » quietone
Status: Needs review » Needs work

NW for test failures.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs work » Needs review
FileSize
2859315-24.patch4.91 KB
interdiff-18-24.txt387 bytes

Now that this is checking for the existence of the profile tables and failing (they do no exist in the the source) this the profile module does not appear in the will be upgraded any more.

Status: Needs review » Needs work

The last submitted patch, 24: 2859315-24.patch, failed testing. View results

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs work » Needs review
FileSize
2859315-26.patch5.12 KB
interdiff-24-26.txt546 bytes

And profile will be in the not upgraded list.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Assigned: quietone » Unassigned
Issue tags: -Needs tests

There are tests, and they are passing, so ready for review

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Title: SQL error from profile_fields when migrating d6 to d8 without Profile module » SQL error from profile_fields when migrating d6 (or d7) to d8 without Profile module
Status: Needs review » Reviewed & tested by the community

In D7, profile was a hidden module. So, slightly updating the title. And this is ready to go now. Plenty of test coverage.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented
Version: 8.7.x-dev » 8.6.x-dev
Status: Reviewed & tested by the community » Fixed

Committed f9132f4 and pushed to 8.6.x. Thanks!

  • catch committed d74e36a on 8.7.x 
    Issue #2859315 by quietone, heddn, jhodgdon: SQL error from...

  • catch committed f9132f4 on 8.6.x 
    Issue #2859315 by quietone, heddn, jhodgdon: SQL error from...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.