Tried to use content_migrate and encountered repeated errors. (see next post)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

solotandem’s picture

Status: Active » Needs review
FileSize
5.54 KB

Simple patch that cleans up the items referred to above.

JoshOrndorff’s picture

This looks promising. At the very least it cleans up coding standards and fixes the fatal error referenced at http://drupal.org/node/1154508

jelenex’s picture

I have a suspicion that these errors show up only in PHP 5.3.x. Can somebody confirm this perhaps?
Anyway, this patch works for me, in a sense that the migration process doesn't crash in the beginning. Still, I'm getting empty DB columns (no data is migrated for any field) but that's probably for another issue..

solotandem’s picture

FileSize
7.66 KB

An improved patch that also deals with not writing records because it mistakenly determined the record was empty (column names like 'field_name_value' need to have 'field_name' removed).

JoshOrndorff’s picture

Awesome. The patch in #4 did it for me. It fixes the fatal error I mentioned earlier, removes some extra white space, and fixes the data migration issue that jelenex mentioned in #3.

Without this patch I was getting the fatal error, but solved it by manually removing the break. After that I was also experiencing fields being migrated but no data (like #3) and this patch cures both.

I'd be happy to provide any more feedback that I can. I'm tempted to mark it RTBC, but maybe someone else could test it out too? jelenex, could you give the patch in #4 a test?

Thanks,
-Josh

JoshOrndorff’s picture

Status: Reviewed & tested by the community » Needs review

Also, this should be elevated since it fixes at least two different problems that completely break the module. I've referenced this post and patch from the issue about the critical error.

-Josh

imclean’s picture

Priority: Normal » Critical

This appears to fix #1100906: AJAX HTTP Error on migrating fields. The field is created and the data migrated without error to the corresponding field_data_ table, however the populated fields aren't appearing on the relevant nodes. The correct nids are listed in the field_data_ table.

ju1i3’s picture

I'm getting this problem with PHP 5.2.14 so it isn't just 5.3.x.

I will install and test patch 4 later today and report back.

jelenex’s picture

Status: Needs review » Reviewed & tested by the community

I've tested the patch #4 with various field types (text, number, file, email) and several thousands of records. So far everything worked well (as opposed to before), so I'm marking this RTBC.

@imclean I'd suggest opening another issue, because that sounds like a different kind of problem. This issue already solves two pretty much different things and would quickly become too bloated.

BTW, I've already posted a fix to the empty data problem in different thread (#1150790: Empty fields show up on migrated nodes) but my solution is less elegant, so I'm closing the other issue..

xevious’s picture

Status: Needs review » Reviewed & tested by the community

I'm having a terrible time with this. I have a CCK image filed with thousands of images that I need to convert to D7. I'm thinking this page might be the fix but I have not been able to get the Patch to work.

An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /batch?render=overlay&id=2493&op=do StatusText: OK ResponseText: Fatal error: Cannot use string offset as an array in /home/drupal7/public_html/sites/all/modules/cck/modules/content_migrate/includes/content_migrate.admin.inc on line 252

Could someone give me idiot instructions for applying this patch?

$ patch < patch -p0 < 1157234-cm_0.patch

xmacinfo’s picture

@xevious: Try -p1 instead.

xmacinfo’s picture

I can confirm that the patch in #4 fixed the problem in #1100906: AJAX HTTP Error on migrating fields.

It's RTBC for me as well.

mstrelan’s picture

I can confirm that the patch in #4 fixed the problem in #1100906: AJAX HTTP Error on migrating fields

It's RTBC for me as well.

solotandem’s picture

FileSize
7.94 KB

An improved patch that uses the correct variable in message (otherwise identical to patch in #4).

marcoka’s picture

#4 fixed the problem, migrate successfull

ju1i3’s picture

I'm getting slightly further after installing the patch (#4). Now I get

An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /batch?render=overlay&id=43&op=do StatusText: OK ResponseText:

some openlog and syslog error messages and then:

{"status":true,"percentage":"19","message":"Processed 3 out of 16.\u003cbr \/\u003e\"Creating field: \u003cem class=\"placeholder\"\u003efield_photo\u003c\/em\u003e"}

xmacinfo’s picture

@ju1i3: Please post which modules or type of fields you tried to migrate. Looks like you were able to successfuly migrate 3 fields.

aspafford’s picture

#14 is working for me. It fixed a php error I was getting ("Fatal error: Cannot use string offset as an array ... content_migrate.admin.inc on line 252"), and content is now being imported correctly.

JoshOrndorff’s picture

+1 for committing #14. This patch already solves more than one problem, so lets try not to report additional issues on this page.

@juli3 try the patch in #14, and if it still doesn't fix your problem, maybe try opening another issue.

-Josh

ju1i3’s picture

Sorry, I can't say which field types, I'm embarrassed to say I didn't note them - it's a test system and I'm having so many problems with Drupal 7 I just reran the migrate fields until it finished them all - some worked, some didn't. If I have any further problems with this on any of my other systems I will update this or as you say open another issue, thanks for your help, Julie

KarenS’s picture

Taking this in bits. I fixed the missing filefield break in a different issue. There are a few typos in the drush file that have nothing to do with any fatal errors, so I fixed them.

This part does nothing but change the name of the variable, which was not necessary.

-        foreach ($instance_value['display'] as $context => $settings) {
-          if ($context == 'label' || $context == 'parent' || $context == 'weight') {
+        foreach ($instance_value['display'] as $display_mode => $settings) {
+          // @todo Is the value being compared to $display_mode supposed to be compared to a key in $settings?
+          if ($display_mode == 'label' || $display_mode == 'parent' || $display_mode == 'weight') {

$context is what that value was called in the D6 data that we are reading from. The D7 field module uses $view_mode now, earlier versions still used $context. This patch for some reason changes it to $display_mode, which doesn't match either the way this was done in D6 or the way it is done in D7. It didn't hurt anything to leave that value alone and it makes the patch harder to evaluate. We should either use the D6 standard or the D7 standard, not make up a new variable. So I have to find all the places that variable was changed and see what else is actually different. And the answer to the question is that those are values that might exist as keys in the $settings array, but they won't always be there.

Now digging through the rest of the patch.

solotandem’s picture

Karen, the point of the rename is you are reusing the name of $context that is passed into the function as a parameter (by reference no less), and the potential for contamination of that variable exists and does occur, and causes other things to fail. So, yes the change is necessary. Further, the value of that key is the "build mode" -- hence the rename.

The point of the comment is that $context will never equal the things you are checking for -- rather the test should be against $settings[something] == $some_value. The "continue" is never reached as those conditions are never met, and the whole clause could be removed.

KarenS’s picture

I see that you found a couple things that were flat-out mistakes, so I'm adding those fixes.

I was puzzling over this next bit, but I think I see what is going on and why it might be broken. The 'is_empty' function wants the simple column name, not the actual db column name. There may be a cleaner way to fix it tho. We probably shouldn't be assuming the column name will always be correctly identified by doing a str_replace() of the field name, even though in most if not all cases that will actually work. We should have that info stored in our field array or one of the field info functions, I just have to figure out where it went in D7 and use it.

+        foreach ($record as $column => $value) {
+          if (strpos($column, $field_name) === 0) {
+            // Add an item for the field without the field_name prefix.
+            // Field API stores the column key without the field_name prefix,
+            // but drupal_write_record() wants the actual column name.
+            $record[str_replace($field_name . '_', '', $column)] = $value;
+          }
+        }
KarenS’s picture

I finally figured out what you were trying to say about $context, you are absolutely right about that. I think I have most of this sorted out now but want to test it a bit. I may not get this committed until tomorrow.

KarenS’s picture

Status: Reviewed & tested by the community » Fixed

I made some additional changes and committed this. Thanks so much for banging through this!

aaronpinero’s picture

Status: Fixed » Active

Sorry I have to mark this as unfixed. I applied the patch listed in #4 and it did remove the AJAX errors but didn't fix the underlying problem (which I first mentioned here http://drupal.org/node/1100906) that none of the content in any of the converted fields is preserved. Everything comes through totally empty. Conformed with a look through the DB (a whole bunch of tables with zero rows). So the patch appears to have been cosmetic only.

solotandem’s picture

Before drawing a conclusion, what dev release did you apply the patch to? Karen just committed some changes that include most of the changes in #4. At this point, you should be testing her latest release.

aaronpinero’s picture

Status: Active » Fixed

@solotandem You're right. Thank you. I was using a (now slightly) outdated version of the module which I had patched, rather than the one committed recently. I installed the new version, refreshed everything from my DB backup and tried again. This time content is being preserved when fields are converted. The issue indeed seems to be fixed.

Thanks for drawing my attention to my oversight, and thank you @KarenS for the update. I was having a devil of a time trying to determine if there was really an issue or if I had done something wrong in my upgrade.

Cheers,
Aa.

bryancasler’s picture

Using the current dev, no patches, this error is still being thrown on two separate occasions. Should I wait for this to be resolved or can it be ignored?

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /sandboxsite4/batch?id=1992&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://IVAWChapterapplication1.doc' for key 'uri': UPDATE {file_managed} SET uid=:db_update_placeholder_0, filename=:db_update_placeholder_1, uri=:db_update_placeholder_2, filemime=:db_update_placeholder_3, filesize=:db_update_placeholder_4, status=:db_update_placeholder_5, timestamp=:db_update_placeholder_6 WHERE ( (fid = :db_condition_placeholder_0) ); Array ( [:db_update_placeholder_0] => 1 [:db_update_placeholder_1] => IVAWChapterapplication1.doc [:db_update_placeholder_2] => public://IVAWChapterapplication1.doc [:db_update_placeholder_3] => application/msword [:db_update_placeholder_4] => 206336 [:db_update_placeholder_5] => 1 [:db_update_placeholder_6] => 1269026651 [:db_condition_placeholder_0] => 93 ) in content_migrate_filefield_data_record_alter() (line 168 of C:\xampp\htdocs\sandboxsite4\sites\all\modules\cck\modules\content_migrate\modules\content_migrate.filefield.inc).
solotandem’s picture

@animelion That error results from a change in the file handling between D6 and D7. In D6, you could have the same file (the uri field) in multiple rows of the file table. In D7, the uri is a unique key -- so no duplicates allowed. I encountered this issue too.

I would suggest you:
-- search your database for duplicate uri's,
-- pick one row to keep,
-- change the references in other tables to the file table id you keep,
-- and delete the duplicate uri's from the file table,
-- rerun the content migration.

BTW, The file table issue is unrelated to this issue, and should be filed as a separate issue.

bryancasler’s picture

Thanks solotandem,

I wasn't exactly sure which thread my error was related to. I think this one over here makes more sense now.
#781088: Updating CCK Fields and Data from D6 to D7 #152

I'm going to try and figure this out today. Is there a sexy SQL command I might use to help in this hunt, or did you do it all by hand?

solotandem’s picture

In the D6 database, try:

SELECT COUNT(*) AS Rows, filepath
FROM files
GROUP BY filepath
ORDER BY Rows DESC

bryancasler’s picture

Thanks solotandem,

I'm following up in the other thread as not to clutter this one. Long story short, I didn't find any duplicates.
http://drupal.org/node/781088#comment-4545832

UPDATE: This discussion is now being continued here #1176186: D6 -> D7 upgrade: Duplicate files cause Integrity constraint violation: 1062 Duplicate entry 'xx' for key 'PRIMARY'

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.