I was trying to upgrade around 5k nodes with image field, all went supposedly good (no errors). But the operation ended up having a bunch of user pictures replaced with files from those node fields. Puzzled.

Less busy content type (a few dozen nodes) was upgraded just perfect.

Any pointers where to look to try to track/fix this behavior?

Comments

protoplasm’s picture

I'm having the same issue. Files are being overwritten in files_managed table and it appears permanent status in the table does not mean it can't be overwritten. This is what is currently holding up my upgrade of a large social network to D7. Has anyone figured out a solution for this?

protoplasm’s picture

Priority:Normal» Critical

I'm changing this priority to critical. I've managed to jump every undocumented hurdle on upgrading a social network from D6 to D7 so far except this one. No matter how I tweak it, migrating content is overwriting my account user pictures which are already in the files_managed table. Does anyone have any insight into why or how this might occur?

protoplasm’s picture

Well, I've found the issue and so I will answer my own question. I'm not sure how it has happened, but some files have the same FID as the FID of the user pictures and so when content is migrated, the files FID is overwriting the existing picture in the files managed section. Don't know how or when this occurred or how to fix it yet, but that appears to be the issue. So I'm beginning to doubt this is a bug in content migrate.

protoplasm’s picture

betarobot’s picture

Priority:Critical» Normal

@protoplasm checked my db and spotted the same: some files would have same FID in d7. But for me it would happen after conversion. Initial database looks just fine. Checking for other pointers, but still see none.

protoplasm’s picture

Hard to imagine no one else has had this issue.

melvix’s picture

Priority:Normal» Critical

I am having this issue. It is absolutely a bug in content_migrate. I would classify it as critical, since it will destroy your data and prevent you from successfully upgrading and deploying a stable site.

It looks like content_migrate_filefield_data_record_alter() will overwrite any records that already exist in the files_managed table, which is where your user avatars will be after you've run all the user module updates.

More specifically, it is the use of db_merge() instead of file_save().

protoplasm’s picture

This very issue has me at a complete standstill in upgrading to D7. In the last six years of using Drupal, I've never found an issue so frustrating as this or stopped me in my tracks. Clearly, the issue begins with content_migrate's overwriting so I believe you're right.

Do you have a work around or solution?

melvix’s picture

This isn't really a solution, but what I did was stash a copy of the users table before the upgrade, then re-import the avatars after content_migrate was done doing its thing using a custom script.

adams.garfield’s picture

Even i have the same problem.Gonna try out this. Importing the avatars after content_migrate will do the trick.!!!
Thanks

protoplasm’s picture

@melvix, do you have a solution you can share?

winniepoo’s picture

I have the same problem.
It's needed to change file fid in content_migrate module if there is file in file_managed table with this fid.
Any solutions?

fugazi’s picture

same problem

protoplasm’s picture

Still an issue, still can't upgrade to 7. If someone could detail their solution, it would be most appreciated.

mrsean’s picture

This is still an issue occurring during a D6 to D7 migration that I am doing...
It would certainly be wonderful if someone would be able to contribute their custom script... @melvix, care to share?

Thanks!

mrsean’s picture

OK I fixed this myself. here is my fix. It also fixes the format of the picture uri's so that Drupal 7 can create image-styles for all of the user pictures, which was broken after i migrated from D6 to D7.

Feel free to use as you wish. I'm not respsonsible if this breaks your site and you dont have a backup of your db. Backup your db and test this on a test version of your site first before doing on prod!

1) before migration, save the following in your D6 webroot as get_user_pictures.php. Then go to the page in a browser to get a list of users who have pictures (www.drupal6site.com/get_user_pictures.php).
Code:

<?php
error_reporting
(E_ALL);

// set HTTP_HOST or drupal will refuse to bootstrap
$_SERVER['HTTP_HOST'] = 'drupal6site.com';
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';

define('DRUPAL_ROOT', dirname(realpath(__FILE__)));

include_once
'includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// Total Posts of this type, for the group
$results = db_query("SELECT users.uid AS uid, users.picture AS users_picture, users.name AS users_name FROM users  WHERE users.picture <> '' ");

while (
$row = db_fetch_array($results)) {
  print (
'"'. $row['uid'] . '"'. " => " . '"'. $row['users_picture'] . '"' . ",<br/>");
}
   
?>

2) Drop the following into your D7 upgraded site webroot as fix_user_pictures.php:

<?php
// Script to migrate user pictures to the appropriate place
// See https://drupal.org/node/1558192#comment-8176679 for more details on why this is necessary
define('DRUPAL_ROOT', getcwd());
require_once
DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// Fill with content from
// https://drupal6site.com/get_user_pictures.php
$user_picture_files = array(
/*
COPY OUTPUT OF get_user_pictures.php, WHICH WAS RUN ON YOUR D6 SITE, TO THIS ARRAY.
EXAMPLE FORMAT OF DATA:
"5" => "sites/default/files/pictures/picture-5-2c086d2efd5c1f41487406efc4cc60d9.jpg",
"7" => "sites/default/files/pictures/picture-7-0b4612c44c01f1cad30302af41747e5c.jpg",
"10" => "sites/default/files/pictures/picture-10.jpg",
*/

);

// Variables to keep track of end results
$no_change_needed = 0;
$no_file_path = 0;
$no_actual_file = 0;
$files_corrected = 0;

// Get a list of all users who have non-empty picture fields
$result = db_query("SELECT n.name, n.uid, n.picture FROM {users} AS n WHERE n.picture!='' ");

// For each of these users...
foreach ($result as $user_record) {
 
   
// Get file details of file currently associated as user picture from db
   
$filename_result = db_query("SELECT fid, uri FROM {file_managed} WHERE fid=:fid ", array(':fid'=> $user_record->picture)); 
   
$file_record = $filename_result->fetchAssoc();
       
    print(
"<br/><br/>User ". $user_record->uid . "<br/>" );
    print(
"Picture Location retrieved from drupal 6 site: " . $user_picture_files[$user_record->uid] . "<br/>" . "DB Picture Location: " . $file_record['uri']  . "<br/>");
   
   
// Compare filename pulled from file_managed table to filename retrieved from drupal 6 site
   
if ($file_record['uri'] === $user_picture_files[$user_record->uid]) {
     
// If a match, no need to fix
     
print " DB picture location matches picture location retrieved from drupal 6 site. No changes needed.<br/>";
     
$no_change_needed++;
    }   
    else {
   
      if(!
$user_picture_files[$user_record->uid]) { 
       
// We don't know what we want to set the user picture to;
        // Bail out as we don't know what to do now.
       
print("No user picture location retrieved from drupal 6 site! <br/><br/>");
       
$no_actual_file++;
        continue;
// end loop iteration
     
}
      if ( !
file_exists($user_picture_files[$user_record->uid] ) ) {       
       
// The file we want to assign as the new user picture doesn't actually exist;
        // Bail out as we don't know what to do now.     
       
print("Physical user picture file NOT FOUND! <br/><br/>"); 
       
$no_file_path++;       
        continue;
// end loop iteration
     
}
   
     
// If db filename and filename retrieved from drupal 6 site don't match, output info
     
print " DB picture DOES NOT MATCH picture file. <br/>";
     
     
//========== Attempt to replace db file name with filename retrieved from drupal 6 site==================
     
      // Add file to db
     
$path_parts = pathinfo($user_picture_files[$user_record->uid]);
           
     
     
// Check if file already exists
     
$already_in_files = entity_load('file', FALSE, array('filename' => $path_parts['basename']));     
     
      if (!empty(
$already_in_files)) {
       
// If you get here, you've most likely already run this script and are re-running it.
        // So, bail out.
       
print("<span style='background-color:yellow;'><strong>picture location retrieved from drupal 6 site already exists:</strong></span> ");
       
print_r ($already_in_files) ;
       
//$file->fid = key(reset($already_in_files));
       
$no_change_needed++;
        continue;       
       
//print " xxx " . $file->fid;
     
}
      else {
       
// Add file to file_managed table
       
$file = new stdClass;
       
$file->uid = $user_record->uid;
       
$file->filename = $path_parts['basename'];
       
$file->uri = $user_picture_files[$user_record->uid];
       
$file->status = 1;
       
$file->filemime = 'image/' . $path_parts['extension'];

       
$file = file_save($file);
       
file_usage_add($file, 'user', 'user', $user_record->uid);     
      }
     
     
// Update user picture
     
db_update('users')
        ->
fields(array(
       
'picture' => $file->fid,
        ))
        ->
condition('uid', $user_record->uid)
        ->
execute();
     
     
$files_corrected++;
     
//exit;
   
}      
}

// Summary
print("<br/><br/>No need to fix: $no_change_needed <br/>Fixed: $files_corrected<br/>Could not fix due to no physical file: $no_actual_file <br/>Could not fix due to no file path in $ user_picture_files: $no_file_path ");
 
// Finally, fix all user picture locations by setting "sites/default/files/pictures" to "public://pictures".
// This will allow image styles for these pictures to be created.
db_update('file_managed')
  ->
expression('uri', "replace(uri, 'sites/default/files/pictures', 'public://pictures')")
    ->
execute();
?>

3) Copy the output from the script in step 1 into the script created in step 2, above the comment "COPY OUTPUT OF... HERE", within the $user_picture_files array.

4) On your d7 site, go to "drupal7site.com/fix_user_pictures.php".

5) Read the output to see what was fixed and what wasn't.

6) View the user profiles on your d7 site to see if the broken pictures were fixed!

darkdim’s picture

Hi!
# 17 I'm afraid this is not a correct approach.
If we correct user picture (avatar), we thus be overwritten migrated cck field.
Or am I wrong?

darkdim’s picture

Who were able to solve this problem?

xqi’s picture

I also run into this issue when upgrading my site (drupal 6) to drupal 7.26.

some users' pictures are removed from the database. some users' pictures got replaced by irrelevant files, such as PDF etc... the physical files are still there but not showing in the database.

this is definitely a bug and I wonder there is any patch for this or not. I googled this issue but it looks like only a small amount of sites run into this issue. it makes me wonder if it has something to do with the site's configuration, such as orphaned data

anyway i am subscribing.