Migrating image fields ended up swapping files to userpic

I'm having the same issue. Files are being overwritten in files_managed table and it appears permanent status in the table does not mean it can't be overwritten. This is what is currently holding up my upgrade of a large social network to D7. Has anyone figured out a solution for this?

I'm changing this priority to critical. I've managed to jump every undocumented hurdle on upgrading a social network from D6 to D7 so far except this one. No matter how I tweak it, migrating content is overwriting my account user pictures which are already in the files_managed table. Does anyone have any insight into why or how this might occur?

Well, I've found the issue and so I will answer my own question. I'm not sure how it has happened, but some files have the same FID as the FID of the user pictures and so when content is migrated, the files FID is overwriting the existing picture in the files managed section. Don't know how or when this occurred or how to fix it yet, but that appears to be the issue. So I'm beginning to doubt this is a bug in content migrate.

Hard to imagine no one else has had this issue.

I am having this issue. It is absolutely a bug in content_migrate. I would classify it as critical, since it will destroy your data and prevent you from successfully upgrading and deploying a stable site.

It looks like content_migrate_filefield_data_record_alter() will overwrite any records that already exist in the files_managed table, which is where your user avatars will be after you've run all the user module updates.

More specifically, it is the use of db_merge() instead of file_save().

This very issue has me at a complete standstill in upgrading to D7. In the last six years of using Drupal, I've never found an issue so frustrating as this or stopped me in my tracks. Clearly, the issue begins with content_migrate's overwriting so I believe you're right.

Do you have a work around or solution?

This isn't really a solution, but what I did was stash a copy of the users table before the upgrade, then re-import the avatars after content_migrate was done doing its thing using a custom script.

Even i have the same problem.Gonna try out this. Importing the avatars after content_migrate will do the trick.!!!
Thanks

@melvix, do you have a solution you can share?

I have the same problem.
It's needed to change file fid in content_migrate module if there is file in file_managed table with this fid.
Any solutions?

same problem

Still an issue, still can't upgrade to 7. If someone could detail their solution, it would be most appreciated.

This is still an issue occurring during a D6 to D7 migration that I am doing...
It would certainly be wonderful if someone would be able to contribute their custom script... @melvix, care to share?

Thanks!

OK I fixed this myself. here is my fix. It also fixes the format of the picture uri's so that Drupal 7 can create image-styles for all of the user pictures, which was broken after i migrated from D6 to D7.

Feel free to use as you wish. I'm not respsonsible if this breaks your site and you dont have a backup of your db. Backup your db and test this on a test version of your site first before doing on prod!

1) before migration, save the following in your D6 webroot as get_user_pictures.php. Then go to the page in a browser to get a list of users who have pictures (www.drupal6site.com/get_user_pictures.php).
Code:

error_reporting(E_ALL);

// set HTTP_HOST or drupal will refuse to bootstrap 
$_SERVER['HTTP_HOST'] = 'drupal6site.com';
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';

define('DRUPAL_ROOT', dirname(realpath(__FILE__)));

include_once 'includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// Total Posts of this type, for the group 
$results = db_query("SELECT users.uid AS uid, users.picture AS users_picture, users.name AS users_name FROM users  WHERE users.picture <> '' ");

while ($row = db_fetch_array($results)) {
  print ( '"'. $row['uid'] . '"'. " => " . '"'. $row['users_picture'] . '"' . ",<br/>");
}
    

2) Drop the following into your D7 upgraded site webroot as fix_user_pictures.php:

// Script to migrate user pictures to the appropriate place
// See https://drupal.org/node/1558192#comment-8176679 for more details on why this is necessary
define('DRUPAL_ROOT', getcwd()); 
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// Fill with content from
// https://drupal6site.com/get_user_pictures.php
$user_picture_files = array(
/*
COPY OUTPUT OF get_user_pictures.php, WHICH WAS RUN ON YOUR D6 SITE, TO THIS ARRAY.
EXAMPLE FORMAT OF DATA:
"5" => "sites/default/files/pictures/picture-5-2c086d2efd5c1f41487406efc4cc60d9.jpg",
"7" => "sites/default/files/pictures/picture-7-0b4612c44c01f1cad30302af41747e5c.jpg",
"10" => "sites/default/files/pictures/picture-10.jpg",
*/

);

// Variables to keep track of end results
$no_change_needed = 0;
$no_file_path = 0;
$no_actual_file = 0;
$files_corrected = 0;

// Get a list of all users who have non-empty picture fields
$result = db_query("SELECT n.name, n.uid, n.picture FROM {users} AS n WHERE n.picture!='' ");

// For each of these users...
foreach ($result as $user_record) {
 
    // Get file details of file currently associated as user picture from db
    $filename_result = db_query("SELECT fid, uri FROM {file_managed} WHERE fid=:fid ", array(':fid'=> $user_record->picture));  
    $file_record = $filename_result->fetchAssoc();
        
    print("<br/><br/>User ". $user_record->uid . "<br/>" );
    print( "Picture Location retrieved from drupal 6 site: " . $user_picture_files[$user_record->uid] . "<br/>" . "DB Picture Location: " . $file_record['uri']  . "<br/>");
    
    // Compare filename pulled from file_managed table to filename retrieved from drupal 6 site
    if ($file_record['uri'] === $user_picture_files[$user_record->uid]) {
      // If a match, no need to fix
      print " DB picture location matches picture location retrieved from drupal 6 site. No changes needed.<br/>";
      $no_change_needed++;
    }    
    else {
    
      if(! $user_picture_files[$user_record->uid]) {  
        // We don't know what we want to set the user picture to; 
        // Bail out as we don't know what to do now.
        print("No user picture location retrieved from drupal 6 site! <br/><br/>");
        $no_actual_file++;
        continue; // end loop iteration
      } 
      if ( !file_exists($user_picture_files[$user_record->uid] ) ) {        
        // The file we want to assign as the new user picture doesn't actually exist;
        // Bail out as we don't know what to do now.      
        print("Physical user picture file NOT FOUND! <br/><br/>");  
        $no_file_path++;        
        continue; // end loop iteration
      } 
    
      // If db filename and filename retrieved from drupal 6 site don't match, output info
      print " DB picture DOES NOT MATCH picture file. <br/>";
      
      //========== Attempt to replace db file name with filename retrieved from drupal 6 site==================
      
      // Add file to db
      $path_parts = pathinfo($user_picture_files[$user_record->uid]);
            
      
      // Check if file already exists
      $already_in_files = entity_load('file', FALSE, array('filename' => $path_parts['basename']));      
      
      if (!empty($already_in_files)) {
        // If you get here, you've most likely already run this script and are re-running it.
        // So, bail out.
        print("<span style='background-color:yellow;'><strong>picture location retrieved from drupal 6 site already exists:</strong></span> "); 
        print_r ($already_in_files) ;
        //$file->fid = key(reset($already_in_files));
        $no_change_needed++;
        continue;        
        //print " xxx " . $file->fid;
      }
      else { 
        // Add file to file_managed table
        $file = new stdClass;
        $file->uid = $user_record->uid;
        $file->filename = $path_parts['basename'];
        $file->uri = $user_picture_files[$user_record->uid];
        $file->status = 1;
        $file->filemime = 'image/' . $path_parts['extension'];

        $file = file_save($file);
        file_usage_add($file, 'user', 'user', $user_record->uid);      
      }
      
      // Update user picture
      db_update('users')
        ->fields(array(
        'picture' => $file->fid,
        ))
        ->condition('uid', $user_record->uid)
        ->execute();
      
      $files_corrected++;
      //exit;
    }       
}

// Summary
print("<br/><br/>No need to fix: $no_change_needed <br/>Fixed: $files_corrected<br/>Could not fix due to no physical file: $no_actual_file <br/>Could not fix due to no file path in $ user_picture_files: $no_file_path ");
  
// Finally, fix all user picture locations by setting "sites/default/files/pictures" to "public://pictures".
// This will allow image styles for these pictures to be created.
db_update('file_managed')
  ->expression('uri', "replace(uri, 'sites/default/files/pictures', 'public://pictures')") 
    ->execute();

3) Copy the output from the script in step 1 into the script created in step 2, above the comment "COPY OUTPUT OF... HERE", within the $user_picture_files array.

4) On your d7 site, go to "drupal7site.com/fix_user_pictures.php".

5) Read the output to see what was fixed and what wasn't.

6) View the user profiles on your d7 site to see if the broken pictures were fixed!

Hi!
# 17 I'm afraid this is not a correct approach.
If we correct user picture (avatar), we thus be overwritten migrated cck field.
Or am I wrong?

Who were able to solve this problem?

I also run into this issue when upgrading my site (drupal 6) to drupal 7.26.

some users' pictures are removed from the database. some users' pictures got replaced by irrelevant files, such as PDF etc... the physical files are still there but not showing in the database.

this is definitely a bug and I wonder there is any patch for this or not. I googled this issue but it looks like only a small amount of sites run into this issue. it makes me wonder if it has something to do with the site's configuration, such as orphaned data

anyway i am subscribing.

I definitely have this exact problem. I'm hoping it might be related to #1732944: Corrupted file URI after filefield migration. The example given there is a files directory named "files" (not sites/default/files) so wondering if all of you have legacy "files" directories like the site I'm working on.

As described above, the problem here is that user pictures are corrupted by the file migration in content_migrate. The path and other information in the file_managed entity for an unlimited number of user pictures is replaced by random pictures which existed in filefields or imagefields.

Here is a complete description of this critical problem, which has the potential to affect EVERY upgrade where there are user pictures and filefields or imagefields.

1. The core upgrade of uploads takes place in system_update_7061(). It makes the assumption that no managed_file entities exist, and just creates new entities numbered exactly the same
2. The core upgrade of user pictures from simple paths to file_managed entities, takes place in user_update_7012(). It creates many new file entities. user_update_dependencies(), however, includes code to ensure that user_update_7012() runs *after* system_update_7061():

  // user_update_7012() uses the file API and inserts records into the
  // {file_managed} table, so it therefore must run after system_update_7061(),
  // which inserts files with specific IDs into the table and therefore relies
  // on the table being empty (otherwise it would accidentally overwrite
  // existing records).
  $dependencies['user'][7012] = array(
    'system' => 7061,
  );

3. The bug: file_content_migrate_data_record_alter() (in content_migrate) comes along, which states clearly

      // Copies imagefield data from the old 'files' table into 'files_managed' and sets file_usage
      // Mostly copied from system_update_7061, which does the same for the D6 core 'upload' module

Since the content migration is done after the core upgrade, the user picture entities have already been added as managed_file entities, and the assumption made in system_update_7061 (that no file_managed entities exist, and therefore the copied code can just create/update entities with the same fid value that existed in the D6 files table) is no longer valid, and leads to massive data corruption (improper image values).

I'm not sure how this can be fixed easily, because it would require knowing updating entities reference a particular file.

Here is a workaround:
1. Before core upgrade, comment out the section of user module's user.install starting with user_update_7012().
2. Complete the core upgrade.
3. Run the content_migration
4. Replace user.install with its normal contents
5. Run update.php again to finish the user module's processing of pictures, which adds new fids to file_managed.

Other than the workaround posted above, does anyone have any recommendations for a patch?

I ran into the same issue, see here:
https://www.drupal.org/node/2881907#comment-12105651

Really surprised this hasn't been addressed. Seems like a major bug.