Create upgrade path from Embedded Media Field (6.x) to Media module (7.x)

To do this, we will need to handle several steps:

1) Discover any fields with emfield (emvideo/emaudio/emimage) field data.
2) Grab the list of providers supported (such as youtube/bliptv/flickr/etc).
3) Map that to supported providers (Media: YouTube, Media: BlipTV, Media: Flickr, etc).
4) Flag any currently unsupported providers.

We can look at the d6 function emfield_audit() to help with steps 1-4: it currently returns an array of providers that are not supported in d6. Can probably be refactored fairly easily for d7.

5) Back up the old field data tables and convert all emfield fields (emvideo/emaudio/emimage) to media fields, probably by simply creating the new fields.
6) Move old data to the new required format, which includes calling each specific provider uri construction, which will return codes like youtube://v/xcf32yy89, brightcove://u/3245yu342/v/7r89w0dc etc. There is currently no straightforward conversion from the old data (which in those cases might store as 'provider' => 'youtube', 'value' => 'xcf32yy89' or 'provider' => 'brightcove', 'value' => '7r89w0dc'. Instead, we need to call it's hook_media_parse() with the $url currently returned by emfield_parse_embed($field, $embed, $module), where $field is the d6 field info, $embed is the $url, and $module is emvideo/emaudio/emimage.

Confusing, but I believe these steps will handle it for supported providers. We would need to ensure we keep the old data in case there are unsupported providers (waiting on the other modules to upgrade to d7), and put the whole thing in a batch. Also delete values as they're processed, so we can safely rerun the batch job if we later add supported modules.

Please let me know if there's anything else that can be done to simplify and streamline the process. I'm particularly interested in anything that could make the upgrade as painless as possible for the end admins.

Also note that this will not handle eminline data. I'm not sure of the best way to handle that, but I suspect it will be easier in some respects (more difficult in others), as media already supports wysiwyg. We might need to create a new format filter module for that, as the inline code is different than in media (and simpler in some respects, as it simply converts a url to the embedded media).

Also, this won't handle thumbnails. But I don't think that will be an issue in *most* cases, as thumbnail handling has already been tackled afaik w/ media. Unfortunately, it will mean that all custom thumbnail data will be lost (during this phase), as there is no such allowance in media. I suppose for the purposes of this task we would simply make a big note on the project page to that effect, so that admins can make their decision based on their specific needs. However, it might be best to *not* delete old data, but to simply mark it done, so that we can go back later to retrieve any other specific data required that's missed this time around.

Aaron, did you want to officially mentor this task?
I'll be sweeping whatever I can into GCI tomorrow morning (Tuesday, 7/7).

Yes, I can mentor this task.

And by 7/7, I mean 7/9. Added with myself as a mentor until Aaron's a mentor on GCI.
Aaron, there's one more step to becoming a mentor, you get to fill out a mentor profile before you're listed as possible mentors.

moving this issue to emfield. not sure if it ever got picked up at gsoc.

Regarding #4 I think the data migration should be as carefully as the CCK migration is. The old data should not be destroyed after the migration is complete to enable us to go back and fix it/try again until it is "perfect". Personally, for my sites losing the custom thumbnails is not a big thing, as we in most cases use the one provided through the API of Youtube/Vimeo etc.
Subscribing ;)

Subscribing.

subscribe.

I guess this migration should be listed on the modules page. And maybe remove the d7 pledge ;)

(I was researching how to write a emfield provider for D7)

subscribe

subscribe

Anything that can be done to help? (Didn't want to just say "Subscribe.")

Subscribe

Hi Justin
Was unable to find any commits after the two you mention in #19. Could you enlighten us on where we are at the moment and do you have anything in your sandbox we should know about? I'll guess there more then me that are looking on migrating emfield to the media module.

@justin: we need to create a new release for the 7.x-1.x branch before we can change the issue version.

Subscribe

Subscribe

Subscribe

subscribe

Subscribe

subscribe.

Subscribe.

subscribe

+1

subscribe

subscribe. the D6 -> D7 upgrade path has been surprisingly painless thus far... but this would really help cross the finish line!

+1

Subscribe too.

subscribe

+2 (separate customers)

http://drupalcode.org/sandbox/kleinmp/1261456.git/commitdiff/64ec01b

This currently ignores all settings, and has only been tested with media_youtube. But it "works".
I'm considering opening another issue to add URI scheme as a key for hook_media_internet_providers(), since I can't see another way to derive it.

Using the 2.x branch of Media.

I forgot to mention, it needs this bug fix for content_migrate: #1266962: When migrating content, reuse altered instance

awesome work! i've got it set to a file field instead, and it's working well. i agree w/ you re. the hook_media_internet_providers() idea; i'll create an issue & patch for that if you haven't already.

Thanx for the project
with latest build from sandbox i had foll issue and it looks that after a night of tries with db i have mostly overcame it by pushing F5 each time migration failed.

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=259&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'youtube://v/1C9nFS51z6o' for key 'uri': INSERT INTO {file_managed} (uid, filename, uri, filemime, filesize, status, timestamp, type) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7); Array ( [:db_insert_placeholder_0] => 1 [:db_insert_placeholder_1] => 1C9nFS51z6o [:db_insert_placeholder_2] => youtube://v/1C9nFS51z6o [:db_insert_placeholder_3] => video/youtube [:db_insert_placeholder_4] => 0 [:db_insert_placeholder_5] => 1 [:db_insert_placeholder_6] => 1321419702 [:db_insert_placeholder_7] => video ) in drupal_write_record() (line 6884 of /var/www/mysiite.com/includes/common.inc).

@Aurochs

[ Powered by #1115636: Issue Macros and Templates - _default ]

This relates to #1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' and #966210: DB Case Sensitivity: system_update_7061() fails on inserting files with same name but different case

http://drupalcode.org/project/emfield.git/commitdiff/e86f55d?hp=2395e941...

this upgrades from emfield d6 to file d7. requires content migrate (part of cck) and media.

@aaron Thanks for the effort but please please add a banner to the project page containing first lines ie

This project is superseded by the media module.

To migrate your http://drupal.org/project/emfield data your need to download the Drupal 7 version of this module together with content migrate which is part of http://drupal.org/project/cck) and http://drupal.org/project/media

Please publish the D7 version so we can all test this too :-)

@clemens.tolboom see http://drupal.org/node/1365642 and http://aaronwinborn.com/blogs/aaron/emfield-dead-long-live-emfield.

After reading this thread, I still have little clarity about how this upgrade path is supposed to work. I attempted to migrate a large number of youtube embed fields and the fields were created -- that was great! However, they did not get populated with anything -- not so great. I used the below code, hacked from this stackoverflow post.

$sql = "SELECT nid, field_video_embed from content_type_video_page";
$results = db_query($sql);
foreach($results as $record) {
    from_youtube($record->nid, $record->field_video_embed);
}

// include the media youtube handler.inc file to use the embed code parsing

function from_youtube($nid, $code) {
    $path = drupal_get_path('module','media_youtube').'/includes/MediaInternetYouTubeHandler.inc';      
    require_once($path);
    $occurs = strpos($code, 'youtube');
    $node = node_load($nid);
    $video_title = (strlen($node->title) > 255) ? substr($node->title,0,250).'...' : $node->title;
    if($occurs !== false) {
        try {
        $code = MediaInternetYouTubeHandler::parse($code);  
        $youtube = file_uri_to_object($code, $use_existing = TRUE);
        $youtube->display = 1;
        $youtube->filename = $video_title;
        $youtube->uid = 62;
        $youtube = file_save($youtube);
        $node->field_video[LANGUAGE_NONE][0] = (array) $youtube;
        node_save($node);
        print "Updating | $node->nid \n";
        } catch (Exception $e) {
            echo 'Caught exception: ',  $e->getMessage(), "\n";
        }
    }
}

Obviously, you'd need to change the field and table names in the sql query to fit your site. This should work for most youtube videos, assuming your node has a media field configured to handle embedded media from youtube using the media_youtube module.

hello sorry if this is long dead which might be a good thing for me. Ive just done a migration 6>7 and it went smotther than i had hoped. I have all the old youtube vids converted and are now emfields. Just need to find out how to convert them to a Media field now?

Thank (i know that no one will answer :(

Reopening. We failed documenting this. The current README miss documentation. The project page also need to updated. Drupal 8 is coming and there will be more sites that are looking for migrations paths.

Upgrade from Drupal 6 emfield. (updated)

Drupal 6

• Disable all your media related modules.
• Perform upgrade as any other Drupal 6 to 7 upgrade.

Drupal 7

• Download, run updatedb and enable cck migration tool.
• Download and enable Media, media_youtube/vimeo.
• Download emfield and run updatedb.
• When done migrating/updating the fields, turn off the emfield module for the very last time. Media module will from now on be your new best friend.

Marked as duplicate: #2021257: Documendation for upgrating from D6+emfield to D7+media

Related:
#1781966: Embedded Media Thumbnail for DRUPAL 7

Migration can be done by using the following sandbox module:
https://drupal.org/sandbox/kleinmp/1261456

1. Remove old emfield and enable this downloaded emfield.
2. Enable content_migrate and media.
3. Go to: /admin/structure/content_migrate
4. Start migrating (drush -vy content-migrate-fields)

Created [#2288445] Closing this issue.

Upgrade from Drupal 6 emfield to drupal 7 and the media module

Could one of the maintainers update the project description with the upgrade steps from #53 please?

At the moment there is nothing there about the migration process and it took me a while to find this issue with the necessary information.

Im getting the following error when attempting to migrate the video field.

Requesting rollback of field "field_video" due to failure to convert record:array (                                                                                                                                                [error]
  'entity_id' => '5703',
  'revision_id' => '36934',
  'field_video_embed' => 'https://www2.City.urbana.il.us/_Video/_Boards_-_Commissions/Community_Development_Commission/2016/03-22-16.flv',
  'field_video_value' => 'https://www2.City.urbana.il.us/_Video/_Boards_-_Commissions/Community_Development_Commission/2016/03-22-16.flv',
  'field_video_provider' => 'zzz_custom_url',
  'field_video_data' =>
'a:5:{s:35:"emvideo_zzz_custom_url_data_version";i:1;s:3:"url";s:110:"https://www2.City.urbana.il.us/_Video/_Boards_-_Commissions/Community_Development_Commission/2016/03-22-16.flv";s:4:"size";s:9:"416649450";s:4:"mime";s:11:"video/x-flv";s:4:"type";s:3:"flv";}',
  'field_video_status' => '1',
  'field_video_version' => '0',
  'field_video_title' => NULL,
  'field_video_description' => NULL,
  'field_video_duration' => '0',
  'delta' => '0',
  'entity_type' => 'node',
  'language' => 'und',
  'bundle' => 'meeting',
)  Cause: exception 'MediaInternetNoHandlerException' with message 'Unable to handle the provided embed string or URL.' in
/Users/cwahlfeldt/Sites/dev.urbanaillinois.us_D7/sites/all/modules/media/modules/media_internet/media_internet.module:119
Stack trace:
#0 /Users/cwahlfeldt/Sites/dev.urbanaillinois.us_D7/sites/all/modules/emfield/emfield.module(143): media_internet_get_provider('https://www2.Ci...')
#1 /Users/cwahlfeldt/Sites/dev.urbanaillinois.us_D7/includes/module.inc(1157): emfield_content_migrate_data_record_alter(Array, Array, Array, NULL)
#2 /Users/cwahlfeldt/Sites/dev.urbanaillinois.us_D7/sites/all/modules/cck/modules/content_migrate/includes/content_migrate.admin.inc(410): drupal_alter('content_migrate...', Array, Array, Array)
#3 [internal function]: _content_migrate_batch_process_migrate_data('field_video', Object(DrushBatchContext))
#4 /Users/cwahlfeldt/.composer/vendor/drush/drush/commands/core/drupal/batch.inc(149): call_user_func_array('_content_migrat...', Array)
#5 /Users/cwahlfeldt/.composer/vendor/drush/drush/commands/core/drupal/batch.inc(99): _drush_batch_worker()
#6 /Users/cwahlfeldt/.composer/vendor/drush/drush/includes/batch.inc(93): _drush_batch_command('369')
#7 /Users/cwahlfeldt/.composer/vendor/drush/drush/commands/core/core.drush.inc(1145): drush_batch_command('369')
#8 [internal function]: drush_core_batch_process('369', '369')
#9 /Users/cwahlfeldt/.composer/vendor/drush/drush/includes/command.inc(362): call_user_func_array('drush_core_batc...', Array)
#10 /Users/cwahlfeldt/.composer/vendor/drush/drush/includes/command.inc(214): _drush_invoke_hooks(Array, Array)
#11 [internal function]: drush_command('369', '369')
#12 /Users/cwahlfeldt/.composer/vendor/drush/drush/includes/command.inc(182): call_user_func_array('drush_command', Array)
#13 /Users/cwahlfeldt/.composer/vendor/drush/drush/drush.php(92): drush_dispatch(Array)
#14 /Users/cwahlfeldt/.composer/vendor/drush/drush/drush.php(61): _drush_bootstrap_and_dispatch()
#15 /Users/cwahlfeldt/.composer/vendor/drush/drush/drush.php(16): drush_main()
#16 {main}

Let me also mention that this happens on hundreds of nodes that include a custom url video field.