On my installation I'm using media 2.x and file_entity.
I've several pdf files (that the module recognize as "application") with many fields associated to this entity.
After upgraded I see that there isn't the "application" file type and all files that I have uploaded are now "Document" type and I've lost all fields and values.

Have I lost everything?
Is there a way to migrate fields from "application" to "document"?

Thanks,
Sergio

Comments

arthurf’s picture

arrubiu’s picture

I don't think is the same.
My files are not "undefined", they are "documents", correctly.

The problems is that, before the upgrade, I've added some fields to "application" file types.
Applications now does not exists and these field are not available on "document" file types.

I'v solved adding fields to "documents" file type but this is not a solution, is a workaround.

ParisLiakos’s picture

You can go in field_config_instance table, find instances with bundle = 'application' and change it to document.
i am not sure i understand though where did you add the fields from on the application type?
there was no ui for that?

arrubiu’s picture

On Structure -> FIle type I manage files and, for every type, I can manage fields.
With the upgrade the file type "application" is deleted

ParisLiakos’s picture

Project: D7 Media » File Entity (fieldable files)
Version: 7.x-2.0-unstable7 » 7.x-2.x-dev
Component: Media field » Code
Priority: Critical » Major

Then you used the dev version, somewhere between unstable6 and unstable7..
Did that fix in your database i proposed worked?

arrubiu’s picture

I was using unstable3 and I've upgraded to unstable7.
I solved simply re-adding fields to the "document" file type, all values were in the database.

dave reid’s picture

Issue tags: +7.x-2.0 alpha blocker

We can't be causing major data loss like this and requiring manual intervention before an alpha release. We should *not* be changing bundles on any existing file entities either. It's a big no-no with Drupal 7.

dave reid’s picture

I think we need to still have the 'old' file types exist rather than go away, but be disabled or have no mime type configuration so that they cannot be used to determine *new* files. This should also ensure that anyone upgrading directly from 1.x get the files assigned the newer, "proper" file types.

ParisLiakos’s picture

Project: File Entity (fieldable files) » D7 Media

Back to media since thats where the update path is

azinck’s picture

Per Dave's comment in #8: there's no particularly good way to manage this. There's no way we can seamlessly migrate users to the new bundles (way too many possible dependencies that we can never know about) so allowing the old types to continue to exist will be less disruptive in the immediate-term. But there are some significant outstanding problems:

  1. How do we adequately warn admins that the bundle names have changed for new content? They'll likely need to change a lot of configuration for their site to continue to function as-desired (Views, Panels, Context, etc.) along with custom code that may have relied on the old bundle names.
  2. Ideally we wouldn't make admins maintain two separate bundles (the "old" and "new" bundles) for what is essentially the same file type indefinitely. Can we ease the migration to the new bundles in a way that's transparent to them? The current update script is not only inadequate/buggy but pretty opaque about what it's doing.

My proposal: allow the old types to continue to exist (rip out the mapping code in media_update_7209) but write some migration code that admins can optionally run. I'm imagining a page that queries file_managed to determine all file types that don't match a current File bundle. It could then give admins the chance to transform that content (and its fields) into one of the existing bundles.

I think that migration could be done with the following steps:

  • Cycle through all files of the "old" bundle type and add their mimetypes to the mimetype list for the "new" bundle
  • Change the 'type' value in {file_managed} for applicable files
  • Pilfer some logic from field_attach_rename_bundle(); namely:
      // Move field instances to the new bundle
      db_update('field_config_instance')->fields(array('bundle' => $bundle_new))->condition('entity_type',  $entity_type)->condition('bundle', $bundle_old)->execute();
    
      // Clear the cache.
      field_cache_clear();
    
      // Let other modules act on renaming the bundle.
      module_invoke_all('field_attach_rename_bundle', $entity_type, $bundle_old, $bundle_new);
    

The advantage is that we can give admins some help text indicating exactly what's going to happen, and give them some agency in choosing which types get mapped where.

ParisLiakos’s picture

Priority: Major » Critical
slashrsm’s picture

Status: Active » Needs work
StatusFileSize
new759 bytes

Attached patch re-creates 'application' file type that is disabled by default. It is a start....

slashrsm’s picture

Status: Needs work » Needs review
slashrsm’s picture

StatusFileSize
new11.82 KB
new11.19 KB

Here we go... Added a page and moved types migration there. Page allows site admin to:
- define mapping between old and new file types
- define whether to migrate fields
- define whether to migrate type's mime types
- define whether to delete old file type at the end

ParisLiakos’s picture

Status: Needs review » Needs work
Issue tags: +Needs manual testing

thanks for working on this!

+++ b/media.installundefined
@@ -908,53 +908,11 @@ function media_update_7208() {
+  // Types migration has been moved to admin/structure/file-types/upgrade',
+  // where can be executed manually.

Maybe move this to docblock so sitebuilders are informed for the upgrade location

+++ b/media.installundefined
@@ -1015,3 +973,23 @@ function _media_update_7204_update_views_display_options(&$display_options, $vie
+function media_update_7212() {
...
+  file_type_save($application);
+  $application = file_type_load('application');
+  file_type_disable($application);

lets make sure file_entity module is loaded before using file_type_* functions

ParisLiakos’s picture

Trying to add the damn tag again

slashrsm’s picture

Status: Needs work » Needs review
StatusFileSize
new990 bytes
new11.88 KB
ParisLiakos’s picture

+++ b/media.installundefined
@@ -1015,3 +974,24 @@ function _media_update_7204_update_views_display_options(&$display_options, $vie
+  $application = (object) array(
+    'api_version' => 1,
+    'type' => 'application',
+    'label' => t('Application'),
+    'description' => t('Multipurpose type - kepped for to support older sites.'),
+    'mimetypes' => array(),
+    'streams' => array(
+      'public',
+    ),
+  );
+
+  file_type_save($application);

Thanks..just thought we might need to wrap this in a conditional in case there is already an application type?

slashrsm’s picture

Makes sense...

slashrsm’s picture

Issue tags: -7.x-2.0 alpha blocker

new release blocker

dave reid’s picture

Issue tags: +7.x-2.0 alpha blocker
aaron’s picture

Status: Needs review » Needs work

It currently reads 'Multipurpose type - kepped for to support older sites.'...should be 'kept to support older sites'.

Additionally, when you enable the application type, I noticed that there are no links to adjust the fields or display mode of the type.

slashrsm’s picture

Status: Needs work » Needs review

Typo fixed.

Additionally, when you enable the application type, I noticed that there are no links to adjust the fields or display mode of the type.

This seem to happen on all disabled file types. After cache clear it works as expected. Do you agree that this belongs to a new issue?

slashrsm’s picture

StatusFileSize
new11.95 KB
new500 bytes

Files....

Status: Needs review » Needs work
Issue tags: -Needs manual testing, -7.x-2.0 alpha blocker

The last submitted patch, media_fix_file_types_upgrade_path_1903224_22.patch, failed testing.

devin carlson’s picture

Status: Needs work » Needs review
Issue tags: +Needs manual testing, +7.x-2.0 alpha blocker
aaron’s picture

Status: Needs review » Reviewed & tested by the community

great!

hyperglide’s picture

As this is 1 of 6 remaining 7.x-2.0 alpha blocker.

I was stirring the queue on the RTBC status or is further testing required. If so is there a way to test this if you don't have data per the issue?

aaron’s picture

aaron’s picture

aaron’s picture

Status: Reviewed & tested by the community » Fixed
slashrsm’s picture

Great! Glad to see this in!

Automatically closed -- issue fixed for 2 weeks with no activity.