Upgrade path from 1.x to 2.x

I'm in the process of checking this out. Not sure I will write any code, but I'm initially curious as to what happens if I already have existing products/product types that have a Commerce File field configured. I will post back with updates.

EDIT: Well, off the bat, it does look like there needs to be an immediate update step:

PHP Fatal error: Call to undefined function commerce_file_field_property_info_callback() in sites/all/modules/entity/modules/field.info.inc on line 30

Probably has to do with old hooks getting called because I just unpacked the new module on top of the old. I'll fix that and try again :)

EDIT 2: So Commerce File REQUIRES Commerce License? Which requires Commerce Entity Bundle? I gotta say, I'm not thrilled with this direction. Truthfully, I'd like to see Commerce File decoupled from Commerce License - the way our site works right now, we don't even care about licenses, really. I like having the ability to add a new file to a product, and if a user has the product, the new file shows up automatically - no need to issue new licenses or sync them up.

On our current site, I actually wrote a custom "My Library" section that bases the list of files solely on the SKUs in a user's purchase history. (Also by node ID to get the initial list of products.) Seems that's a better way to go; having an entire database dedicated to licenses - which gets HUGE when you have hundreds of thousands of customers like we do - seems awfully redundant, and TBH was one of the things I really didn't like about any of the previous "product file" solutions in UC or Commerce.

https://drupal.org/node/1395488#comment-7658191 has some of my reasoning around designing the 7.x-2.x branch.

1) Commerce File 1.x was about 5 thousand lines of code. Commerce License 1.x and Commerce File 2.x are 1 thousand each. So we did manage to remove half of the codebase in the process.
2) The license entity is just what you're describing. It is a single row that has an uid, a product_id, a status and some timestamps. No Field API fields. Can't get any simpler than that. You need to have a record of purchase, and the license is exactly that.
In 2.x the files are always taken from the product, they are not stored on the license, so there is no need for any kind of a "Sync" operation.

Also, the other dependency is Entity Bundle Plugin, not Commerce Entity Bundle. It's a really small module that allows us to have a separate entity class for each entity bundle. See my explanation here.

Ah yes, sorry for getting the name of the module wrong. That does actually sound easier (and less code is always good).

I guess the first step is removing references to the LicenseEntity in the products, since that appears to be the (now-unmet) code dependency that is breaking here (and as you described, Licenses are now just rows in a database and not properties of a field).

Hi there,

I was wondering about an upgrade path as well. When i simply remove the old module and add the new one i loose all my licenses!

And i'm not able to deactivate version 1 before the upgrade as the Field Types depend on it.

I'd really like to use the download count feature.

We are using 1.0 and love the new features for 2.0. We are interested in sponsoring development on this upgrade path. @bonjanz would you be able to provide direction?

Thanks,

Bobby Dean
TheBlessedLife.com

Here's direction.
4 steps. The first and the last implemented, the ones in the middle described.
Now a coder can write the rest, test with a good set of sample data (such as yours rtdean93, or yours torgospizza) and call it a day.

So, the update path works like this:
You have a 7.x-1.x install. You replace the files with 7.x-2.x, and run update.php. No disabling or uninstalling.

/**
 * Update from Commerce File 1.x
 */
function commerce_file_update_7200(&$sandbox) {
  // Initialize the sandbox.
  if (!isset($sandbox['progress'])) {
    $sandbox['progress'] = 0;
    $sandbox['max'] = 3;
    $sandbox['product_fields'] = array();
  }

  // Cast the progress to an int. This ensures that the right step is called
  // even when it has substeps which move the progress bar along
  // (so step 2 might be called multiple times to process various groups of 
  // licenses until done, with progress being 1.1, 1.2, etc).
  $progress = (int) $sandbox['progress'];

  if ($progress == 0) {
    // Enable commerce_license
    module_enable(array('commerce_license'), TRUE);

    // Find the product types that contained the legacy commerce_file field.
    $fields = field_read_fields(array('type' => 'commerce_file'), array('include_inactive' => TRUE, 'include_deleted' => TRUE));
    foreach ($fields as $field) {
      $instances = field_read_instances(array('field_name' => $field['field_name']), array('include_inactive' => TRUE, 'include_deleted' => TRUE));
      $bundles = array();
      foreach ($instances as $instance) {
        if ($instance['entity_type'] == 'commerce_product') {
          // Save the field names that are tied to products, so that we don't
          // need to filter it again in next steps.
          $sandbox['product_fields'][$instance['field_name']] = $instance['field_name'];
          $bundles[$instance['bundle']] = $instance['bundle'];
        }
      }
    }

    // Enable commerce_license and commerce_file 2.x for the previously used
    // product types.
    variable_set('commerce_license_product_types', $bundles);
    variable_set('commerce_file_product_types', $bundles);
    // @todo Enable commerce_license for all product line item types (some
    // sites have more than one).
    variable_set('commerce_license_line_item_types', array('product' => 'product'));

    // Clear the cache. This will create the new fields.
    drupal_flush_all_caches();

    $sandbox['progress'] = 1;
  }
  elseif ($sandbox['progress'] == 1) {
    $fields = field_read_fields(array('type' => 'commerce_file'), array('include_inactive' => TRUE, 'include_deleted' => TRUE));
    foreach ($fields as $field) {
      if (!in_array($field['field_name'], $sandbox['product_fields'])) {
        // This field is not attached to products, ignore it.
        continue;  
      }
      if ($field['storage']['type'] !== 'field_sql_storage' || !$field['storage']['active']) {
        // This field is not in SQL storage, ignore it.
        continue;
      }

      $table_name = _field_sql_storage_tablename($field);
      // Select the entity_id, {$field_name}_fid and {$field_name}_data columns.
      // Load each $product by the entity_id, and update its commerce_license_type
      // field (to "file"), commerce_license_duration field (by looking at the duration
      // in the data column from the query), commerce_file field (fids from the query).
      // Save the products.
    }

    $sandbox['progress'] = 2;
  }
  elseif ($progress == 2) {
    // Do a db_query on commerce_file_license. Based on each row, create a 
    // new commerce_license entity and save it.
    // Maintain a mapping between the old and the new license ids.
    // Then query the field data table for the commerce_file_license_line_items
    // field. The entity_id is the old license_id while 
    // commerce_file_license_line_items_line_item_id is the line item id.
    // Load all of those line items, and set the commerce_license reference field
    // (by translating the old license id into the new one).
    $sandbox['progress'] = 3;
  }
  elseif ($progress == 3) {
    // Delete the old fields, perform 1.x uninstallation.
    commerce_delete_instances('commerce_file_license');
    commerce_delete_fields('commerce_file');
    variable_del('commerce_file_license_help_text');

    $sandbox['progress'] = 4;
  }

  $sandbox['#finished'] = $sandbox['progress'] / $sandbox['max'];
}

This is an awesome start, thanks @bojanz!

EDIT: Realized step 1 takes care of my problem. (That's what I get for testing with code snippets first.)

I'm working on fleshing it out further to handle the "described" steps. Hopefully I can submit a patch that will allow people to upgrade.

@torgosPizza
How is this progressing? Any timeframes?

Would like to have a beta1 with the upgrade path, but if it's far away, I'll tag an alpha2 soon.

I'm glad you're working on this @torgosPizza (is that a MST3K reference?)!

Thank you all for your input. @torgosPizza - any update?

Yes, still working on this. Encountering some issues in Drupal (missing a bundle property at the moment) but it's getting there! I hope to post something soon.

Thank you @torgosPizza!

Still rooting for you @torgosPizza.... Let us know if you need any assistance.

Thank you

Bobby Dean

I'll be happy to help test this out too. I have a site in development with over 1000 digital products currently in v1.x. I'm inclined to wait for the upgrade path to v2.x before going live with our new site. Happy to help test. Sounds like @torgosPizza is getting close. Cheers everyone !! ~ peter

@torgosPizza
Can you post your current progress, please?
This is our only RC blocker, so if someone else wants to take over the effort, it would be great not to redo what you already did.

Hey bojanz,

Unfortunately I had to abandon it for now - sorry I didn't get back to you sooner, been on vacation. I wish I could post my current progress, but it appears I never checked any of it into subversion :/ That means all the time I spent on it is gone. I could try to redo it, since I've done it once and I'm back now - however there were still problems with the code as it had been written. For example all of my files were being added to one product; I'm sure this was a simple error that I could fix again given some time.

I'm about to start in a new direction with our migration, and if I can get back to this upgrade path I'll be happy to contribute some more to it.

Okay, no problem.
Who can provide me a db dump (+ files) of a commerce_file 1.x install that I can test on?
(You can delete the users and sanitize any non-essential info)
I'd also accept an install with some dummy data, as long as I don't have to build it ;)

Are there any more updates and wisdom to share on this upgrade path from commerce file 1.x to 2.x ?

I'm planning to experiment a little today and see if I can fix my own specific needs. Since I'm still in development, using 1.x, I'm willing to sacrifice all my licenses and start over. I'm not willing to sacrifice all of my commerce file field associations, however, I jwant to keep my commerce file fields pointing to the 1500 + Downloads I've already associated with each product node and display.

I can see how if you were already in production mode, with real customer file licensees you may want a programatic upgrade path. As I mentioned above my goal is actually...

1) Ensure Products retain commerce file associations after upgrading from 1.x to 2.x
2) Take advantage as the license as entity architecture in 2.x to import old orders and customers using commerce feeds and map my soon to be old download store's customers to drupal commerce development instance.

I'll post my experience and success.

cheers !

~ peter

Hmmm... I understand the issues challenging this. Not a simple issue. @bojanz and @deggertsen how far did you all get. I'm happy to help and am extremely motivated to see the fruits. Enjoy a nice weekend.

The world needs more patience. Thanks for developing this great module.

Very Best,

Peter

Howdy Drupal Friends,

This upgrade path is still a major issue for us. I've been pulling some hair out and Its the only major roadblock standing in the way of the launch of our new website. Any updates from anyone on this upgrade path or a solution? I'm stuck and could use someone's help pretty please. Willing to beg, barter or hire for a solution.

Kindest regards,

Peter

@deggertsen @bojanz @recrit

Happy New Year!
Are any of you available for some advice on the missing upgrade path from 1.x to 2.x of commerce file. I seem to be pretty stuck and would love help from the community. Who can help us ? We'd be eternally grateful!

I even tried to just kill the fields within our product content type and start over but tables and data in the DB are still tripping us up, even though I tried to be careful and remove tables related to the 1.x prior to installation.

So here is the Issue.

I can't seem to uninstall the module in order to even bypass an upgrade and there seems to be no upgrade path. Sort of a catch-22. I've been following advice in the forums and manually dropping tables, but I'm still stuck and could use some additional advice.

I've been reading and trying the advice here...

https://drupal.org/node/1309140

What am I not getting? Thanks in Advance.

Best,

Peter

Unfortunately, I won't have time to make the upgrade path.
Therefore, I am no longer counting this issue as a release candidate blocker.
Patches welcome, as always.

I can't believe this module doesn't have an upgrade path.

At the very least, can you please put this in huge big bold letters on the project page? I just did an upgrade from 1.x to 2.x (since 2.x is the supported version) to fix an issue. However, this created other major issues because there is no upgrade path in place. I can't believe this is the supported version of this module and there is no upgrade warning anywhere on the page.

By the looks of it, I won't be writing one. An upgrade path will have to come from the community.

@davidwbarratt
Upgrade paths between major branches are often missing in Drupal, especially in Commerce.
Lack of resources in most cases.

bojanz,

Then you should write fewer features to devout more resources to the upgrade path. I'm not saying you should support older versions, I'm only saying you should provide an upgrade path from one version to the next. Which is easier to you? Providing an upgrade path or supporting older versions? I suppose your solution is to simply do neither?

This really weakens my confidence in not only this project, but Commerce as a whole. :(

Thanks!

If there wasn't going to be an upgrade path, this project should have been deprecated and a new project should have been created (since it's really two completely different projects now).

Thanks!

@davidwbarratt
Welcome to open source. There are no promises, no rules, no guarantees.
Especially when the code changes the owner like it did between 1.x (recrit) and 2.x (me).

bojanz,

I'm only saying that there is a responsibility as a maintainer, and I think you are failing in that responsibility. I'm not saying that an upgrade path even needs to be provided, I just think that that needs to be clearly communicated.

Thanks!

@davidwbarratt
I've added a note to the project page, and I'll gladly welcome you as the 1.x maintainer if you want to fix bugs in that branch and/or write an upgrade path.

bojanz,

Thank you, I think the messaging is perfect and is all that is required at this point. :)

You're a Gentleman and a Scholar.

Basically @bojanz had it outlined in #7. I would suggest starting there - but if you don't have time, I might be able to hammer this out fairly quickly. I've been working with Commerce File 2.x for months now, so I have a pretty good grasp of what the comment entails. 2 years ago I hadn't any of that experience so I ran into a wall.

@deggertson, let me know if you want to take it on or if you'd like me to take another stab at it. (Not sure if I'll have time for a while, though, so if you can understand the issue feel free to take a crack.)

Right, luckily for us @bojanz was nice enough to pretty much take care of half of it for us, and explained what else needs to be done (basically the parts involving switching from the old fields to the new entity-based fields). Since it does involve a change with how the products are presented it's a little bit harder than most issues I've run into.

How urgently do you need this? I'm in the middle of a few things at RiffTrax but could possibly make some time soon in my off hours. If you'd be willing to sponsor some development and you needed this asap, I would of course be willing to take a serious look.

Crap! No, I was never able to complete it. The full-time job ended up taking every available cycle. I'm in a bit of a lull but I can't guarantee Il'l have time to get to it soon.

Sorry about that! I really wanted to get this done for you.