progress, req'd by ultimate_cron, fails @ db-update '7101 - Change schema to SQL 99 compliance'

I'm not sure what could cause this scenario. If the table layout has already been updated then the 7101 update won't work again.

Which version of Progress were you updating from?

I see 2 immediate solutions to this problem:

1.) Completely uninstall/reinstall Progress and the modules that depends on it.
2.) UPDATE system SET schema_version = 7101 WHERE name = 'progress';

Solution 2 is probably the best/easiest/least invasive.

I tried to reproduce the error, but with no "luck". I suspect that Drupal's update mechanism has failed in someway, leaving the Progress module in a partially updated state. I'm closing this ticket accordingly, as I cannot see a solution to fix what occurred to you.

BTW, Background Process did not appear in your post, did you remember to upgrade this as well? It is necessary to upgrade Background Process to at least 7.x-1.13 in order for Ultimate Cron to function properly.

I'll reopen this ticket temporarily, to make sure it's visible to you.

We were having a similar problem and have spent some time debugging the issue.

The root of the problem is the way the &$context is used in the progress_update_7000 and progress_update_7101 hooks. There's a couple of things wrong here, with the first being that unlike working with the batch API in forms, it's not $context that is passed in to hook_update_N(), but $context['sandbox']. The second issue is that $context['sandbox'] is only preserved between subsequent calls to the same hook_update_N(), i.e. when you have a long running hook_update_N() that is doing its work in small chunks to avoid timeouts.

This means that progress_update_7000 sets $context['sandbox']['major_verison_upgrade'] correctly (although is could just be $context['major_version_upgrade']), this information is then thrown away as this particular hook_update_N() has completed its work.

Hence, when progress_update_7101 is called, $context is actually empty, so the major version upgrade text can never work properly.

So, that sets the scene for this error to occur. For example, we were running progress-6.x-1.5 on our site, so our database was at version 6101. We have started an upgrade to Drupal 7, and have upgraded progress to 7.x-1.4. This means our database already contains the changes in update progress_update_7101, but as the major_version_upgrade protection doesn't work, Drupal attempts to run the db_change_field functions, leading to errors.

With this kind of update, where it has to be applied both Drupal 6 and 7 releases, the Drupal 7 version must take account of the fact that a change may have been applied in Drupal 6. The attached patch should be applied to the Drupal 7 release only, as it adjusts the code to check if each field that needs changing exists before attempting to change them.

Hmmm. Crap.

I see the problem. However, your solution is exactly what I'm trying to avoid, see: #1479414: hook_update_71** run on D6 to D7 upgrades.

I'm thinking of perhaps storing the data i put in $context, in a bit less volatile storage.

Thoughts?

Unfortunately, I don't think there's a "neat" solution to this problem. Drupal 6 and 7 are not going to change this behaviour.

If you add an update to both the Drupal 6 and 7 branches of the module, the Drupal 7 branch *must* check if the update was applied by the Drupal 6 branch (e.g. someone upgrading from Drupal 6 to 7). The only way to do this with the current API is by performing db_field_exists(), db_index_exists() and db_table_exists() checks.

The current data you're putting in $context would have to change anyway, as what you can't hardcode what updates may or may not have run the way you currently are - what about someone upgrading from an earlier version of the module to the current version? You'd have to record the fact that each update has run, once it has run.

The current data you're putting in $context would have to change anyway, as what you can't hardcode what updates may or may not have run the way you currently are - what about someone upgrading from an earlier version of the module to the current version? You'd have to record the fact that each update has run, once it has run.

This shouldn't be a problem, since the documentation states that you must upgrade the modules to their latest versions, before doing a major upgrade.

I believe it should still be possible to make a neat workaround somehow.

BTW (and I haven't tested it yet), are you sure $context (or $sandbox) isn't transferred across requests? I think that should be the whole point of it, no?

I'm thinking out loud here:

Instead of using the context and then doing a:

<?php
  if (!empty($context['sandbox']['major_version_upgrade'][7101])) {
    // This udate is already part of latest 6.x
    return;
  }

Perhaps it would be possible do something like this:

<?php
  if (_ultimate_cron_is_counterpart_installed(7101)) {
    // This udate is already part of latest 6.x
    return;
  }

Because I can control the releases of 6.x and 7.x respectively, I can also make sure that each release is aware of previous update-hooks, meaning that 7.x will always know if there's a counterpart for 7101 in the latest 6.x version.

I think this is heading down a route of making a tricky subject much more complicated.

At the point progress_update_7101() runs, you don't know if the user has just done a major upgrade from progress-6.x-1.5 (or earlier) or they're already on the 7.x branch and are doing a minor upgrade from progress-7.x-1.3 (or earlier).

You have to consider when the user is performing the update, e.g.

• Is the user doing a major or minor upgrade and are they doing this at the same time progress_update_7101() was released or several months later?
• Is the user upgrading from the previous minor release, or have they skipped several minor releases?
• At the time progress-6.x-1.5 and progress-7.x-1.4 were released, is it possible a user could have been on progress-6.x-1.4 and believe they were on the latest version, then upgraded straight to progress-7.x-1.4?

The situation for progress is relatively simple, as there are just two updates, but my view of this problem is based on dealing with the same issues in the background_progress and ultimate_cron modules. These other modules have more updates and releases, which means users are more likely to encounter problems running upgrades, as we did when we started our Drupal 7 upgrade a few weeks ago.

$context/$sandbox works like in the batch API - it's preserved during a single operation of a batch (useful for a very long running operation) but not between operations.

I think this is heading down a route of making a tricky subject much more complicated.

I don't think I agree, in fact just the opposite. What I'm trying to do, is make a simple mini-framework where a piece of boiler-plate code determines whether or not a specific update hook should run, instead of every single operation in each update hook needs to check the state of the system before applying.

At the point progress_update_7101() runs, you don't know if the user has just done a major upgrade from progress-6.x-1.5 (or earlier) or they're already on the 7.x branch and are doing a minor upgrade from progress-7.x-1.3 (or earlier).

The 7200 hook will tell us if a major upgrade is being performed.

Is the user doing a major or minor upgrade and are they doing this at the same time progress_update_7101() was released or several months later?

Please examplify?

Is the user upgrading from the previous minor release, or have they skipped several minor releases?

How does this affect anything?

At the time progress-6.x-1.5 and progress-7.x-1.4 were released, is it possible a user could have been on progress-6.x-1.4 and believe they were on the latest version, then upgraded straight to progress-7.x-1.4?

Well .... I guess you're right, but then the user is doing it wrong, right?

$context/$sandbox works like in the batch API - it's preserved during a single operation of a batch (useful for a very long running operation) but not between operations.

I guess you're right. Don't know how I got it to work in the first place then though? I must have missed something.

Your patch has been committed.