Replace default "update" module behavior with something that makes sense for distributions

I went through the code in the oa_update module. It's actually pretty generic (other than appending oa_ in front of stuff) that I think we could easily re-use it with some minor changes.

I wonder if that would almost be a good separate contrib module for distributions to use in general.

After further investigation it appears that this won't be a simple / straight port. The oa_update module displays the distribution update status and hides all modules including those not included / required by panopoly.

I think what we want to strive for is hiding the status update for:
* panopoly_* modules
* contrib modules required by panopoly_* modules

And display status updates for any module manually added by the site admin. A great example of what may show up here is the 'diff' module if added manually. It's not a required contrib module.

This may require some additional thought as well given that panopoly_* modules may be added independently of the panopoly distro and panopoly may be a base distro.

So during my research I've found a couple of interesting things we may be able to leverage.

Commerce_Kickstarter simply uses hook_update_status_alter() to remove anything in its .make files. We could do something similar in each of our panopoly_* modules by reading the .info / .make files.

I found the Update Status Advanced Settings module that lets the admin configure each project and whether or not a status update is checked, etc. More detail on what it does can be found in the readme.

I think in the long-run it would be nice to add update_advanced to our required modules and then hook into it to pre-configure the modules we don't want to be updated. It doesn't look like we can easily hook_alter it now but it we should be able to supply some patches to make it work better with distros in general.

I don't know if we could get update_advanced ready for 1.2 however so maybe the commerce_kickerstarter approach is a good 1.2 solution and update_advanced for 1.3.

Also forgot... another option is still to create our own panopoly_update but make it so it doesn't hide every module.

So breaking this down into the three main panopoly use cases...

1) Individual panopoly_* modules
Not sure we want to try and do anything here. If someone adds a panopoly module to their non-panopoly site then I think it may be more confusing if we start mucking around with their update screen / status.

2) Panopoly as a base distro
Thinking in terms of OpenAtrium... they run their own oa_update module so we probably don't want to do any modification to the update screen. We should leave that up to the distro.

3) Panopoly as main distro
Here we basically want to hide the panopoly_* modules from being updated individually. Theoretically we shouldn't have an individual module updated without the distro being updated so I think we'd be fine hiding the panopoly_* modules. The contribs that we depend on we want to make it very apparent that an admin updates them at their own risk.

The question then becomes how do we accomplish #3 without impacting #1 or #2?

* Add Update Status Advanced Settings to the drupal-org.make and panopoly.info files. We have to do this at the distribution/install profile level otherwise we're forcing distros like OA to include the module.

* In each panopoly_* module we conditionally check for the existence of update_advanced and programmatically set certain modules to hidden. This should at least allow #1 and #2 to use this logic if they desire, but not force it.

* In panopoly_core we check if panopoly is the install profile and make sure we add it to the updates page (like oa_update). We also hide the panopoly_* modules from appearing since the update the distro and not individual modules.

FUNNY THAT YOU MENTION ADVANCED STATUS MODULE i just re-installed it after thinking i didn't need it due to Drush, so let me report:

(1) the module needs debugging on the site-reports end of things. for some bizarre reason, it will not recognize the commands to either ignore or never update a module.


(2) meanwhile drush will respect the settings and ignore/never ping the blacklisted modules, unless there is a security update. in this case though, because the security update happens to be CTools, even if Panels wasn't explicitly marked as so, it also pinged as needing an update:

Name                  Installed Version  Proposed version  Message
 Chaos tools (ctools)  7.x-1.3+33-dev     7.x-1.4           SECURITY UPDATE available
 Panels (panels)       7.x-3.3+63-dev     7.x-3.4           Update available

(3) Drush has that pm-update --lock function but i found it problematic due to it running update for all modules BEFORE setting the function of locking out blacklisted modules. there should be a way of blacklisting them BEFORE running update. i dont think there's a way to set it up in the drush config file ---but that would then set a dependency on drush for Panopoly, no?

btw, i found this article and it looks like going the hook route makes more sense if we don't want to add yet another contrib module to the bunch. my feeling though is that, if advanced update were properly debugged for accurate update reports via the web UI, it would be useful alongside a hook/alter for the Panopoly modules (not the contribs) themselves. am thinking that, as a platform for building other distributions, this combo may come in handy.

with all this in mind: what happens when there's a security update for CTools or Panels? will the Panopoly team release updates for these? Or are site builders meant to update those? This question refers back to my previous post (#2190579: [META] We have no best practices for updating to new Panopoly releases). i want to capture whatever we discuss here for the documentation draft/patch am contributing.

@liza... thanks for the input on update_advanced!

I think overall the goals with this are to...

a) Hide the panopoly_* modules from update.
b) Show the panopoly distro in update.
c) Default configure update_advanced to *never* check for updates on contrib modules that the panopoly distribution/modules require. This piece here still allows an admin to change the behavior if they so desire.

We'll definitely want to make sure update_advanced is working. By making use of it we can help promote it and no reinvent the wheel. We'll help in plugging any holes in that module the same way we help with other contribs we rely on.

I don't think we'll get everything done for panopoly 1.2 so we may have a stop-gap solution in 1.2 and a much more thorough solution in 1.3.

@caschbre

that sounds like a plan; but i was also wondering if it would be of any value to have a pop-up message for security updates. remember that security updates will anyway trigger a response from advanced_update and once that happens, the site will not stop triggering the alert until it's been taken care of. i think it would be good to add something to the effect: "The Panopoly Team has been alerted about this security update; please wait for our next release".

am looking at triggers/actions to see if there's a way to do this and just pull out a feature; but am not sure it can't be done that way. maybe am missing something, but not seeing it with Rules either. but there's that hook/trigger function that could be coded into the new Panopoly update module.

@liza... we'll do the best we can with it to make it the least annoying as possible but still give the admin control. I have to run panopoly sites as well so I definitely feel your pain. :-)

Attached is a patch that:

* Adds update_manager module to panopoly_core.make file.
* Adds hook_update_projects_alter() to add the panopoly distro to the modules list.
* Adds to the install/update hooks to enable & configure the update_advanced module.

I'm sure a bit more could be done to beautify the update report, but this at least stops modules provided by the panopoly distro from checking for updates. And because it's using update_advanced, an admin does have the capability to enable update checks on a module-by-module basis.

This logic should only execute when the install profile is 'panopoly' so that other distros based on panopoly are not affected.

I definitely think the update manager report can be made more "pretty". The update_advanced module pretty much just lets you mark which modules you want to ignore when checking for updates. They all still get listed, but those set to 'never' get shown with a note. I made the note say that the modules were provided in the Panopoly distro so update at your own risk.

I also didn't make the update_advanced module a dependency. It's in the panopoly_core.make file and enabled if the profile=panopoly and the update module is enabled. Then I configure update_advanced based on any module that is a dependency of the panopoly modules.

Child distros can use this, but not sure we want it to work the same way. If I have a child distro using "some" of the panopoly modules, I want to see when those modules are updated. I also manage the other contrib updates.

The way I'd approach this patch would be as a step 1. It turns off update notifications for modules provided in the Panopoly distro. Then we can make another patch later to make the page look better.

I definitely think the functionality of update_advanced is useful though it doesn't make the update report any fancier. It's written by dww and has moderate usage (~3700). It looks like it provides an interface to some of the update manager features that are not exposed through the UI, like the ability to not receive update notifications on a module or provide notes on a module. There's no new schema to it, etc.

Our other options are to write a new custom module similar to oa_update or add the logic into something like panopoly_core.

Any preference on direction if not update_advanced?

Ok. Take a look at let me know what you think. The update_advanced module provides the functionality you're looking for with child distros or any site that uses some of the panopoly_ modules. The only thing lacking compared to oa_update is the presentation of it unless i'm missing something else oa_update does.

Looking at the patch, it appears we've got to loop through the modules we care about and add them to the list of ignored modules.

That's correct. We basically look for any dependencies. The original patch does this based on the panopoly.info file and it seems to pick up everything that I could see. Any module I manually add to the site is not affected.

Edit: I just realized you said we need to ignore any module in the .make file. I can change the patch to look for any module installed in profiles/panopoly/modules/* as opposed to reading dependencies.

So, we'd have to do this on install for each panopoly_* module if we wanted them to have their .make file modules ignored when used individually (which we do!).

I'm not so sure on this one. If I have a distro that uses some of the panopoly_* modules, I want to get a status update on those. I think it's up to that distro maintainer to manage how they deal with status updates. Or are you thinking of a difference scenario? As it stands, all modules built / required by the panopoly install profile are set to 'never'.

We'll also have to be sure to update this as things are added to the .make file.

Yeah, I'm not sure of an easy way around that. It should be easy enough to add though... it's just adding an element to an existing array.

The "Update" tab on the "Available updates" report.

Yeah, I was thinking of this as well but didn't want to make too many changes. I also wasn't sure what we wanted here. At first I thought about just hiding the tab since we wouldn't want users updating panopoly profile modules that way anyway.

I'm not super crazy update_advanced, I think I'd still prefer a solution that's like a more flexible version of oa_update...

Can you describe what you like about oa_update and / or dislike about update_advanced? Reason I ask is it seems more appropriate to try and make an existing module better instead of replicating it. There's an open issue in the update_advanced queue about making that module better so distros could use it. I'm wondering if we helped enhance that module to it's more distro friendly then we can get more distros using it.

Why do not do like Commerce Kickstart? https://github.com/commerceguys/commerce_kickstart/blob/7.x-2.x/commerce...

/**
 * Implements hook_update_status_alter().
 *
 * Disable reporting of projects that are in the distribution, but only
 * if they have not been updated manually.
 *
 * Projects with insecure / revoked / unsupported releases are only shown
 * after two days, which gives enough time to prepare a new Kickstart release
 * which the users can install and solve the problem.
 */
function commerce_kickstart_update_status_alter(&$projects) {
  $bad_statuses = array(
    UPDATE_NOT_SECURE,
    UPDATE_REVOKED,
    UPDATE_NOT_SUPPORTED,
  );

  $make_filepath = drupal_get_path('module', 'commerce_kickstart') . '/drupal-org.make';
  if (!file_exists($make_filepath)) {
    return;
  }

  $make_info = drupal_parse_info_file($make_filepath);
  foreach ($projects as $project_name => $project_info) {
    // Never unset the drupal project to avoid hitting an error with
    // _update_requirement_check(). See http://drupal.org/node/1875386.
    if ($project_name == 'drupal' || !isset($project_info['releases']) || !isset($project_info['recommended'])) {
      continue;
    }
    // Hide Kickstart projects, they have no update status of their own.
    if (strpos($project_name, 'commerce_kickstart_') !== FALSE) {
      unset($projects[$project_name]);
    }
    // Hide bad releases (insecure, revoked, unsupported) if they are younger
    // than two days (giving Kickstart time to prepare an update).
    elseif (isset($project_info['status']) && in_array($project_info['status'], $bad_statuses)) {
      $two_days_ago = strtotime('2 days ago');
      if ($project_info['releases'][$project_info['recommended']]['date'] < $two_days_ago) {
        unset($projects[$project_name]);
      }
    }
    // Hide projects shipped with Kickstart if they haven't been manually
    // updated.
    elseif (isset($make_info['projects'][$project_name])) {
      $version = $make_info['projects'][$project_name]['version'];
      if (strpos($version, 'dev') !== FALSE || (DRUPAL_CORE_COMPATIBILITY . '-' . $version == $project_info['info']['version'])) {
        unset($projects[$project_name]);
      }
    }
  }
}

@dsnopek... I'm about a day or two out from releasing a new contrib. It's currently a sandbox project. Once I get the readme / project page content updated I'll turn it into a full project, however it should pretty much be fully functional.

I used the concepts from oa_update, and commerce_kickstart, along with some of my own needs as a child distribution to put this together. Ideally this can be used with any distribution and we can all avoid recreating the wheel. :)

The gist of this module is that it scans for .make files and then through some configuration options, alters the various update manager admin pages.

Attached is a new patch that adds the Distribution Update Status Manager (distro_update) to panopoly_core.make/.info. The new contrib is not panopoly specific but should work with most distro's. It's driven off of .make files.

Ok, here are two patch files. One for panopoly_core which includes distro_update in the .make file. And the other for panopoly to add it to the .info file and creates the panopoly.install file with an update hook to enable it.

This is slick. Tested it out, works awesome.

@dsnopek... unfortunately I can't reproduce the issue with drush. With a fresh install of panopoly drush is not an issue but it sounds like with some legacy sites $ drush up is wanting to update some modules. What's odd is if I take a database copy from one of those legacy sites, drush still doesn't cause an issue for me.

So maybe some caching at the environment level?

I've created a ticket in the distro_update queue regarding drush. #2324635: Drush pm-updatecode support

I also have a few fixes in the -dev version of distro_update to handle some issues with commerce_kickstart and openatrium. I'll be rolling beta2 later today with those fixes.

I can create a new patch for panopoly that uses beta2... but wanted to see if we need to wait until we figure out the drush issue? (someone else will have to debug drush since I can't reproduce).

I was thinking the same thing about panopoly_core vs panopoly_admin. I went with panopoly_core because I think this is a module that any distribution should use and making it available helps expose it. :-) The distro_update module is not panopoly specific and the goal is to work with any distro. I see more and more distro's creating their own version of oa_update (or similar) and this should help alleviate that.

But I'd also say we should check with OpenAtrium given they're the biggest panopoly user. Otherwise it's a small enough module that shouldn't have an impact if someone decides not to enable it. It's not like other modules (e.g. admin, admin_menu) that we include in the .make but panopoly never makes use of.

I updated the panopoly_core patch to reference distro_update beta2. The panopoly distro patch should still be correct.

Attached is an updated panopoly_core patch that references beta3. beta3 fixes the two issues noted in #41.

Awesome! I'm super nervous! lol.