I just discovered a serious omission in the implementation of both update.module and update_status.module. All along, we had planned to have a huge scary warning displayed to the site admin when their currently-installed release has been unpublished at d.o. This is built into the .xml release history files, and had been our intention from the beginning of the XML-based redesign (what's now the 5.x-2.* series in contrib, and in core). However, I just checked the code, and this critical functionality is totally missing. :( I think myself and Earl simply forgot about it when getting everything else done, and no one ever caught it on its way into core.

However, there are related problems with the current logic in update status, both involving multiple recommended versions for a given version of core and edge cases regarding security releases.

Therefore, in spite of all the other things we need to get done soon, I think fixing all of this at once is the best approach, since we're already going to have to be touching the exact same area of the code. I have a proposal for a complete solution over at g.d.o, so let's discuss there, first, then come back here to address this (IMHO) critical bug in update.module.

CommentFileSizeAuthor
#80 update_status_5x-2_197186_5.patch27.99 KBdww
#79 update_status_5x-2_197186_4.patch27.98 KBdww
#78 update_status_5x-2_197186_3.patch28.04 KBdww
#73 update_197186_29.patch28.81 KBdww
#72 update_197186_28.patch28.75 KBdww
#70 update_197186_27.patch28.58 KBdww
#68 update_197186_26.patch28.58 KBdww
#66 update_197186_25.patch28.11 KBdww
#65 update_197186_24.patch27.24 KBdww
#63 update_197186_23.patch27.33 KBwebernet
#62 update_197186_22.patch27.24 KBdww
#61 update_197186_21.patch26.25 KBdww
#60 update_197186_20.patch26.22 KBdww
#59 update_197186_19.patch24.83 KBdww
#58 update_197186_18.patch24.8 KBdww
#57 update_197186_17.patch24.27 KBdww
#54 update_197186_16.patch24.24 KBdww
#51 update_197186_15.patch24.47 KBdww
#50 update_197186_14.patch24.48 KBdww
#49 update_197186_13.patch23.6 KBdww
#48 update_197186_12.patch23.51 KBdww
#47 update_197186_11.patch22.49 KBdww
#46 update_197186_10.patch22.49 KBdww
#43 notsupported.png14.66 KBcatch
#38 update_197186_9.patch19.75 KBdww
#37 update_197186_8.patch18.88 KBdww
#20 update_197186.patch_7.txt17.98 KBdww
#17 update_197186.patch_6.txt17.83 KBdww
#17 update_status_5x-2_197186.patch_2.txt17.67 KBdww
#15 update_status_5x-2_197186.patch.txt17.53 KBdww
#14 update_197186.patch_5.txt17.7 KBdww
#10 update_197186_strings.patch_4.txt16.87 KBdww
#8 update_197186_strings.patch_3.txt16.78 KBdww
#5 update_197186_strings.patch_2.txt13.35 KBdww
#4 update_197186_strings.patch_1.txt10.84 KBdww
#2 update_197186_strings.patch.txt5.15 KBdww
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww’s picture

Priority: Critical » Normal

This isn't critical. The usual case we care about is when a security problem is found, and you need to upgrade. So long as a new release comes out marked as a "security update", update.module already warns you correctly. And, in general, the security team hasn't been unpublishing the older, vulnerable releases, anyway.

Furthermore (although it's got its own problems), the "recommended branch" stuff on the project node triggers a warning if you're on 5.x-1.* and the maintainer dropped support and recommends everyone upgrades to 5.x-2.*. So, there's at least _some_ way for project maintainers to alert their users they're dropping a given branch, and update.module warns them about that, too. No, it's not ideal, since it'd be nice to have multiple supported branches, but at least there's a mechanism to warn if you're running code that's not supported anymore.

So, the only case that update.module doesn't warn you properly is basically if the module is abandoned, and the security team decides it's insecure, and unpublishes the project and all its releases. In that case, you get no warning, since there's no newer security update release to trigger it, nor a newer recommended version. However, that's pretty darn rare, and I'm not sure it's worth calling this bug "critical" for a rare case.

That said, I'd still be very happy to see my proposal implemented and committed before 6.0... If only the people who were most vocal about the problems with the current design actually helped fix it, instead of just complaining. Oh well.

dww’s picture

This is a work-in progress, but here's a draft of the additional strings I'll need to add to update.module to fix this bug and the problems discussed at http://groups.drupal.org/node/7440. Folks should review the wording, see if it's all clear, etc. Meanwhile, I'll get the rest of the patch written that will notice these errors and report them.

Note: the patch looks bigger than it really is, since I just moved some case statements around so the messages would be listed in order of severity. The only new strings to review are for UPDATE_NOT_AVAILABLE and UPDATE_NOT_SUPPORTED.

Open questions:

A) Should I renumber the existing UPDATE_* PHP constants to match the same severity order? No one should be using the ints, only the constants, so I don't see this as an API change, but technically it is, so I didn't do it yet.

B) In theme_update_report(), should all of the status values inside the "version-status" div include a span, or just the ones with the serious errors?

Thanks,
-Derek

dww’s picture

Title: Update.module doesn't warn sites when their current release has been unpublished. » String freeze: Update.module doesn't warn sites when their current release has been unpublished.
Status: Active » Needs review
dww’s picture

Getting closer: I added more functionality and realized I forgot a few more strings -- the 'value' on the status report for the new conditions, for both core and contrib.

Another open question:

C) Will we *ever* unpublish a core release node? Will we ever unpublish a core release node without also marking it insecure? We could leave that case statement out of the code if we think we'll never do that. Seems safer to leave it in, but I'd be willing to take it out if people object to the "bloat".

dww’s picture

Still untested, but a start on the changes to update.compare.inc for the new logic...

dww’s picture

Another open question:
D) What should the XML look like for the new major/branch-specific attributes?
Please see http://groups.drupal.org/node/7440#comment-23536

catch’s picture

Status: Needs review » Needs work

OK everything in there looks great except:

At least one of your modules or themes is no longer available. Upgrading is highly advised!

Looks like the actual module is out of stock (or out of print) - so how can I upgrade if it's no longer available??

I'd say

At least one of your modules or themes has been withdrawn. Upgrading is highly advised!

Or "The installed version of at least ..", maybe to make it really clear it's about what's on the server and not the entire module.

'cos it's essentially a product withdrawal right?

dww’s picture

Status: Needs work » Needs review
FileSize
16.78 KB

Still untested, but I think this might be everything we need in the compare logic. All that remains is to potentially modify the parsing code based on what's decided with open question D (http://groups.drupal.org/node/7440#comment-23536), then we can begin real testing. Questions A-C are just minor (mostly cosmetic) changes to the patch.

Another open question:

E) Do we want to keep the existing security update warning behavior? See http://drupal.org/node/196652 for background. If a release node is *not* marked as "Insecure", but there's a security update available between the currently installed version and the recommended version for a given project, should we always flag it as a security update? That's how it works now, and it means that when an SA goes out, the security team and/or project maintainer doesn't have to retroactively mark the effected release nodes as "Insecure". However, this opens up the possibility for edge cases where a given release isn't really insecure, but the site admins are warned as if it was. If we alter this behavior, and make the new client rely on the fact that older releases will be specifically marked "Insecure", it's technically more correct in some cases, but more work for the security team. :( Certainly, we'd want tools to select a set of releases and mark them all insecure in bulk. However, even with that, it's more work, so I'm not 100% sure it's even worth solving the edge case. Especially since we'll now handle multiple supported branches, the edge case is even more rare and (IMHO) unimportant.

@catch: No, it's not that the whole project has been unpublished (eek, I guess we need to handle that case, too). :( The current code is to handle the case where a specific release node was unpublished (e.g. an old branch that's totally abandoned).

Anyway, even though this obviously needs work, I'm setting back to needs review, since I want other eyes on the text (and hopefully the logic in the patch), and since your criticism was based on a misunderstanding of what this is trying to accomplish (which is itself proof that the strings in the patch aren't yet clear enough). ;)

catch’s picture

dww:

Sorry, I understood what the situation is - but was trying to give an example of the misunderstanding the text would cause, might have done this a bit overdramatically...

Anyway I think this text would minimise confusion:

The installed version of at least one of your modules or themes has been withdrawn. Upgrading is highly advised!

Then it's clear what exactly has been withdrawn rather than the whole module or theme.

No, it's not that the whole project has been unpublished (eek, I guess we need to handle that case, too).

For me that's the same as unsupported. However, there's no upgrade path in that case so the proposed text wouldn't deal with it. There's also the case where a release could be marked unsupported in favour of a new project - plenty examples of that in contrib (dashboard -> panels for example).

That gives us three types of unsupported:
1. 1.x > 2.x
2. 1.x to a different project
3. Entirely abandoned

With 1 and 2, users should normally be reading release notes and the rest anyway.

The installed version of at least one of your modules or themes is no longer supported. Upgrading is highly advised! Please see the project homepage for more details.

I can't think of examples where actual project nodes get deleted unless it was spam or something - which wouldn't have a real module attached right. Probably they shouldn't be unpublished but left with a note to an issue or something to explain. Proposed text would probably deal with that edge case well enough if it's acceptable for the other two.

dww’s picture

@catch: Gotcha, sorry. ;) New patch with most of your proposed changes.

The only thing I'm torn on is this "has been withdrawn" language. If we go that route, we should be consistent, both with the text you mention, but all the other places in the patch I'm talkinga but "not available", including the PHP constant and variable names. So, let's agree on exactly what we want to call it:

- "unpublished": the original proposal, but that's too much drupal-jargon.
- "not available": dww's current proposal, catch doesn't like it.
- "withdrawn": catch's proposal, dww doesn't like it. ;)
- "abandoned": too much like "unsupported".
- "recalled": kinda like "withdrawn"...
- ???

minor edit: this is hereby known as open question F.

dww’s picture

BTW: we could certainly change project-release-create-history.php to generate something like the following when an entire project has been unpublished:

<project>
<title>Badness 10000</title>
<short_name>badness10000</short_name>
<link>http://drupal.org/project/badness10000</link>
<status>unpublished</status>
</project>

Some could rightfully argue that anonymous users are never supposed to be able to even read the titles of unpublished nodes, so we shouldn't be doing this. By that logic, we shouldn't include as much information about the unpublished releases, either, but I don't think we care...

Anyway, the above would let us distinguish when the whole project was unpublished for some reason. I'm not convinced we need this.

catch’s picture

recalled - that's what I should've posted in the first place. +1

dww’s picture

I got some feedback in IRC about my open questions. According to chx:
A) Yes, renumber.
B) chx isn't a themer, so he doesn't care, but he suspects more spans with reasonable classes can't hurt.
C) Yes, it's possible we might unpublish a core release node, so we might as well leave that check in.

After further research over at http://groups.drupal.org/node/7440#comment-23536, it seems like we have an answer for D). We'll use something like this to represent a project where there are 2 supported branches, and the newer is recommended, but users of the older don't need to upgrade:

<recommended_major>2</recommended_major>
<supported_majors>2,1</supported_majors>
<default_major>1</default_major>

So, 2 and 1 are both supported but 2 is the recommended version for new downloads. These are the only things specified by the project maintainer. default_major is just for backwards compatibility with older update(_status) clients, and is automatically generated by taking the MIN() of the supported_majors...

E) chx thinks i'm too concerned about edge cases, and that we shouldn't bother. So, we could forget about the new "Insecure" release node term, and stick with the existing "Security release" logic... I'm still torn.

F) What to call it when a release node is unpublished. -- Still no clear winner. See #10 above for the options.

dww’s picture

FileSize
17.7 KB

New patch that addresses A-D. Still waiting for input on E and F.

Note for the XML purists out there: we can change the format of the XML in D7 if we wish. We'll be able to assume PHP5, which means vastly better XML parsing, which means we can do more "correct" XML for this. We can segregate the XML generation for 7.x and above from 6.x and below, so we can change the format without any concern about breaking older clients. So, please don't freak out about the "lame" solution to problem D. Thanks. ;)

dww’s picture

Here's an initial backport of my latest patch to update_status 5.x-2.x-dev (DRUPAL-5--2 branch in CVS). Folks should be able to try this out on a D5 test site (where there's a *much* richer set of contribs to play with). I still haven't messed with project-release-generate-history.php, so project.d.o doesn't have much interesting XML history files for 5.x to play with yet, but that's coming next.

dww’s picture

New patch for core with a few improvements:

- If <supported_majors> is defined, but our current major isn't in it, we know the current release is unsupported. Given that, we might not even need an "Unsupported" term at all. But, it doesn't hurt to let either indicate an unsupported release, in case there are reasons why we want to mark specific releases as unsupported, even if the branch its on is still supported.

- Cleaned up some of the logic/flow of the comparison code so that security updates always trump any other status, and to avoid an empty if clause i had that used to help clarity.

dww’s picture

Whoops, the patch. Oh, and a re-roll of the same changes for update_status 5.x-2.*

dww’s picture

See http://drupal.org/node/203313 about the project UI changes for indicating multiple supported branches. Input there is most welcome.

Gábor Hojtsy’s picture

- "Since it's new XML, we know the current release is unsupported." what's new XML? Probably the format. This comment is quite unclear.
- How it is not possible that core might be usupported? Isn't that case to cover when Drupal 8.x is released and 6.x goes unsupported? On the other hand, you cover UPDATE_NOT_AVAILABLE for core, which is IMHO what's less likely (that a previous Drupal release gets unpublished). Am I misled by something?
- In the last switch, I don't think having each concatenation on its own line makes it more clear, in fact IMHO it looks quite ugly.

Otherwise the changes look good (on review, not tested).

dww’s picture

FileSize
17.98 KB

@Gabor: Thanks for the review. New patch tries to clarify the comment.

re: UPDATE_NOT_AVAILABLE vs. NOT_SUPPORTED for core: see comment #4 above and open question C. So far, the only feedback I got was from chx, saying that we might unpublish core release nodes, so that's in there. In terms of NOT_SUPPORTED, I was originally planning to just have that case triggered via the supported branches stuff, which is specific to contrib (since core only ever has a single branch, as explained in the comment). However, it we wanted to go the route where we marked core release nodes as "Unsupported" via the Release type vocabulary, and never unpublished the nodes, that's fine. I just need input on how we want to handle the core workflow for this stuff, which is why I asked. ;)

re: the relative beauty vs. ugliness of the final switch statement: I personally find this easier to read, it ensures nothing is longer than 80 chars wide, and that coding format was included in the original patch that went into core (after much scrutiny). I'm just adding a few more cases to the switch, and making them all have spans, using the existing format for consistency. If it really bothers you, I can change it, but I disagree it's "quite ugly". ;) (I also think this format makes it easier for themers to copy the function and modify the parts they care about (the spans) if they ever want/need to do that).

dww’s picture

Note: I'm also still waiting for input on:

  • Question E: Should we retain the existing security update logic with it's minor edge-case bug, or change that?
  • Question F: What should we call it when a release node is unpublished?
Dries’s picture

Question F seems to suggest that having both UPDATE_NOT_AVAILABLE and UPDATE_NOT_SUPPORTED is a bit blurry. If I had to pick one, I'd say UPDATE_NOT_SUPPORTED because well, your version is no longer supported. However, ideally, both would be merged -- at least in my mind.

I'd ignore E for now -- it might create more confusion than it solves. We can revisit this problem later on. For now, I'd postpone this edge case.

dww’s picture

@Dries:
re: E: great, duly ignored. ;)

re: F: In the past, the only way we had to indicate a release was bad was to unpublish it. Now, we've got other options. That's why it might seem murky. Perhaps we should just remove the whole "unpublish the release node" option from our workflow, and rip out "NOT_AVAILABLE" entirely. However, I think of "NOT_SUPPORTED" as "we're no longer fixing bugs here -- use at your own risk", while "NOT_AVAILABLE" (or whatever we end up calling it, if we keep it) as "this is known to be really bad, don't use at all". Arguably, the only time we'd unpublish something is if it was known to be insecure, in which case, we could just mark the nodes "Insecure" or put out a "Security update" release, etc. Not matter what, "NOT_AVAILABLE" is going to be rare. The example I used elsewhere was for an abandoned project where we unpublish the releases. There'd be no newer security update release, so there'd be no other way to indicate that the releases were dangerous unless we either a) add the "Insecure" term to the release type vocab and flag the existing releases with it, or b) continue to support NOT_AVAILABLE. It's really a question of if we want to continue to unpublish releases as part of our workflow or not... Also, there's the possibility for critical bugs that aren't security per se (e.g. a data corruption bug), which might be why we want to unpublish.

New question:
G) Should we flag it when the whole project node is unpublished? See comment #11 for more. There are definitely cases where we unpublish project nodes, and we probably want to indicate to update.module users when it happens. Most of the same arguments about NOT_AVAILABLE still apply here -- e.g. the project is abandoned, but known to be problematic. There's no new security update to point people to, but we just want to say "this project is evil, no one should use it". Unpublishing the project is usually how we handle that, but update.module doesn't know about it...

dww’s picture

See http://drupal.org/node/204140 about updating project-release-create-history.php to use the new data from http://drupal.org/node/203313 to add the new attributes we care about to the XML release history files. A volunteer for that would be most welcome -- it's easy.

Gábor Hojtsy’s picture

I think it is much more clear to not unpublish stuff but to add markers to them, like 'Insecure', 'Unsupported', etc. We can make our live listings not include these, or include them with warnings, and we can make these display themselfs with huge warnings. Unpublishing stuff hides the tarball without information on what happened. Tarballs might be out, with the CVS tags, they can be regenerated, etc. If we actually provide information on why was it discontinued ('Insecure', 'Unsuppotted', etc), it gives us much more information. We did not have these tools before as you note, but now that we have them, why use the unpublish workflow?

dww’s picture

Just got back from a week off-line -- catching up with things...

We've long used the ability to unpublish projects to indicate that they're dead, unsupported, etc. If we stopped doing that entirely, and moved to a purely flag-based approach, this wouldn't be as much of a concern. But, we *do* unpublish projects, and sometimes releases. Furthermore, it's possible to both unpublish and delete release nodes, so it seems like we should have a plan for dealing with it when it happens, or completely disallow it on d.o (even for site admins).

I'm a little torn on how to go forward here, since it's really a wider question about the workflow for d.o admins. However, I'm worried that a long debate about it will introduce too many delays to actually fixing the underlying problem here. :( Any input would be welcome. Meanwhile, I guess I'll go back to http://drupal.org/node/203313 and http://drupal.org/node/204140 to make more progress there while we figure out exactly what we want in here.

Chris Johnson’s picture

I think I agree with Gábor here -- and with Derek. So ideally we want to head in the direction Gábor discussed. But right now, I think we need to do something to handle the current situation without adding in additional delays.

So I'd suggest we go with Derek's suggested solutions here and make a real effort to re-visit the workflow question after the D6 release.

hunmonk’s picture

i would agree with Chris's agreement with Derek ;)

this late in the release cycle, we shouldn't be making fundamental changes in the way that projects are handled. let's get things fixed based on our current workflows, and we can tailor a more complete solution in 7.x at our leisure.

Gábor Hojtsy’s picture

Priority: Normal » Critical

Elevating importance, as this is important to fix for security.

dww’s picture

I should note: my current patch in here is actually flexible enough to handle both workflows. It knows if a release node is flagged with either "insecure" or "unsupported" and displays the right thing accordingly. It also handles the "not available" case for unpublished releases. So, really, if this patch goes in, we can revisit our workflow on d.o and move towards the entirely flag-based approach without breaking existing update.module clients in the wild.

Assuming Dries/Gabor can live with that, the only remaining thing in here is handling similar stuff for when the entire project is either unpublished or flagged as "unsupported" or "insecure" (question G).

@Gabor: Yes, I've considered this critical all along, but chx talked me out of it when I wrote comment #1. ;) This patch (and the related changes to project*) is my top priority these days.

Dries’s picture

Re: question G -- regardless of workflow decisions, Drupal allows nodes to be unpublished so I think we should deal with it properly. I think flagging it in the XML-file is a good idea.

There is a question around terminology though. We can use the word "unpublished" but it's sort of Drupal-specific. Is that the word you would use outside the context of Drupal? Or would something like "revoked" be more natural? This is minor, but I think we should use the proper semantic name for such a flag rather than spreading Drupal-isms (assuming that is the case).

dww’s picture

Right, I agree the terminology of "unpublished" isn't great, which is exactly what Question F is about. Note that the XML just uses the drupal-ism <status>published</status> vs. <status>unpublished</status>. But, the open question is what the UI should say (which has implications for internal variable names, PHP constants, etc).

I think "not available" is ok, but probably isn't strongly enough worded (it's not sufficiently scary). "Revoked" or "recalled" is probably better. I kind of like the ring of "revoked". Any other suggestions? I can make a little more progress on other related things before I come back to re-wording this in this patch, so if folks have other good ideas, let's hear them. Thanks.

keith.smith’s picture

Regarding Question F, I like "withdrawn", but understand that others don't.

What about "embargoed"? :)

Gábor Hojtsy’s picture

Revoked or withdrawn all sound good to me. Let's not take too much time on nitpicky details of naming here :) I agree that "not available" sounds that it is only a temporary error.

catch’s picture

'revoked' sounds good.

dww’s picture

Assigned: Unassigned » dww
Status: Needs review » Needs work

Ok then, "revoked" it will be. This patch therefore needs work for 3 things:

- s/NOT_AVAILABLE/REVOKED/
- print a scary "revoked!!" error when a project is unpublished
- support both the flagging and unpublishing workflow for core release nodes

FYI: both http://drupal.org/node/203313 and http://drupal.org/node/204140 are now done and installed on http://project.drupal.org which should significantly aid in testing this patch. Just set your 'update_fetch_url' setting to 'http://project.drupal.org/release-history'.

I'll take 15 minutes now and see if I can get all this done before I completely crash. ;) Stay tuned...

dww’s picture

Status: Needs work » Needs review
FileSize
18.88 KB

This fixes all 3 items from my previous comment. The code was in fact ready in under 15 minutes, but I spent some extra time setting up some stuff to facilitate testing. ;)

I added "Insecure" and "Unsupported" terms in the p.d.o "Release type" vocabulary, so folks can start playing with those. I set core's 6.0-beta1 release as "Unsupported" so folks can see what that looks like with this patch by hacking their system.module VERSION constant (and again, pointing 'update_fetch_url' to p.d.o as explained above).

It's after 3am here, so I can't promise this is absolutely rock solid code, but it's pretty darn close. This needs serious review now. I'm *REALLY* hoping this is RTBC and committed before RC2 is released.

dww’s picture

FileSize
19.75 KB

I was thinking about some edge cases, and decided to take a stab at code to support flagging projects with "unsupported" and "insecure", too, even though we're far from having those tools in the workflow on d.o itself. With some manual hacking on the XML release files on p.d.o, this seems to work as expected. I'm a little hesitant to put code into core for stuff that doesn't actually exist yet, but given that it's going to be basically impossible to make any further changes to the D6 update.module after about RC2, this might buy us some nice flexibility for the next year on d.o. There are definitely advantages to flagging projects instead of completely unpublishing them, so it *would* be nice to support that at some point.

And, even if you forget about all that, I think we'll want to specify this <project_status> attribute to "unsupported" if a given project drops support on all branches for a given version of core via the new UI at http://drupal.org/node/203313 (see http://drupal.org/node/204140#comment-683157). So, I think we'll need this patch for correct functioning, even if it's just because a maintainer dropped support for a given version of core at some point in the future...

p.s. The script that's generating the XML release files is still a little buggy (e.g. the project links are fubar), so don't let that scare you when testing either of these patches. ;)

catch’s picture

OK I have everything set up to test this, but for some reason
define('VERSION', '6.0-beta1'); in system.module is still reporting as 6.x-dev on the available updates page, so I'm not getting a warning. Everything else is good to go though. I can only assume I'm doing something stupid, but any help appreciated.

dww’s picture

@catch: did you start from CVS or a tarball? See if your core .info files define version once or twice. If they override the "version = VERSION" with a 2nd entry from the packaging script, changing VERSION won't help you...

catch’s picture

tarball and yes they do. Hacking system.info didn't help me either btw.

dww’s picture

@catch: that's right. It's a little hard to explain, but update.module can't know which .info file to trust as "the canonical .info file to use" in a large project with many modules. So, it just uses the first one it finds. I just double checked and it's modules/block/block.info that it happens to use for core. ;) Therefore, you should either deploy from CVS for testing this, have a nice little sed/perl 1 liner to find all your .info files and rip out all the extra "version = 'foo'" lines that aren't "version = VERSION", or just hack block.info instead of system.info. ;)

p.s. I doubt it's a good idea, but we could add a special case for core to try to force it to use system.info instead of the first .info file it finds, but that just seems like bloat for no particularly good reason, besides that's what some folks might expect.

catch’s picture

FileSize
14.66 KB

guess which one I chose.

Looks great to me, screenshot attached.

dww’s picture

note to testers: you should check out both the available updates report (admin/reports/available) and the status report (admin/reports/status) in each of the various combinations you try out. thanks.

webernet’s picture

I could be wrong, but isn't isset($release['terms']) && isset($release['terms']['Release type']) equivalent to isset($release['terms']['Release type']) ?

dww’s picture

FileSize
22.49 KB

@webernet: lo, I tested it and you're correct. The single isset() works without any PHP errors. Phew. ;)

New patch:
- Removes the cruft isset()s as per #45.
- Updates the phpdoc for update_calculate_project_data() to explain the new logic.
- Smarter handling of the supported majors list.
- Ensures we never list an "Also available" release that's not itself supported.
- Cleans up a few other minor code comments.

dww’s picture

FileSize
22.49 KB

Bowing to pressure from chx and webernet in IRC, and to make Gabor happy, here's a new version that consolidates these:

    $row .= '<span class="not-current">';
    $row .= t('Update available');
    $row .= '</span>';

into these:

    $row .= '<span class="current">'. t('Up to date') .'</span>';

;) Everything else untouched.

dww’s picture

FileSize
23.51 KB

chx and I were discussing this patch in IRC, and he suggested a potentially cleaner approach to update_requirements() logic. Once we handle core, when looking through all of contrib to figure out what status to use for the status summary, we can just sort the projects by their status values (which are nicely ordered numerically in the right order of precedence). Then, we just pull the first project off the sorted pile and use that status to figure out what to say. Seems to work nicely in local testing, but more testing and other eyes would be appreciated.

Note that the sort function has to be a little funky regarding negative status values, since we always want to consider those last (they're error conditions like a module with an .info file that's lacking enough information to do our thing, or we can't find any available releases for the project, etc).

dww’s picture

FileSize
23.6 KB

And lo, more simplification for update_requirements(). Added a new _update_requirement_check() helper method to avoid a lot of duplicate code between the core and contrib cases.

Moderately tested and seems to be fine. Again, more eyes and tests welcome.

dww’s picture

FileSize
24.48 KB

Further testing results in a yet better patch. This handles some of the more obscure cases, like if you unpublish all the release nodes for a given version of core. The previous code used to incorrectly call this "no available releases" (and throw a PHP notice if you were running a dev snapshot) but we now correctly retain the UPDATE_REVOKED status and mark it as your release has been revoked (unpublished). In general, this logic is smarter about bailing out once we find the right status instead of continuing to process something which might clobber what we already know. Also, I trimmed a little stray whitespace that had snuck into previous patches.

My "let's see if I can still break it" testing is coming up pretty empty. Everything seems to be working nicely as far as I can tell. I know this is a big patch, but it's sorely needed. Not sure what else I can do here...

dww’s picture

FileSize
24.47 KB

Minor typo in a comment.

dww’s picture

Some questions about _update_requirement_check()...

Currently, if your version of core is stale, the status message for core includes the recommended version (if we know it), but only for two cases: NOT_SECURE and NOT_CURRENT. This seems a little wonky.

Should we...

A) Just remove the version stuff entirely? That'd simplify the function (and in fact, allow us to just pass the status, not the whole project array).

B) Keep this info, but append it to the 'value' after the switch, so we handle it once? Something like:

  switch ($status) {
    case UPDATE_NOT_SECURE:
      $requirement['value'] = t('Not secure!');
      break;
    case UPDATE_REVOKED:
      $requirement['value'] = t('Revoked!');
      break;
    case UPDATE_NOT_SUPPORTED:
      $requirement['value'] = t('Unsupported release');
      break;
    ...
  }
  if ($status != UPDATE_CURRENT && $type == 'core' && isset($project['recommended'])) {
    $requirement['value'] .= ' '. t(version @version available)', array('@version' => $project['recommended']));
  }

That'd be nice from a code/consistency perspective, but I guess that's worse for translations (e.g. RTL) because it hard-codes the order of the 2 chunks of the text. :(

C) Just duplicate the whole core special case in every single case in the switch?

D) Not care, and leave the code as is (which covers the 2 most common cases for core, anyway)?

Any input would be welcome, but this is certainly not critical to hold up the rest of the patch. It does have implications for the string freeze, so I guess it'd be nice to finalize this before the next release.

Gábor Hojtsy’s picture

IMHO go with (B), it is not an RTL issue at all (it is the job of browsers to reverse stuff when needed), and just reinforces the logical relation of information here. It is not a concatenated flow of words, but two distinct components: status and version availability concatenated.

dww’s picture

FileSize
24.24 KB

I spoke to Gabor and he prefers (B):

IMHO go with (B), it is not an RTL issue at all (it is the job of browsers to reverse stuff when needed), and just reinforces the logical relation of information here. It is not a concatenated flow of words, but two distinct components: status and version availability concatenated.

So, here's a re-roll.

merlinofchaos’s picture

A visual review of this code looks good. Will need to find some time to test this.

dww’s picture

Status: Needs review » Needs work

Hehe, if you checkout core from HEAD (6.0-dev), and don't use cvs_deploy, the status report stuff is broken:
"Drupal core update status Up to date (version 6.0-beta2 available)" colored in red as an error.
And the resulting message is just: "See the available updates page for more information.".

I know what's up... it's a bug in _update_requirement_check(). stay tuned for a re-roll.

dww’s picture

Status: Needs work » Needs review
FileSize
24.27 KB

Indeed. We just didn't handle negative status values properly in there.

dww’s picture

FileSize
24.8 KB

Upon further consideration, #57 is the wrong approach. We shouldn't just lie and say core is "Up to date" if we don't know the status. This patch adds support for a REQUIREMENT_WARNING if the status is unknown.

dww’s picture

FileSize
24.83 KB

After consultation with webernet in IRC, we decided to slightly change the wording for the status report strings when you hit the REVOKED or NOT_SUPPORTED cases. In many cases, if something is revoked, there's nothing to upgrade to. So, the text now reads:

The installed version of at least one of your modules or themes has been revoked and is no longer available for download. Upgrading or disabling is highly advised! See the available updates page for more information.

Otherwise, identical patch.

dww’s picture

FileSize
26.22 KB

This might be too much feature creep, but here's a version that better distinguishes an unpublished project from an unpublished release by filling in the 'extra' array for projects that are "bad" with verbose, human-readable messages. It'd be more of a pain to split UPDATE_REVOKED into UPDATE_PROJECT_REVOKED and UPDATE_RELEASE_REVOKED, but we could do that, too, if we wanted to change the "Revoked!" text, too...

dww’s picture

FileSize
26.25 KB

Bah. PHP notice in previous patch, and slightly new wording based on suggestion from webernet.

dww’s picture

FileSize
27.24 KB

In IRC, Goba was in favor of a similar message when a release was revoked, not just a project revoked.
Also, based on wordsmithing by keithsmith and webernet, s/highly/strongly/ ;)

webernet’s picture

FileSize
27.33 KB

A couple more changes to the text.

webernet’s picture

In my testing, this seems to be ready to go. One more round of testing/review (especially cases where there are multiple 6.x branches) would be good though.

dww’s picture

FileSize
27.24 KB

1 more tiny wording change (s/./!/ for one of the serious messages) and fixed a code comment.

dww’s picture

FileSize
28.11 KB

Yay, I found another pretty obscure edge case that threw PHP errors...

- Install 6.x-1.x-dev for a module.
- Create a 6.x-1.0 official release.
- Unpublish the 6.x-1.x-dev release.

We get errors in update.report.inc where we assume (foolish me!) that if we're running a -dev, that a -dev release must exist. ;) Now fixed (just added 2 trivial isset() calls in the right places).

If these are the only kinds of bugs I'm finding, we've got to be pretty damn close to RTBC. ;)

dww’s picture

FYI: #66 was broken before this patch, it's just another bug being fixed here, not a new bug introduced by earlier patches. Also, sorry I misspoke -- they're PHP notices, not errors. But still, we should avoid them. ;)

dww’s picture

FileSize
28.58 KB

More verbose code comment about why we never recommend the admin downgrade their site:

      // Make sure we never tell the admin to downgrade. If we recommended an
      // earlier version then the one they're running, they'd be screwed by DB
      // migration issues, since Drupal never supports a DB downgrade path for
      // their data. In the unfortunate case that what they're running is
      // unsupported, and there's nothing newer for them to upgrade to, we
      // can't print out a "Recommended version", but just have to tell them
      // what they have is unsupported and let them figure it out.
jrabeemer’s picture

// earlier version then the one they're running, they'd be screwed by DB

We can say this better. :-/

dww’s picture

FileSize
28.58 KB

*sigh* Ok, here's a G-rated version to appease the do-gooders... ;)

      // Make sure we never tell the admin to downgrade. If we recommended an
      // earlier version than the one they're running, they'd face an
      // impossible data migration problem, since Drupal never supports a DB
      // downgrade path. In the unfortunate case that what they're running is
      // unsupported, and there's nothing newer for them to upgrade to, we
      // can't print out a "Recommended version", but just have to tell them
      // what they have is unsupported and let them figure it out.
dww’s picture

p.s. momendo, in the future, how about suggesting better wording in cases like this? also, how about testing or reviewing the patch? you're nit-picking the wording of a code comment -- it's not user-facing text. this is a critical bug i'm fixing here, and the only feedback you have is to gripe about "screwed" in a code comment? please. ;)

dww’s picture

FileSize
28.75 KB

Found another pretty obscure edge case that was broken:

- install 6.x-1.0
- only 6.x-1.0 and 6.x-1.x-dev exist
- mark 6.x-1.0 as "unsupported"
- check for updates

It would tell you 6.x-1.0 is unsupported, but that 6.x-1.0 was recommended, since it always tries to recommend official releases over -dev. However, we need to skip unsupported releases, much like unpublished ones, for the purposes of finding a recommended release.

I created a bogus 6.x-1.0 release of coder on p.d.o, so you can test this by installing coder 6.x-1.x.dev on your site, and modifying coder.info with:

version = 6.x-1.0
dww’s picture

FileSize
28.81 KB

Similar edge case existed for release nodes marked "Insecure". We'd only hit this case if there was never a valid "Security update" release node published. Basically, you'd have to have *all* of the release nodes for a given version of core marked "Insecure" to hit it. But, that could happen, so this is now fixed, too.

greggles’s picture

Status: Needs review » Reviewed & tested by the community

dww and I just tested a few scenarios:

1. The case of using 6.x-1.x-dev, the release node gets unpublished, there is no 6.x-* to refer them to, the user learns that "No available releases are found"
2. using 6.x-1.0, 6.x-1.0 goes unsupported, recommend that the user downgrade to 6.x-1.x
3. using 6.x-1.x, 6.x-1.0 goes unsupported, 6.x-1.x is marked insecure, inform the user that the module is insecure

Given these tests, webernet's tests, this seems to functionally be doing quite well, so let's RTBC.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Latest patch looks good to me. All Dries' and my concerns were properly taken care of and there was lost of testing and edge case covering going on here. So thanks for the hard work! Committed.

dww’s picture

Title: String freeze: Update.module doesn't warn sites when their current release has been unpublished. » Update.module doesn't warn sites when their current release has been unpublished.

woot! Thanks! ;)

dww’s picture

Title: Update.module doesn't warn sites when their current release has been unpublished. » Update_status.module doesn't warn sites when their current release has been unpublished.
Project: Drupal core » Update Status
Version: 6.x-dev » 5.x-2.x-dev
Component: update.module » Code
Status: Fixed » Patch (to be ported)

Oh yeah, we gotta backport this to contrib... There are some attempts at this earlier in the thread, but the latest patch includes significant changes since then, so we should probably just start over with a clean backport. I'd unassign myself in the vain hope someone else would help, but that's foolish. I know good and well this will only get done if I do it... Such is life.

dww’s picture

Status: Patch (to be ported) » Needs review
FileSize
28.04 KB

Backport to DRUPAL-5--2 branch of update_status. Pretty well tested. If anyone else wants to review/test, that'd be lovely.

dww’s picture

Whoops, sorry, wrong patch...

dww’s picture

Eek, that one was slightly broken, too, due to the confusion of UPDATE_STATUS_* vs. UPDATE_* constants. :( I should probably just rename all the constants in the contrib version to match D6 core, but that's for another issue.

dww’s picture

Title: Update_status.module doesn't warn sites when their current release has been unpublished. » Update.module doesn't warn sites when their current release has been unpublished.
Project: Update Status » Drupal core
Version: 5.x-2.x-dev » 6.0-rc2
Component: Code » update.module
Status: Needs review » Fixed

Discussed this with Earl previously -- he thought I should just commit and let -dev users test and report any problems they might find. ;) Committed to DRUPAL-5--2 of contrib, along with a new update_status.pot. Moving this issue back to the core queue for posterity.

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.