I'm not sure what we did, but so far, D7 is a disaster.
At least, for Mollom, I considered that plenty of stuff would get easier. Negative.
Required fix for this patch to produce any remotely useful test results at all: #297860-36: Regression: Convert session.inc to the new database API (and _sess_write should use a merge)
I'm not trying to leverage any Form API improvements (or any other improvements) in this patch yet, so this is a straight port.
However, for any reason, the "Form administration" tests are failing, and I have no idea why. I've changed that test to use the fake testing server... but. When looking at the actual verbose debug output, then the CAPTCHA image is served from xmlrpc.mollom.com and not from the fake server... (?)
In addition to that, various assertions that test for a certain message are reported as fails, but when looking at the actual verbose debug output, the expected message is properly displayed... (?)
Something goes horribly wrong there.
Lastly, Mollom is a very nice example for why the new categorization below "Configuration and modules" is a monster #fail and will lead to opposite of the desired effect, i.e. a much more worse UX. I seriously bet you won't find Mollom's admin pages without looking at the patch.
Comment | File | Size | Author |
---|---|---|---|
#32 | mollom.d7-pre-render.patch | 7.58 KB | sun |
#28 | mollom-HEAD.28.patch | 17.38 KB | sun |
#26 | mollom-HEAD.26.patch | 17.62 KB | sun |
#25 | d7-form-api-cleanup.patch | 17.59 KB | Dries |
#24 | mollom-HEAD.24.patch | 18.74 KB | sun |
Comments
Comment #1
Dries CreditAttribution: Dries commentedThanks for starting to work on an upgrade! :)
- I haven't looked at the patch closely enough yet, but given the many form API hacks that we had to put in the Drupal 6 version of the module, maybe they are no longer supposed to work in a straight port ...
- We need to fix #297860-36: Regression: Convert session.inc to the new database API (and _sess_write should use a merge) because those changes were required to make the XML-RPC error handling work; see #578470: XML-RPC error handling sometimes fails silently. It looks like that patch was accidentally reverted. Because it is accidentally reverted, XML-RPC errors are never reset, which makes it pretty unreasonable to do more than one XML-RPC request per HTTP request. If the first XML-RPC call causes an error, all subsequent calls will incorrectly have the same error set. Clearly, we need to fix that in core. It was pretty broken in Drupal 6 too, so it is not really a regression -- Drupal 6 was in an equally bad state ... ;-)
- As for the IA in Drupal 7 -- I guess I'd expect a link called 'Mollom spam protection' under 'Content authoring' or something. Not sure if that is what you did, but that is where I'd look initially.
- The other thing we'll need to "work around" in Drupal 7 is the new session handling. To enable more aggressive caching, Drupal 7 no longer keep a session if there is nothing to save in that session. As a result, session IDs are not maintained across page requests. Mollom uses the session ID to identify record-replay attacks across page requests. This also affects Pressflow in Drupal 6 because they backported the session handling: see #562374: Mollom incompatible with Pressflow's (and Drupal 7's) session handling -- causes CAPTCHAs to fail -- ideally we'd be able to backport that fix to the Drupal 6 module.
Comment #2
sunIncorporated the patch from #562374: Mollom incompatible with Pressflow's (and Drupal 7's) session handling -- causes CAPTCHAs to fail, which fixes a couple of tests.
Also fixed a couple of more issues. Getting closer. ;)
re 1) Nope, my revamp actually removed all Form API hacks ;)
re 2) Thanks for committing that fix.
re 3) Good guess, but I seriously think that this category is not really appropriate...
Comment #3
sunAlright. I was finally able to understand why the tests are failing... #663992: Verbose debug output is cached by the browser OMG. :P
Comment #4
Dave Reid@sun: Hah. That is a DrupalWTF moment. :)
Comment #5
sunI fear that this will end in yet another >100 KB patch...
Comment #6
sunoh my. It definitely helps to clear the entire browser cache. Spent another couple of hours looking at totally weird but entirely stale debug pages. :-/
Comment #7
sun1 failing test remains.
Comment #8
Dries CreditAttribution: Dries commentedThanks for the clean-up, sun. Great job. It's actually great to see such a patch, if only to evaluate how hard it will be for module maintainers to upgrade their modules. I only skimmed the code for now, but here is what I got.
Redundant 'cid' in the fields() statement?
I saw you did the same earlier on in the code -- if I'm not mistaken. It didn't had a TODO.
Not sure what this is.
We should probably use REQUEST_TIME for these.
Comment #9
sunThe remaining failing test/assertion cannot be resolved within Mollom module - see the @todo I added.
Actually, there are two larger @todos added in this patch. One is bound to Mollom.module's architecture, the other one is about D7's render system, and the latter is the one I mean here.
Note that this patch is still rolled against DRUPAL-6--1; I believe that it should apply cleanly against HEAD though.
Lastly, this patch is still a straight port of the current code, not taking into account any new options in D7. The latter should always be a separate step in porting modules. (You know what happened to CCK in D6, eh? ;)
Comment #10
sunoh, I *love* your reviews! :)
Fixed those.
Not sure about the mollom_form_load() comment though... I guess you mean similar code in mollom.test -- but that code doesn't unserialize the configured fields, just checks whether a $form_id has been stored.
Comment #11
sunFixed some last bits. I think this is ready to fly. We cannot resolve the remaining fail without entirely rewriting the implementation (which we should still try, but I'm not even sure whether it's possible at all).
Comment #12
sunOh. My. God.
I quickly patched batch.inc + form.inc in D6 to get the total elapsed time for running the tests to compare them with D7....................
D6 == 1 min 37 sec
D7 == 4 min 5 sec
errr, and we want to release? No way.
Comment #13
Dries CreditAttribution: Dries commentedWhile Drupal 7 is probably slower than Drupal 6, it obviously shouldn't be that much slower. The performance degradation is probably either the direct or indirect result of the testing framework. I believe installation of Drupal became significantly slower due to (i) switching to InnoDB, (ii) using batch API to install Drupal -- I believe we install one module per batch, and that we do expensive cache clearing in between each batch step, (iii) we do more extensive clearing between tests, (iv) etc. That said, I agree that running the tests is rather slow.
I'll do a more in-depth review tomorrow! :)
Comment #14
sunuh oh - something went wrong during syncing DRUPAL-6--1 with HEAD... All synced files went into the tests/ subdirectory ;)
EDIT: See http://drupal.org/cvs?commit=304076 for what I mean.
Comment #15
Dries CreditAttribution: Dries commentedDang. I think I fixed it the broken CVS HEAD but I haven't really tested it yet.
Comment #16
Dries CreditAttribution: Dries commentedReviewing this patch, it is a rather straightforward port of perfectly solid code. It is hard not to like it, so I have committed to CVS HEAD. I made the following notes when reviewing the patch.
"Drupal 7"-ifications for a follow-up patch:
mollom_node_insert
and leverage the fact that the node ID is available in the form submit handlers.Other things I noticed:
$ /Applications/acquia-drupal/php/bin/php scripts/run-tests.sh --url http://cvs.localhost:8082/ --php /Applications/acquia-drupal/php/bin/php Mollom
, using the Acquia Drupal stack installer. I couldn't run them from within the administration UI -- see below. I get more than one error though:Drupal 7 core bugs that I ran into:
Notice: Undefined variable: args in overlay_close_dialog() (line 525 of modules/overlay/overlay.module)
.Comment #17
sunNeed to talk to Crell or grep throughout core to find out whether this is possible.
Rename to mollom_exit() and eventually move below mollom_form_submit().
The human-readable permission strings need work.
Actually, I implemented this for testing purposes only. Can probably be removed, sorry.
Investigate whether the old code was wrong or whether D7 has a bug here, and why D6 behaved differently.
We probably want to add an internal system variable to track the current "healthiness" of the module's configuration - i.e. if there are no keys, or invalid keys, or whether we have no valid server list, then always invoke the fallback behavior. To be implemented as new (first) #process or #element_validate of the 'mollom' element.
I already spent hours tinkering about this, but still have no idea how we could resolve that.
That's quite a critical regression, because it basically means that no renderable array in the page template can output a message. The only idea I had so far was to eventually inject all messages in an ESI-style approach, i.e. replacing a replacement token/macro in the fully rendered page.tpl.php HTML markup with the actual messages. Quite a hack. Perhaps we simply went too far with usage of render() for the main page content (if the $content variable in page.tpl.php was an already rendered HTML string, then this problem would not exist).
As a workaround for Mollom, it would only be resolvable, if we would revamp the entire implementation, and perform all form alterations and validations during element and form validation.
There are a couple of fixes like this that need to be back-ported. Created #665270: Backport fixes from port to D7 for that.
You are right that we can now try to leverage the 'post_id' mapping property that was intended for this purpose (to be processed in mollom_form_submit()).
In addition to that, I considered whether we can perhaps simplify the current 'report delete callback' that can be defined in hook_mollom_form_info() - and change it to the $form_id of the real/original delete confirmation form of the entity instead.
Hence, instead of duplicating functionality (we always delete), we would only integrate with the existing deletion functionality/workflow.
This is another bug in Drupal core -- Node module registers $node->content['links']['node'] and assigns theme_links() for the defined links in there. The current design of node/comment links seems to have intended to build separate lists of links for each module.
That, however, looks very wonky in the rendered output, and also doesn't make much sense from the generated HTML markup.
Not sure whether there is already an issue for that, but since the fix is likely to be an API change (just skipping the second array level?), we should fix it soon. Not sure whether I'll have time for that.
This review is powered by Dreditor.
Comment #18
sunAttached patch does the trivial fixes.
Comment #19
Dries CreditAttribution: Dries commentedCommitted the bug fixes in #18. Thanks.
Comment #20
Dries CreditAttribution: Dries commentedWith the latest bugfix patch applied/committed, I now get the following:
Is that consistent with your results? You mentioned you only saw one fail so it sounds like it might be inconsistent, and that we have to do some more investigation to figure out why.
Comment #21
sunI just ran the tests again (via GUI) and still get
894 passes, 1 fail, 0 exceptions, and 213 debug messages
whereas this 1 fail belongs to
Fallback behavior
77 passes, 1 fail, 0 exceptions, and 19 debug messages
and is the fail I documented + explained in-code as well as above.
Comment #22
Dries CreditAttribution: Dries commentedOdd, I'm using the following PHP version on the command line:
Anyway, I'll try switching to the UI instead.
Do you happen to run a patched core?
Comment #23
sunIndeed, very odd.
But anyway, let's try whether we can revamp all of this for D7 now. :)
erm, also included -- can we remove that duplicate tests/tests/ directory? ;)
Comment #24
sunThanks for removing the test files!
Comment #25
Dries CreditAttribution: Dries commentedThis shouldn't happen if there is a session.
Made some minor text changes. Update patch attached.
Comment #26
sunuhm, wait - do we automatically disable page caching if there is a session for the current user in HEAD? If this is true, then we need to update Mollom's code in various locations. For now, I didn't merge that changed comment.
All tests pass for me.
What's left? - Entirely killing the #pre_render.
Comment #27
sunerrr, this is the wrong location to prevent page caching ;)
I'm on crack. Are you, too?
Comment #28
sunFixed that.
Comment #29
Dave ReidComment #30
Dries CreditAttribution: Dries commentedGreat, I re-ran the tests with the latest patch and I got nothing but passes:
Comment #31
Dries CreditAttribution: Dries commentedOh, I committed the patch in #28 to CVS HEAD.
Comment #32
sunResolved a couple of @todos:
- Removed #pre_render.
- Implemented hook_modules_uninstalled().
- Removed debugging code.
Comment #33
Dries CreditAttribution: Dries commentedLooks good to me, and all tests pass so committed to CVS HEAD.
TODO for Dries: I want to verify manually that the Mollom session IDs remain intact across different page views / form steps. That is critical for Mollom to work correctly.