Latest cache updates result in scrambled output

hmm... maybe a conflict with your zlib configuration in PHP? Could you please the value of the option "zlib.output_compression" in your PHP settings (path: admin/reports/status/php) ?

If you're using compression in PHP, then try this change in admin_menu.module ?

   // Determine if the client accepts gzipped data.
-  if (isset($_SERVER['HTTP_ACCEPT_ENCODING']) && function_exists('gzencode')) {
+  if (isset($_SERVER['HTTP_ACCEPT_ENCODING']) && function_exists('gzencode') && zlib_get_coding_type() == FALSE) {
     if (strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') !== FALSE) {

Ok, here's a patch that applies the change in #1, also extends the inline documentation of that snippet.

I had this problem too, and disabled gzencode, not familiar with this stuff, but while you are at it:

There seems to be logical mistake in the next line, the elseif can never be true

if (strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') !== FALSE) {
      $encoding = 'gzip';
    }
    elseif (strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'x-gzip') !== FALSE) {
      $encoding = 'x-gzip';
    }

Oops. True.

All this snippet comes from the example module I posted on the issue where client-cache was implemented in admin_menu. And I added this check for x-gzip compression based on code I had for something different... but the fact is Drupal itself does not perform this check when dealing with page compression, so I guess we can get rid of the check for x-gzip here.

Hence, patch updated.

Applied the patch in #5. This does not completely fix the issue: on the first page after a flush cache, admin_menu comes up, then on subsequent views, the hyeroglyphs reappear.

FWIW, the setup is like this:

• PHP 5.2.9 on Apache 2.2.8 on Win32
• _SERVER["HTTP_ACCEPT_ENCODING"] deflate, gzip, x-gzip, identity, *;q=0
• _SERVER["HTTP_TE"] deflate, gzip, chunked, identity, trailers
• ZLib Support enabled
• Stream Wrapper support compress.zlib://
• Stream Filter support zlib.inflate, zlib.deflate
• Compiled Version 1.2.3
• Linked Version 1.2.3
• Tested in IE8, FF3, O9

This might be related to #431202: Identical cache $cid incompatible with CacheRouter

The same problem.
Patch #5 doesn't solve the problem after clearing the cache.

Cross-referencing potentially duplicate issues:

#442560: Admin menu disappears (randomly), needs either cache to be emptied on every page or page refresh several times
#431202: Identical cache $cid incompatible with CacheRouter
#424960: Latest cache updates result in scrambled output

Please have a look at each one of them; reconsider whether anything in any follow-up might give you a clue, and report back your findings (in the corresponding issue).

Additionally:

1) Do you have JavaScript enabled?

2) Do you get any JavaScript errors?

3) Do you get a "ok-looking" raw HTML output when accessing the cache URL directly? (i.e. http://example.com/js/admin_menu/cache/98ho3inf2p1qwrf...)

4) Is Drupal installed in a sub-directory? Any other potential specials on your host/server?

1) Javascript working properly
2) No javascript errors
3) Accessing directly /js/admin_menu/cache/80daa77d7b21e13b349ace33f84396ac contain rubish as well
4) No

Tested as well with PHP 4.x and PHP5.x

Ok, I've tested latest development, I can't reproduce it on PHP5 anymore.
Can anyone (after clearing cache) could reproduce it on PHP5 or only on PHP4?

Regardless of #7... comment by drupal24 in #4 had a point, so something like the patch in #5 is still necessary.

Patch for scrambled output based on similar bug in Drupal core #43462: cache_set and cache_get base_url brokenosity
+ fix for condition in #4

Tested on PHP4 as well.

@ #13: I believe we can add a check for isset($_SERVER['HTTP_ACCEPT_ENCODING']) before variable_get('page_compression', TRUE), so that we can save a few steps if the user agent does not support compression.

Note that this is not done in the patch you've linked to because Drupal does not know if the user agent supports compression while caching the page. Drupal performs this check whenever the page is served from the cache.

Assigning proper status.

@markus_petrux: Do you want to be this patch's champion? If you want, just let me know when it is ready. :)

I am unable to enable zlib.output_compression in PHP with Drupal 6, so it's hard to tell for me. I think it's Drupal that's buggy here.

Anyone able to enable zlib.output_compression in PHP with Drupal 6?

Anyway, here's a new version of the patch that I was able to test successfully with zlib.output_compression enabled and disabled, and Drupal page cache enabled and disabled, for the particular request of the admin menu. My Drupal pages are full of garbage when I enable zlib.output_compression (regardless of page cache option), but the admin_menu worked with all combinations. I think it worked with the admin_menu because this request does not play with the caching mechanism in Drupal.

Drupal's page cache (which sends compressed output) only works for anonymous users.

     if (!empty($encoding)) {
       header('Vary: Accept-Encoding');
       header('Content-Encoding: ' . $encoding);
-      $content = gzencode($content, 9, FORCE_GZIP);
+      // Compress content only if zlib.output_compression option is disabled.
+      if (zlib_get_coding_type() == FALSE) {
+        $content = gzencode($content, 9, FORCE_GZIP);
+      }
     }

If I'm not mistaken, also those header() lines need to moved into the condition.

Comparing the current code to http://api.drupal.org/api/function/drupal_page_cache_header/6, makes me wonder whether we're setting Content-Encoding properly. It seems like core only sets that header when it sends truly gzipped data (see also http://api.drupal.org/api/function/page_set_cache/6).

I tested latest patch and for some reason, when I enabled zlib compression in PHP, the admin_menu was garbage unless I include the Content-Enconding header. The Vary header is for caches to keep separate versions of the page based on user agent capabilities.

I would suggest that the patch is tested by those that reported problems.

Subscribing. I get scrambled output due to gzip compression.

@mcload: Could you please describe which is your configuration in regards to gzip compression? How do you have this enabled in PHP? How do you have configured Drupal page cache, etc?

Also, have you tried the patch in #16? Any difference with or without that patch?

I tried the patch in #16 and it worked! Thank you. Problem is solved for me by the patch.

I do not use Drupal's cache. JS/CSS optimizations are enabled. I enabled Apache gzip compression to php, css and js files by adding this to main folder's .htaccess file.

php_value zlib.output_compression 16386
AddHandler application/x-httpd-php .css .js

BTW, my gzip compression method can not compress JS and css files optimized by Drupal. I started a topic in the support forum at http://drupal.org/node/484264 . I would be glad if you share how you compressed optimized js/css files in that topic.

With some additional inline comments, basically explaining the same as #18, I would be ok to commit this patch.

Aha, thanks for confirming the patch in #16. :)

Regarding compression of js/css files... these files are not processed by PHP, so it should be done by the web server. With Apache, you could use mod_deflate, but there are other alternatives that may better suit your particular needs based on available infrastructure, etc. A quick link to Drupal search reports a few posts about it:

http://drupal.org/search/apachesolr_search/mod_deflate

We have squid in front of our Apaches, so a combination of mod_deflate + Vary header suits our needs perfectly well. The squid server will cache different versions of each file based on the compression method supported by each browser.

Edited: @sun #22: Oh, I just read your post after submitting my follow up. hmm... ok, I'll try to roll a patch with a bit more comments. :)

@sun: I've been thinking again about your comment in #17, trying to understand what was really happening, and I ended up writing the whole stuff from scratch, trying to apply questions where the answer is (or should be) simple.

The result is the attached patch. There are a lot of combinations, and I think I tried a few. It may need more testing though. The good thing, I hope, is the logic applied to perform or not perform gzip compression is pretty simple and clear.

So I would say... please test. If you still see garbage rather than the admin menu, then maybe the problem is somewhere else. If someone reports problems, we would need to know if Drupal page compression is enabled, if the site is using mod_deflate or any other method of compression, also how is the PHP zlib extension configured, also which browser, because there may be user agents that tell they support compression while they don't, so better test with various browsers to see if that makes a difference.

I forgot to update the issue status, oops!

Wow, that novel is a pretty good read! :) Sounds and looks sane now, though I will probably rework that a bit before committing ;)

@mcload + others: Can you test markus_petrux's new patch, please?

I tested it. It is working for me.

Reworking the comments as well as forward-porting and back-porting the conditions properly will take a fair amount of work I need to spare some time for, if no one beats me to it.

Here's a version that reduces the novel (lol) a little.

I opted to post the code because it might be reviewed easily.

Note that the conditions and the order they are evaluated are the same as the patch in #24.

/**
 * Menu callback; Output administration menu for HTTP caching via AJAX request.
 */
function admin_menu_js_cache($hash = NULL) {
  // Get the rendered menu.
  $content = admin_menu_output();

  // @todo According to http ://www.mnot.net/blog/2006/05/11/browser_caching,
  //   IE will only cache the content when it is compressed.
  // Determine if the client accepts gzipped data (quicker than other checks).
  if (isset($_SERVER['HTTP_ACCEPT_ENCODING'])) {
    if (strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'x-gzip') !== FALSE) {
      $encoding = 'x-gzip';
    }
    elseif (strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') !== FALSE) {
      $encoding = 'gzip';
    }

    // Perform gzip compression when the following conditions are met:
    // a) The user agent supports gzip compression.
    // b) Drupal page compression is enabled. Sites may wish to perform the
    //    compression at the web server level (mod_deflate, etc.).
    // c) The PHP zlib extension is loaded, but the zlib compression option
    //    is disabled (zlib.output_compression = Off).
    if (!empty($encoding) && variable_get('page_compression', TRUE) && extension_loaded('zlib') && zlib_get_coding_type() === FALSE) {

      // Vary header is included to tell caches to keep separate versions
      // of the admin menu based on user agent capabilities.
      header('Vary: Accept-Encoding');

      // No one else performed the compression, so if we do, we are also
      // responsible to send the proper encoding header.
      header('Content-Encoding: ' . $encoding);

      // Finally, compress the output.
      $content = gzencode($content, 9, FORCE_GZIP);
    }
  }

  $expires = time() + (3600 * 24 * 365);
  header('Expires: ' . gmdate('D, d M Y H:i:s', $expires) . ' GMT');
  header('Last-Modified: ' . gmdate('D, d M Y H:i:s') . ' GMT');
  header('Cache-Control: max-age=' . $expires);
  header('Content-Length: ' . strlen($content));

  // Suppress Devel module.
  $GLOBALS['devel_shutdown'] = FALSE;
  echo $content;
  exit;
}

Feel free to rewrite though. I just hope it helps.

Uhm. A real patch would have been a lot easier.

Is attached patch correct? (Slightly reworked comments)

Thanks for reporting, reviewing, and testing! Committed to all 3.x branches.

A new development snapshot will be available within the next 12 hours. This improvement will be available in the next official release.

i'm still seeing this behaviour - even with the current version of admin menu.

it's not easily reproduceable, as it seems so random; a site can be working perfectly for severeral weeks and then, next time i visit, i'm getting the garbage characters where admin menu should be. sometimes reloading the page a few times fixes it, sometimes running update.php does the job, sometimes clearing caches works - and sometimes nothing will get things back to normal and i have to disable admin menu for a while, until the problem goes away as mysteriously as it arrived.

until it's definitively fixed, i've taken to disabling admin menu on client sites, as i'm getting too many panicky emails saying, "my site is broken..."

re-opening. not fixed for me.

Unfortunately, it seems like no one else was able to reproduce the issue you've raised, @madra.

@madra, if you still experience this problem, can you create a new issue, and make sure to post a lot more details about your platform, browser(s), and so on? Thanks.

Reverting status to fixed.

Linking issue link to this post as its very similar if not same.