Please could someone explain why Imagecache images are still served through PHP? It seems that only full not cached images are served through x-sendfile.

Could it be that image cache/style images when used somehow remove the xsend header? Can this be fixed?

CommentFileSizeAuthor
#10 support_for_image_derivatives-2621550-10.patch4.93 KBñull
#7 xsendfile_image.tar_.gz1.97 KBñull
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

ñull created an issue. See original summary.

damien_vancouver’s picture

damien_vancouver CreditAttribution: damien_vancouver as a volunteer commented

Please could someone explain why Imagecache images are still served through PHP?

Are you sure they are being served through PHP? Usually they are served by your webserver directly, and if you check the path to the file, it will be something like: http://example.com/sites/default/files/styles/large/public/field/image/f... PHP is not running at all in this case, Apache is serving the .JPG file in the URL directly. Since PHP/Drupal isn't involved and the 'system/files' handler is nowhere in the path, so XSendfile never has the chance to do its magic.

As far as I can tell, Imagecache/Style derivatives do not work on fields stored in private files. I just created a new content type with an Image field saved in private files and when I go there I see the full size image, imagecache is just not functioning.

I can find several issues discussing problems with styles and private filesystems, and the Insecure Image Derivatives fix from Drupal 7.20 seems to be when those problems started.
See: #1903190: Allow image style derivatives of private images to be stored on the public file system , #1440312: Image style previews don't show when using a private file system, and #1438940: Private stored default images give access denied for some examples of issues that are open still regarding this problem. Most of those seem to be concerned with 403 Forbidden errors when trying to edit the styles, which seem to be all saving to sites/default/files. I suspect somewhere along the way the styles were just disabled altogether for private files, and that is why I do not see them on my test site, only the original unscaled, huge image.. in both teaser and body view.

As far as how to make XSendfile work for those image paths, I'm not exactly sure how you'd do that, it would have to be in the webserver configuration or with a webserver module. Maybe you could trick it into working with mod_headers somehow. But without the X-Sendfile: header being present, the mod_xsendfile webserver module doesn't do anything, as you can see if you look at its source.

Does that explanation make sense? The slow-ness that XSenfile would be solving (using PHP to serve the file) isn't present if you're not going through PHP, which you probably aren't. So while it's not working, that's not a big deal unless PHP is running too.

If you somehow have it working using private files and the files are being fetched by PHP, please post the path that it's using and that may give us a clue as to how to make it work.

ñull’s picture

ñull CreditAttribution: ñull as a volunteer commented
Title: Imagecache (styles) always PHP served » Support for image derivatives
Component: Documentation » Code
Category: Support request » Feature request

After studying the code of both xsendfile and the core image module, I conclude that image has its own menu callback for the derivatives and it sends the files using core file_transfer. To make it use xsendfile we would need to add menu_alter with callback for this and provide our own call back function. I will let it rest for some time and come back to this once a I feel inspired to write this code.

damien_vancouver’s picture

damien_vancouver CreditAttribution: damien_vancouver as a volunteer commented

So, you're sure that your imagecache derivatives are coming from private files?

What is the URL to one of your derivative files? No matter what I tried I couldn't get it to serve them via private files. Do you know how I would reproduce?

Your analysis of how to do it sounds correct to me.

ñull’s picture

ñull CreditAttribution: ñull as a volunteer commented

The derivative certainly is served through PHP. The steps to reproduce:

  • Install image, insert and wywsiwyg and ckeditor
  • create a private file path outside document root
  • Create a content type (in my case Page) with the image field set
    to private and open the insert collapsed form to allow at least one
    style and the original sized image and don't forget to activate
    insert
  • Change the permissions to disable view access for anonymous users.
  • Disable tokens by adding this to settings.php: $conf['image_allow_insecure_derivatives']=TRUE;This is to make sure that the token system does obscure our proof of
    concept.
  • Create a test node with the content type, write some text, upload
    an image and use the insert button to insert both an image style and
    the original to the node body and submit. See the derived smaller
    and original image together appearing in the node view.
  • For later testing copy the URL of the derivative (in my case
    something like
    http://localhost/system/files/styles/medium/private/web-hosting.png)
  • Log-out and see the access denied message.
  • With your browser now try to access the previously copied the
    image URL. If you still see the image, try to clear the cache and
    access again. Eventually you should see an access denied message,
    disallowing you to access the derivative. After log-in you should
    see it again.

Edit: May be of little consequence, but the secure file system is set as the default.

ñull’s picture

ñull CreditAttribution: ñull as a volunteer commented

To proof that xsendfile does not handle the image derivatives you disable it in .htaccess:

<IfModule mod_xsendfile.c>
    XSendFile off
</IfModule>

...leaving the xsendfile module active. Now you log-in again and go back to the test node we just created. It will still display the derivative image but the original image will not, because it should be served through the now deactivated xsendfile. When you check with firebug, you will see that the request header of the original image will include the X-Sendfile //my-full-private-folder-path/files/web-hosting.png

ñull’s picture

ñull CreditAttribution: ñull as a volunteer commented
FileSize
xsendfile_image.tar_.gz1.97 KB

I here submit my hack, a sub-module that overrides the core image_style_deliver. Likely this can be much simplified, but that is than the work for somebody more savvy.

damien_vancouver’s picture

damien_vancouver CreditAttribution: damien_vancouver as a volunteer commented
Assigned: Unassigned » damien_vancouver
Status: Active » Needs work

Awesome, thanks for the steps, I'll try again to reproduce.... I must have missed something along the way.

So, looking at your module, you've basically done the same thing as in the main xsendfile module, except using hook_menu_alter to override system/files/styles/%image_style and swap out the file_download for xsendfile_file_transfer as needed.

I don't see what you've done there as being any more hacky than XSendfile itself... So don't trash talk it! :)

I will try and get the problem reproducing for me with your instructions, and then get your code into the -dev branch of the module so we can test it out. XSendfile obviously should work on all images if possible, including derivatives! Thanks for your work figuring this out!

ñull’s picture

ñull CreditAttribution: ñull as a volunteer commented

When I deactivate xsendfile with the setting "Files transfered by PHP" then it will no longer display the derivatives. In my sandbox I removed the if ... else here and am testing this, because, according to my debugger for some reason it is not reading the variable_get xsendfile_transfer. In fact it is not needed because withing xsendfile_file_transfer this is tested anyhow.

ñull’s picture

ñull CreditAttribution: ñull as a volunteer and commented
Status: Needs work » Needs review
FileSize
support_for_image_derivatives-2621550-10.patch4.93 KB

The tested code as sub-module. Should it be a sub-module? If not, may be I should include it in xsendfile?

ñull’s picture

ñull CreditAttribution: ñull as a volunteer and commented
morbiD’s picture

morbiD CreditAttribution: morbiD commented
Status: Needs review » Needs work

I've tested the patch and it seems to work nicely, but being old code it suffers from this issue which was fixed in core a couple of months later: #1891228: image_style_deliver can create invalid headers

It was a simple 2-line fix so it should be easy to incorporate here.

Regarding the submodule, I think most people would probably expect this behaviour by default, so I'd say the code might be better off in the main module, perhaps with an option to turn it off in the module's admin settings?