Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
This patch pulls the default value of the mime_extension_mapping variable out to its own function so it's accessible to module developers who want to use or alter the mapping.
Comment | File | Size | Author |
---|---|---|---|
#87 | filemime-331171-87.patch | 10.4 KB | pwolanin |
#85 | filemime-331171-85.patch | 10.85 KB | pwolanin |
#81 | filemime-331171-81.patch | 10.08 KB | pwolanin |
#78 | filemime.patch | 9.38 KB | mfb |
#76 | filemime.patch | 9.43 KB | mfb |
Comments
Comment #1
Anonymous (not verified) CreditAttribution: Anonymous commentedYou need to move the variable_get to the file_mime_extension_mapping and store the array data in a static variable.
Actually, why are you using the variable system anyway? Why not just store the data in a static variable and allow the static to change if a value is given?
Comment #2
mfb@earnie: I don't understand your comment about moving the variable_get() to file_mime_extension_mapping(). The whole point of the patch is to make the default value of the variable accessible, not the configured value of the variable.
A variable is used so sites can permanently modify their mime_extension_mapping, although possibly it could have been implemented using drupal_alter() instead, allowing sites to modify it dynamically.
Comment #3
drewish CreditAttribution: drewish commentedreally needs a test.
Comment #4
mfbok.. I'd say a few more assertions for the existing file_save_data() and file_save_upload() tests would be a good enough test coverage but let me know if someone has ideas for more exhaustive tests.
Comment #5
Anonymous (not verified) CreditAttribution: Anonymous commented@mfb: Moving variable_get() to the file_mime_extension_mapping() function will allow for what you want but I don't have to specify it in the call.
So
becomes
If you do
in the function.
The static $map will cause the run time build of the array to occur once in each session refresh instead of each call to the function. The return of the variable_get values allow the use of the function for the map value without the user needing to worry with where the values might come from.
There also needs to be an administration UI if you keep the variable_get.
Comment #6
mfbI added some mime-type related assertions to the existing file and upload tests.
@earnie, the new function is intended to provide access to the *default* value of the variable; your code would only give me access to the configured value. The idea is that developers could use this array in their own functions, besides file_get_mimetype(). Admittedly, it might be I'm the only person in the world interested in this...
Comment #7
Anonymous (not verified) CreditAttribution: Anonymous commentedThen you don't need the function. A defined constant would be better.
Comment #8
mfbI thought about a constant, but this value isn't needed on the vast majority of page requests, and is a rather large array so takes a fair amount of memory. It seems to me like too much overhead to make it a constant.
Comment #9
Anonymous (not verified) CreditAttribution: Anonymous commentedToo much overhead is why I think it either needs to be static or and constant. Building the large array every time you need it is huge. Then you're storing it in the variable system as well. If it's a constant then the PHP optimization engines have a chance to cache it.
Comment #10
mfbAnother reason it cannot be a constant, PHP does not allow arrays as constants.
Comment #11
mfbuse a static variable in case the function is called a second time (e.g. multiple file uploads in one request)
Comment #12
drewish CreditAttribution: drewish commentedtests are an improvement but still needs some dedicated tests for the new and altered functions. we need to:
- call file_mime_extension_mapping() and make sure it returns an array
- call file_get_mimetype() with the default and a made up mapping and check that the right type is returned
i know that these functions are tested indirectly by other tests but we need some tests that specifically target these functions and their edge cases.
Comment #13
drewish CreditAttribution: drewish commentedi transposed the function names in that last comment.
we should also test loading mappings from variables vs. parameters in file_get_mimetype().
Comment #14
mfbQuestion about core unit tests: Should we be using "UnitTest" as the class name suffix?
Comment #15
Anonymous (not verified) CreditAttribution: Anonymous commentedSee http://cvs.drupal.org/viewvc.py/drupal/drupal/modules/simpletest/tests/ for examples.
Comment #16
mfbWhat I was asking about is the test writers guidelines at http://drupal.org/node/325974 suggest using *UnitTest for unit tests. But right now there is only one *UnitTest class in core, DrupalErrorHandlerUnitTest.
Comment #17
drewish CreditAttribution: drewish commentedHere's some tests. Also improved the PHPDoc and renamed the variable to match the function name (giving it some namespace protection).
Comment #18
c960657 CreditAttribution: c960657 commentedI assume the typical use case for modifying the default mapping is adding or replacing a few entries, not supplying a completely different mapping. Wouldn't it be better to let the setting be applied on top of the default mapping?
Comment #19
mfbI'd agree that this would be preferable. Possibilities include making the default mapping an internal static array and applying the setting array, if it exists, on top of it, or using drupal_alter() in the unlikely event that various modules need to merge their additions/replacements.
Comment #21
mfbhere's another possibility: get rid of the variable, and just use drupal_alter to allow the mapping to be altered. This seems like the "right" way to allow large arrays to be altered by contrib modules, rather than using a variable.
Comment #22
Damien Tournoud CreditAttribution: Damien Tournoud commentedI like this last approach.
Only one thing: the static in file_mime_extension_mapping() serves no purpose except hogging memory. It should be removed.
Comment #23
mfbI did find that the static offered a speedup if the function is called twice (e.g. two file uploads in one request) presumably because PHP only has to parse the large array statement once. But, now that I think about it, file uploads are not the requests you worry about speeding up by a fraction of a second so I would agree. another thing is there was a leftover variable_del('file_mime_extension_mapping')
Comment #25
mfbDoes PIFR tell us which assertions failed? I don't have any failures.
Comment #27
c960657 CreditAttribution: c960657 commentedReroll.
Comment #28
c960657 CreditAttribution: c960657 commentedThe last patch contained an unrelated change to registry.inc.
Comment #29
mfbLooks good. One minor tweak, in some cases file.test uploads a jpeg rather than png (it selects a random image) so we can only check that MIME type is image/something.
Comment #30
c960657 CreditAttribution: c960657 commentedYour updates to the patch look sane.
Comment #31
BerdirCopy paste of my "crazy" idea which I posted at http://drupal.org/node/370958#comment-1435888
a idea that was mentioned in the hook_variable_info() issue was to create a table to store the mimetype array. I've also seen that there is work for a file.module, it could even provide a interface to add additional mimetypes. Or a contrib module could do it.
Some advantages such a table would imho have:
- Better handling of large amount of entries, does need less memory
- Doesn't need to be cached/saved in the variables table as a big blob
- Possibilites to manage the mime types and create a UI to manage the mimetypes
- We could reference to that table, instead of storing the mimetype for each file entry
Comment #32
mfb@Berdir: I don't have anything against a table, but note this patch already gets rid of the variable.
Comment #33
Berdir@mfb
Oh, I see, I havent really looked at the patch before. A table might be a bit more flexible, but this is easier and don't need a update function and so on. I'm happy as long as we don't have to load the whole array from the cache on each request.
Comment #34
mfbchasing HEAD
Comment #35
BerdirI really want this in, simply because it will cut the hook_variable_info() patch in half :)
I'd suggest adding the explanation in #29 to those tests. Just from looking at the tests, I'ts not obvious why substr() is needed there.
Other than that, it looks good to me.
Comment #36
mfbOK, I added code comments for the substr() calls.
Comment #37
BerdirOk, another small point.
AFAIK, the assert messages at the end should all be translated, but some of these aren't...
Comment #38
mfbThanks. Looks like upload.test needs further cleanup in this area, but that's out of scope for this patch.
Comment #39
BerdirAgreed.
Ok, patch looks fine to me, All tests pass and there are tests for the new features. This makes it easier to extend these mimetypes for other modules. Also, this removes a big part of the hook_variable_info() patch (and cache) and therefor simplifies that patch severly.
This is imho RTBC.
Comment #40
chx CreditAttribution: chx commentedI do not get or agree with the array_reverse. You are altering, so alter the array instead of appending.
Comment #41
mfb@chx: the rationale for array_reverse() is that it allows for slightly simpler code on the contrib module side, because you can append to the array rather than prepending.
Without the array_reverse, if you wanted to map 'ogg' => 'audio/ogg', you would need to prepend it to the array -- i.e.
$mapping = array('ogg' => 'audio/ogg') + $mapping;
-- otherwise the built-in 'ogg|ogx' => 'application/ogg' would have precedence. Also, if you altered one of the mappings and the built-in set of extensions for the mapping was later modified, then your alteration would end up being appended to the array and have no effect.For now I left the array_reverse() but I did need to re-roll.
Comment #43
mfbchasing HEAD
Comment #45
catchSee also #473652: file_get_mimetype() unneeded loop.
Comment #46
mfbThe main thrust of the original feature request has already been resolved by file.mimetypes.inc :) However, I rerolled all the extra bits in this patch, including the ability for contrib modules to alter the mimetype mapping and some extra tests.
Comment #47
mfbFixed a typo in function documentation.
Comment #48
mfbComment #50
mfbComment #51
pwolanin CreditAttribution: pwolanin commentedin function file_get_mimetype() you load the mapping and call drupal_alter() on each function invocation?
Comment #52
mfb@pwolanin: Yes. Were you suggesting a static cache of the default mapping? If so see comment #22 and #23.
I'm adding a Performance tag as I think there's already a performance boost (if a module needs to alter the default mapping) in not storing a huge mime_extension_mapping array in the variable table.
Comment #53
pwolanin CreditAttribution: pwolanin commented@mfb - if this is only used at upload time, then I'd agree it's fine.
Comment #54
BerdirIt is used for every request, for the favicon... http://api.drupal.org/api/function/template_preprocess_page/7
But it might make sense to remove that check and only do it when the favicon is changed.
Comment #55
mfb@Berdir: Hmm, interesting :( Including file.mimetypes.inc every time we render a page is not ideal. Yes this should be cached the same way we cache an uploaded file's mime type.
Comment #56
mfbOK this patch adds another performance optimization, replacing the call to file_get_mimetype() on every page load with theme_get_setting('favicon_mimetype'). The favicon_mimetype is set whenever a custom favicon is configured.
Comment #57
mfbMoved a few more lines of logic from page preprocess to theme settings submit.
Comment #58
mfbOops the merge conflict in file.test needed more fixups than I thought, hopefully passes now.
Comment #59
mfbComment #60
aaron CreditAttribution: aaron commentedsubscribe
Comment #62
mfbChasing HEAD.
Comment #63
aaron CreditAttribution: aaron commentedNo, no, no. We don't want people to have to manually enter the favicon MIME type. Look at #531476: Add favicon mimetype theme setting first, which would take care of that. It's RTBC as soon as someone can give it a look-over.
Comment #64
mfb@aaron, there is nothing manual in this patch about favicon MIME type, it's automatic. Can you explain what's wrong with this patch? I do think it's a good idea to move it to separate issue since it's not directly related to altering the mapping.
Comment #65
aaron CreditAttribution: aaron commentedAh, misread the patch. Yeah, that looks okay. Though I think using getimagesize info for favicons would be better anyway. Mine's just a different approach, and we could redo that patch afterwards.
Comment #66
aaron CreditAttribution: aaron commentedHow about you remove the favicon MIME type out of this patch, since it's not really related, and we can give it a proper read, looking at the different options (drewish and i came up with two alternates at the other issue already).
Comment #67
aaron CreditAttribution: aaron commentedFor instance, I'm pretty sure (without testing) that this will end up with no favicon MIME type on maintenance pages, despite best intentions, as there's no access to the db. That's the benefit of using variable_get; we could set that in $conf in settings.php.
Comment #68
mfbWell my patch uses theme_get_setting() which is used elsewhere in theme.maintenance.inc. Also it sets a default value for favicon_mimetype which should work fine if the theme setting doesn't exist. Anyways I've removed the favicon stuff from this patch so let's address it at #531476: Add favicon mimetype theme setting.
Comment #69
aaron CreditAttribution: aaron commentedThis looks really good to me now. You should probably put something in the changelog.txt about it, then I'd say it's RTBC. (Of course, we'll also need to update upgrading dox to note the change from the variable_get.)
Comment #70
mfbadded entry to changelog and documentation for the new hook to system.api.php
Comment #71
aaron CreditAttribution: aaron commentedawesome work! looks rtbc to me.
Comment #72
drewish CreditAttribution: drewish commentedI'd love to avoid changing these lines:
since it's not in the scope of the issue and there's still discussion about using t() in assertion messages—translators hate them since they clutter things up and anyone doing core dev work has to be able to read english.
Comment #73
BerdirYeah, I wrote in #331171-37: Allow modules to alter the MIME extension mapping rather than setting a huge variable that they should be translated, but in the meantime, the remove translation from tests issue has landed, so these should be removed again.
Comment #74
mfbOK, removed the t() calls from *.test
Comment #76
mfbChasing HEAD
Comment #78
mfbChasing HEAD
Comment #79
aaron CreditAttribution: aaron commentedi don't think there's anything else holding this up now.
Comment #80
pwolanin CreditAttribution: pwolanin commentedthis doesn't make sense to me to have in the stream wrapper:
Why would we call drupal_alter on it every time it's used, instead of just the first time?
Comment #81
pwolanin CreditAttribution: pwolanin commentedsomething like this.
Comment #82
drewish CreditAttribution: drewish commentedthat makes way too much sense. mfb any objections?
Comment #83
mfbIt looks good to me. This function is typically only called when file(s) are uploaded so the memory-hogging static variable shouldn't be a problem, and does speed things up (if anyone is actually benchmarking multiple-file uploads).
Comment #84
aaron CreditAttribution: aaron commentedAs we introduce a new hook_file_mimetype_mapping_alter(), I wonder if we should document that in system.api.php? Otherwise, looks great to me.
This review is powered by Dreditor.
Comment #85
pwolanin CreditAttribution: pwolanin commentedyes it should be added. New patch.
Comment #86
aaron CreditAttribution: aaron commentedI was being blind, Peter, sorry about that... :P The patch at #81 has that. Thus, #81 is RTBC, and #85 now has a duplicate API description (which would probably break the API module -- should an issue be created to test that API hook_functions aren't duplicated?).
Comment #87
pwolanin CreditAttribution: pwolanin commentedhmm, I wrote more extensive doxygen, so I merged my doxygen with the existing code example for the hook.
Comment #88
mfbLooks good.
Comment #89
pwolanin CreditAttribution: pwolanin commentedtagging
Comment #90
Dries CreditAttribution: Dries commentedReviewed this patch and it looks good. Committed to CVS HEAD. Thanks mfb, pwolanin, drewish et al. Good work.
Comment #92
mattgilbert CreditAttribution: mattgilbert commentedIs there a Drupal 6 version?
Comment #93
mattgilbert CreditAttribution: mattgilbert commentedIs there a Drupal 6 version?
Comment #94
mfb@mattgilbert: In Drupal 6, http://drupal.org/project/filemime module provides a UI to set the mime_extension_mapping variable to whatever you want (can also read in a mime.types file from the server file system).
Comment #95
jhodgdonWas this change announced on the 6/7 module update guide? If it was, or if it didn't need to be, please revert status/tag change.
Comment #96
jhodgdonI couldn't find anything about MIME types on the 6/7 module page, so I added this:
http://drupal.org/update/modules/6/7#mime-types
If someone wants to review, that could be useful -- I based it on the patch in #87, which appears to be the only one that was committed. I think it's OK...
Comment #97
mfbLooks fine. Technically speaking, the mime type mapping couldn't really be altered in Drupal 6, it could just be replaced (the default mapping is in code as the variable's default value). So this issue added the ability to alter the mapping rather than just replace it.
Comment #98
jhodgdonRight. I figured most modules would have done a variable_get(), changed the array, and then done a variable_set() probably, which is kind of a "poor man's alter".