Problem
Currently features doesn't have a notion of module provided defaults (that may be overridden by features). This has lead to a lot of confusion in the file_entity and media issue queue.
There is a workaround to deal with that situation athough it does not work in all cases (see #2192391: Default file entities are not exportable by features (Sibling Issue) for part of the history):
Modules save their defaults to the database in hook_install(). Feature exports then work, but the features are initially overridden. Also if a module wants to override the defaults provided by another (ie. media overriding file_entity) they might override user-made changes in the database made between the two module installs.
Proposed solution
Ideally precedence would look something like this (later items override earlier items):
- (non-feature) module provided defaults.
- Features.
- User modifications.
Any solution that deals with this problem has to make (at least) the distinction between those 3 - otherwise it will fail in any scenario where a distinction it does not make becomes relevant.
Current status
The patch provided in #23 does two things:
- It makes sure the non-features implementations of a ctools api are called before their feature counterparts. Thus ensuring that features always override non-feature modules. This is done using
hook_ctools_plugin_api_alter(). - The definition of a conflict is changed slightly: Previously a component was in conflict if there was another module defining the same data. With the patch it's only a conflict if the offending component is also declared in the module's .info file. In essence features only conflict with other features.
| Comment | File | Size | Author |
|---|---|---|---|
| #57 | 2446485-57-module-defaults-vs-features.patch | 6.12 KB | torotil |
Comments
Comment #1
torotil commentedI can confirm that simply increasing the module-weight has the desired effect.
Comment #2
torotil commentedComment #3
hefox commentedComment #4
torotil commentedComment #5
torotil commentedSorry for the noise. I've rephrased the issue to be more open to different ways of solving this.
Comment #6
torotil commentedHere is a patch that changes the way how the components in the features interface are listed. It now only hides components that are provided by other enabled features.
This means that if a component is provided as a module default (without using features) it still can be changed and exported as a feature.
This is still only a partial solution, but I need feedback on whether that is an option to solve this.
Comment #7
steinmb commentedBumping this issue. Coming from media and file_entity issues trying to work around this. Did non of the feature maintainers come back to you on this?
Comment #8
torotil commentedThanks for pinging this issue. I took another look at the approach taken in #2192391: Default file entities are not exportable by features (Sibling Issue) and was able to implement a generic solution for all ctools export based feature components. This should be able to replace the patch from the file_entity issue completely.
Comment #10
torotil commentedHere is a re-roll against the latest 7.x-2.x - my local checkout was quite old I guess.
Comment #11
torotil commentedComment #13
torotil commentedTrying again as it failed due to a ominous "CI error".
Comment #15
torotil commentedAh, yes. PHP 5.3 compatibility.
Comment #16
torotil commentedChanged the summary to make the problem more obvious and describe the current state of affairs.
Comment #17
torotil commented@steinmb: So far there is no reaction from feature maintainers - none at all.
I'm also setting this to be a bug-report again as it is actually reveals a hole in features design (at least wrt ctools exportables) that leads to features not being able to handle it's most basic use-case properly.
Comment #18
torotil commentedFix a notice for some schemas not defining the apis-subarray.
Comment #19
adr_p commentedTried this patch on 2.10. Default file types (image, video etc.) showed up in the components available for export, but after exporting the feature I don't see any code being generated.
Comment #20
mikeoharaConfirmed. Tried to export features with the media image preview settings, and there was no code exported after applying this patch.
Comment #21
steinmb commented@torotil something the testers missed or does this needs work?
Comment #22
torotil commentedI guess I need to retest this with the current versions of file_entity and media. Something seems to have changed. So for now this is "needs work".
Thanks for testing!
Comment #23
torotil commentedHere is another attempt. This changes the definition of a conflict also in
ctools_component_features_export().Comment #24
torotil commentedPlease try out the new patch!
Comment #26
torotil commentedUps - the new array syntax has slipped in. Here is a PHP 5.3 compatible version.
Comment #27
alisonTried patch, it might work, but I'm getting a slew of messages on admin pages :( I even updated from features 7.x-2.10 to -2.x-dev, but the messages still happen. Here's a sampling:
(they repeat a zillion times on the screen)
(when I'm on a page to recreate a feature, I get both of the above messages -- when I'm on the create-new-feature form, I just get the Undefined index message a whole bunch of times; I also get the Undefined index message on other admin pages, though just once; the errors did not change when I switched Features module version, other than the slight line number differences)
About my env:
Comment #28
torotil commentedYes there are some schemas that are ctools exportable but don't have this information defined. Even after studying the code I can't say when this is the case. For now I would deal with it by simply checking for it.
Thanks again for testing! This patch should remove the notices mentioned in #27.
Comment #29
daniel korteI couldn't get this to work with the media module (Related issue #2104193: Default file entities are not exportable by features (Media File Entity Overridden)). The file displays would not export.
I updated the patch to address the Warning in #27 by checking for both strings and arrays. For me, the warnings appeared with module dependencies which were always strings instead of arrays. That could possibly be fixed elsewhere though.
Comment #30
torotil commentedDaniel thanks for testing. In this case the code you've modified already expected the correct data structure - the data-structure delivered by
features_get_conflict_map()was wrong. Here is a patch fixing the the actual failure instead of hiding the error messages.Comment #31
torotil commented… and another one to fix a logic error.
Comment #32
alisonThanks, all! Unfortunately I'm still getting an error repeated a whole bunch on my "recreate" screen :(
Warning: Illegal offset type in ctools_component_features_export() (line 167 of /home/alison/workspace/d7/itc-sitedata/sites/all/modules/contrib/features/includes/features.ctools.inc).(I did go up to -dev again, like the last time I tried it out.)
Is there anything I can do or any info I can share to help troubleshoot? Thank you!
Comment #33
jenlamptonIt's working for me, file entities are exporting correctly now with the -dev version + the patch in #31. I'm not seeing the issue reported in #32.
Comment #34
jenlampton...Hesitantly changing to RTBC, but I think this still needs more testing.
Comment #35
brockfanning commentedI think it's close, but the problem in #32 is still there, and it's definitely not just a harmless warning - it's actually affecting the logic. In line 167 of features.ctools.inc the $other_feature variable is an array, but is being treated like a string. I can also recreate the warnings on the command line by doing something like a "drush features-diff my_feature", and it causes the features-diff to think things had changed that hadn't.
I'm not familiar enough with the code to know what $other_feature should be reasonably expected to be, but this could be maybe fixed by either doing an array_shift() on $other_feature to get the first element, or using a foreach to loop through the array.
Comment #36
torotil commentedAh yes. That happens when function signatures are not properly documented. Is this better @brockfanning ?
Comment #37
brockfanning commentedLooks better, but this time running a "drush fd my_feature" it's failing with: Fatal error: Function name must be a string in /var/www/drupalvm/sites/all/modules/contrib/features/features.export.inc on line 868
Comment #38
torotil commentedCould you just delete the
()in this line and try again? I really don't know how those crept in.Comment #39
brockfanning commentedSure thing, that solved all the PHP issues. I ended up with strange behavior though - I tested it on a random Features module on my site, and when I did a "drush fd" it gave output implying that a lot of new variable (strongarm) components needed to be added to the Feature. They were all variables related to random node types, such as these variables for the "book" type:
* field_bundle_settings_node__book
* language_content_type_book
* menu_options_book
* menu_parent_book
* node_options_book
* node_preview_book
* node_submitted_book
And I do have a separate Feature related to the book type that does already have those variables exported to it. When I did a "drush fu" for first Features module, it did actually add all of the book variables to the Feature. So the end result is I've got 2 enabled Features with the some of the same strongarm components exported to them both.
Comment #40
brockfanning commentedI should mention I'm testing with Features 2.10.
Comment #41
torotil commentedAnother try :)
I was able to reproduce the issues mentioned in #37 and #39 and both are fixed for me with the new patch.
Comment #42
torotil commentedComment #44
torotil commentedComment #45
brockfanning commentedWith #41 I was able to export a file type to a Feature, which is something I haven't been able to do before. I tested the drush commands: fl, fu, fr, and fe, and all worked as expected. So nice work, one successful test here. I hope some other folks can test also.
Comment #46
torotil commentedThanks for testing and your responsiveness!
Comment #47
brockfanning commentedAn update after a bit more testing: I've had to remove this patch because of some issues, though I'm not totally sure what in the patch might cause them. @torotil, hopefully this makes sense to you. Here's what I do to recreate the problem:
* Use drush to do a full cache clear, "drush cc all"
* Visit a page that renders a file entity
This results in fatal PHP errors, because the file entity does not have a "type" property. The best I can tell is that drupal_get_schema is running at a point early in the bootstrap, before file_entity_schema_alter has added "type" to the file_managed schema. This causes a database query to load the file entity without the "type" column, which results in a file entity without a type, which triggers the PHP error.
Once drupal_get_schema is properly cached, it's not a problem. So the first page load after a full cache clear is really where the problem occurs.
Again though, I have no idea why this patch would cause something like that. But sure enough, if I remove the patch, the problem goes away.
Comment #48
torotil commentedWow that's interesting. Could you try to get a full backtrace for when the fatal error happens? That would tell us which code path is taken to get this early call drupal_get_schema().
Comment #49
joseph.olstadComment #50
torotil commentedI'm unable to reproduce #47. I did:
According to #47 this should be enough to reproduce the fatal error.
Is someone else able to reproduce this?
Comment #51
torotil commentedI have this patch in use on a few dozen sites since around a year now. I’ve not experienced any fatal errors as described in #47.
I’m setting this back to needs review.
Comment #52
joseph.olstadAnyone else using this patch?
Comment #53
jenlamptonI'm still using the patch in #41 on Features 7.x-2.11
Comment #54
izmeez commentedAttached is a re-roll of the patch in #41 against the latest features-7.x-2.x-dev 2019-10-20
Comment #55
torotil commentedUpdating to the re-roll because of the recent 2.12 release. Still using this patch in 2020.
Comment #56
donquixote commentedHi
I have not studied this issue until now, and need some help to get started with it.
A starting point could be to improve the docs:
- Improve the issue summary, by adding an example.
- Improve the docs in the patch!
The doc might already have the info I need, but it looks a bit awkward.
Why does a @param doc start with "Get the ..."?
"An associative keyed by" probably should be "An associative array keyed by".
I get the impression that this function behaves equally to
features_get_component_map(), except:- Always a specific component, the parameter cannot be NULL.
- Only return enabled modules.
This should be reflected in the doc comment and in the function name.
Btw the doc on
features_get_component_map()contains an interesting@returndoc. We could do something similar here, e.g.:Not sure if people other than me like this.
It would be faster to fetch module_list() only once, and then use array_intersect().
The list from module_list() has the format $[$module] = $module, so we can use keys or values likewise.
I never really liked the naming patterns in features.
What is a "$component", "$component_type" or just "$type"?
One could think that the names should be used like
$component_type = 'field_instance'; $component = 'body';.But in most places, they are used like
$component = 'field_instance'; $identifier = 'body';, or sometimes$name = 'body';.We should follow the most common conventions for variable names that we find elsewhere in features.
Comment #57
torotil commentedHere is a new version based on your suggestion. I have changed:
features_get_conflict_map()intofeatures_get_claimed_components()and updated its docs.$GLOBALS['features_ignore_conflicts']to the calling functions to make it’s signature simpler.$keyto be consistent withfeatures_get_components_mapI don’t think so: Since
array_intersect()has to work with unsorted arrays the call is O(n), where n is the number of enabled modules. With the current implementation it’s O(log(n)) (lookup in a hash) even if it’s nested in two function calls (module_list()has a static cache). We could do an explicit foreach instead ofarray_filterto have the best of both, but this is more code and seems like a premature optimization.I’d argue that features doesn’t have a consistent naming scheme (like most of the Drupal universe) due to the lack of an architecture documentation (list of concepts and their name). Introducing this and making the naming consistent is a big undertaking in itself and perhaps not something that we should do as part of this issue.
Let’s hope the tests still pass. 🤞
Comment #58
lwalley commentedI can confirm with patch from #57 I am able to export and take ownership of "file_type" and "file_display" components, with default hooks in place i.e. without needing file_entity patch in #2192391-83: Default file entities are not exportable by features (Sibling Issue) or media patch #2104193-113: Default file entities are not exportable by features (Media File Entity Overridden).
We are exclusively using
drushto manage features so I have not tested UI.One quirk to note is that if there is an alter hook that appends data (e.g. hook_file_default_types_alter() is used to append a mime type to the array), it will be added again on each feature export so you'll end up with duplicate entries. I've noticed this before in features so I don't think it is caused by the patch in #57 just something to watch out for.
Another minor weirdness is when running
drush fda -yit will sometimes list our custom feature with media exports in it but just says it's in the default state; the expected behavior is that if there is no diff the custom feature would not be listed at all.I have not had time to uncover the cause of either if these minor issues, and do not consider them release blockers.
Comment #61
joseph.olstadPushed this into dev , this also permits closing of an issue for file_entity
#2192391: Default file entities are not exportable by features (Sibling Issue)
Comment #62
torotil commentedHi! Thanks for taking the time to merge the patch. I hope it is still of use to some even though the Drupal 7 end-of-life is now only months away. At least it’s a nice gesture.
Issues like this are the reason we’ve decided to move away from Drupal as a company years ago. It took 5 years for a maintainer of a module used on 250k+ sites to review a rather fundamental issue even though it has lead to workarounds in other popular modules (file_entity/media) already.
Comment #63
donquixote commentedSorry I did not follow up on this since the messages from 2020.
And thanks to @joseph.olstad for picking it up again.
For the record, this got pushed to Jan 2025.
https://www.drupal.org/about/drupal-7/end-of-life
EDIT: I think I was waiting for a 3rd party to rtbc, and then did not check on it again.
Comment #64
joseph.olstadhttps://www.drupal.org/project/features/releases/7.x-2.15