Hello,
We are currently "stuck" in 2.19 and wanting to go to 2.26 to resolve security conflicts. We have had no luck in doing an upgrade because the features override steps described here are not working for us as we are interpreting them:
https://www.drupal.org/node/1313910
Here are the issues we are running into:
When creating a feature override generated module, we allow it to auto-detect dependencies, and we proceed to check ALL of the feature overrides to create one super override bundle module. When we turn that exported module on, we are surprised to see that there are conflicts showing straightaway, for example:
Conflicts with: oa_config (Enabled) in features_override_items [filter.panopoly_wysiwyg_text, wysiwyg.panopoly_wysiwyg_text], oa_media (Enabled) in features_override_items [views_view.media_default], oa_panopoly_users (Enabled) in features_override_items [page_manager_handlers.pm_existing_pages_user_login_panel_context, panelizer_defaults.user:user:default], oa_worktracker_duedate (Enabled) in features_override_items [field_group.group_oa_worktracker_task_info|comment|comment_node_oa_worktracker_task|form, field_group.group_oa_worktracker_task_info|node|oa_worktracker_task|default, field_group.group_oa_worktracker_task_info|node|oa_worktracker_task|form, panelizer_defaults.node:oa_section:oa_section_worktracker, variable.field_bundle_settings_node__oa_worktracker_task], oa_archive (Enabled) in features_override_items [flag.trash, views_view.trash_flag_bin], oa_search (Enabled) in features_override_items [variable.search_active_modules]
Then, when we try to disable the exported module we just enabled, all heck breaks loose and the site is essentially broken. The modules page won't load at all, and there are errors all over /admin/structure/features. The only thing I CAN do at that point is re-enable my module with drush, and then the site comes back. This is problematic because part of the steps to upgrade are to make sure your module is OFF and then to revert all features to stock so you are ready to upgrade. If I can't ever turn a module like that OFF once it's ON, I'm in a bind for sure!
We made a hypothesis that this inability to turn the module OFF was because of the dependencies we included when generating the module. Perhaps, features is turning off a bunch of other modules that are needed when we disable our module? So we tried generating the module with NO dependencies to test that theory. The result of that though is a module that breaks Drupal the second you drop the files into sites/all/modules. We didn't even have to turn it on, and the site was throwing errors everywhere.
So, we are just totally stuck on this. We need to get to 2.26, but we cannot afford to manually recreate every customization we've made to views and panels and such. And we need to be able to update to a newer version of OA quickly to react to any security vulnerabilities that could be announced on any given day.
Does anyone have any advice on what we are doing wrong? If so it would be greatly appreciated. I would even be happy to talk to folks on the phone, so if you reach out to me via the contact capabilities on drupal.org, I'm happy to respond and setup a phone call (be sure to note in this post too that you are contacting me, in case your note is caught in SPAM).
If we can figure this out, we'll feel a lot better about doing more Open Atrium sites, but where we stand right now, we are regretting the decision not to just use stock Drupal 7 where we can quickly use drush to update a site. Perhaps with your help, we can resolve this.
Thanks!
Comments
Comment #1
Argus commented2.19 is a long time ago. Some very serious updates have passed since then. You indeed need to update!
That is not the way to do it. You need to break down your overrides into sane parts. Check how Open Atrium uses Features. For instance: bundle the overrides by content type, permissions, layouts, variables etc.
Comment #2
strings6 commented@Argus:
OK, I can see how that goes. That does sound more manageable in theory. I have some questions about that though:
1). I'm curious, when you make features out of a smaller subset of customizations, in practice, do you add those generated modules to your site and enable them? And if so, do you revert to stock settings after the feature is generated, so that the feature module itself puts those changes in place OR do you leave them as pure database changes and leave the generated featured modules out of the picture until upgrade time?
2). Another question: Do you group things by themes, like "worktracker_edits", and then would include the changes to views, panels, permissions, etc. that all relate to worktracker in that feature? Or do your bundles look more like, "All panels changes", "All permissions changes", "All views changes", etc?
3). Last question: Do you actually need to make features to cover ALL of the overrides, or will there be some overrides you ignore? It seems like there are SO many overrides in the list, that I'm actually wondering if my goal is to make the overrides page show ZERO overrides once all my generated feature modules are enabled, or if that isn't going to happen. Remember that one of my issues was that even when I made one "super override bundle" feature and enabled it, that my overrides pages STILL showed some overrides existing, as well as a list of "conflicts".
Comment #3
Argus commentedI guess best practice would be to create Feature Override for each OA Feature you have changed. On top of that, if you added custom content types, some new views or panels pages you could save those as a new Feature. You then install those Feature Overrides and Features as modules (you can put them in a subdirectory of
/sites/modules). After that you can (and should) safely revert all native OA Features.All this is explained in the webinar "Best Practices For Updating Your Open Atrium Site", from 14:11 and further. There is also a documentation page on the subject of updating.
Comment #4
strings6 commentedI'm starting to get a better understanding of how feature overrides should be done in theory. The video was helpful, and the documentation has been helpful as well. But I'm still having difficulty when it comes down to making the feature override modules. When I click on "Create a Feature" I see a list of 187 Feature Overrides, and 553 or so below that under the advanced Feature Overrides. Focusing just on the 187 in the first section, I see a number of things in there, but I do not see everything that is listed when I delve into a particular "Overridden" link from the Open Atrium Features list.
For example, our site shows that "Open Atrium Config" is Overridden, and going into that I see the following Overridden things:
advagg_core_groups advagg_ie_css_selector_limiteradvagg_ie_css_selector_limiter_value chosen_jquery_selectorchosen_minimum chosen_minimum_multiplechosen_minimum_single file_default_schemefile_entity_file_upload_wizard_skip_schemeog_context_negotiation_group_contextog_context_providers_weight_group_context pathauto_reduce_asciipreprocess_cssLooking then at the available Feature Overrides on the "Create a Feature" screen, I don't see anything like many of these (for example, the string advagg does not appear). It seems like it would be easier to featurize an entire View or Panel that I know I've modified, but my understanding is that would result in a very high likelihood that I would not get many new features when I update Open Atrium, and that I should instead only featurize the exact Overridden fields and things that have actually changed.
Here is the full list of Features that say they are "Overridden":
Open Atrium Package:
------
Open Atrium Appearance
Open Atrium Buttons
Open Atrium Config
Open Atrium Core
Open Atrium Discussion
Open Atrium Events
Open Atrium Home
Open Atrium Layouts
Open Atrium Media
Open Atrium News
Open Atrium Panopoly Users
Open Atrium Permissions
Open Atrium Search
Open Atrium Sections
Open Atrium Subgroup Intergration for Spaces
Open Atrium Tour Defaults
Open Atrium Users
Open Atrium Wiki
Open Atrium Work Tracker (this actually says "Needs Review" instead of "Overridden")
Open Atrium Work Tracker - Due Date
Panels Package:
------
Command Buttons
Panopoly Package:
------
Panopoly Core
Panopoly Search
Panopoly Users
Panopoly Widgets
Panopoly WYSIWYG
I'm looking for advice on how to logically create feature override modules for these. Any advice is much appreciated.
Thanks.
Comment #5
Argus commentedIt sometimes takes a lot of searching and trial-and-error to find the right override for a particular change made. It seems most logical and manageable to create a separate Feature Override for each of the OA Features. Start with a simple override like oa_home, check if all your changes are saved in it after reverting. Document all changes you cannot find and change them back manually after updating. In the end that may be the shortest route, although not pretty.
Comment #6
strings6 commentedDo you think that things went badly when we tried to make just one giant override module, because when we turned that beast of a module off, there was just no way it was going to turn everything off in the right order to avoid errors?
I'm wondering this, because I'm trying to decide what the more efficient way to generate these override modules would be without so much pain. Is it the goal to have all 187 overrides captured into feature override modules, or will there be a few (or many) unused/ignored?
If the goal is to get all 187 of them into an override module, then in theory, one might not have to fully know where each of them goes within the Open Atrium features (let's face it, trying to figure THAT out at this point is a nightmare). Perhaps I could just create smaller override modules, grouping like kind things together that appear not to step on each other. For example, what if one feature override was ALL the permissions changes? And what if one was all the tweaks with the word "worktracker" in them? Could that work out OK in the end, so long as I cover them all?
On the flip side, if there are many feature_overrides within the 187 showing on our site that should be ignored, and would actually cause problems if added to a feature override module, then my idea would not work. If that were the case, I would have to do the trial and error method. Being completely honest though, this would be a huge reason not to use OA for me in the future. I want quick updates that don't take hours of time, especially when something updates as often as OA.
What do you think?
Mike Potter said in his video:
In theory, the features override module can solve the problem. But in practical use, it is turning out to be so difficult to use, that it's just a different way to spend many hours of time. Like the many hours I would spend doing manual overrides without it.
Comment #7
Argus commentedThere will be many unignored. That list just displays all possible overrides, whether changes are made or not. With some work it should be possible to track most changes you made, especially when they are changed panels, panes and pages. Permissions are more difficult to track down, I also have had problems with them, then again these are rarely changed by an update, so it should be safe to leave them unreverted.
Comment #8
strings6 commentedOh, I see. So my actual overrides might actually be a very small subset of those shown in that first "Features Override" section on the "Create a Feature" page?
I have some questions to help clarify what things I have to consider for feature overrides. Can you let me know if the assumptions below are correct?
I should make feature overrides if:
- I edited a view that comes with OA
- I edited a panel that comes with OA
- I edited a content type that comes with OA
I should NOT need to worry about:
- NEW views I added
- NEW content types I added
- Group or nodes I have created
Uncertain (where does these fall?)
- If I use the "Customize this Page" button (Panelizer I think) to add content, such as a view, to a specific page?
- If I changed the status names (added or removed any too) for the Work Tracker?
- What about settings that come set a certain way and I change the value?
- What about settings that I change from empty to some other value, like what the home page is set as?
- What if I created a NEW panel variation?
Thanks for sticking with me so far.
Comment #9
potassiumchloride commentedYou are not alone. After weeks of trying to follow the video and documentation guidance and so many hours of work, we have just bit the bullet and done a straight update without Features Overrides. It was simply too time-consuming and we never ever found a combination of overrides that would allow us to proceed with the update while keeping our customizations. What a giant mess. Now we are in the process of re-creating all of those customizations and trying to keep track of them in one list so that we can try to figure out how to use Features Overrides in the future.
Comment #10
strings6 commented@KCl:
I'm sorry to hear about your troubles with it too. While it can be comforting to know that one is not alone, I was kind of hoping it was "just me" and that the magical key to unlock the secrets of how to do features overrides the right way would eventually come to me! Based on what I'm experiencing though, it does not surprise me that others have trouble with it as well.
@OA maintainers
I just keep having this thought, and I figure I might as well share it here in case it sparks some great new development (or at the very least, someone can explain why it would not be possible). See, I have this idea...
I keep staring at two types of pages in OA. The first page is the one where I previously looked at the list of OA Features and saw that some are showing "Default" status, while others say "Overridden", and I have then clicked on the word "Overridden" for one of the overridden features. So visualize me staring at that type of page where it's showing me the exact "things" that have been overridden. My thought as I view this page is, "It knows. It knows better than I do what has been overridden. Hmmm."
The second page that I stare at is the "Create a Feature" page. We know there is a section on this page showing all the possible feature overrides. Here lies the daunting task of checking the right things... Then I think to myself again, "But it knows..." And then I have this idea:
Can't this be automated somehow? On page #1, I see that Open Atrium is aware of all the overrides. But on page #2 Open Atrium is making me select the feature overrides to include in a feature override module. Why should I have to do this? What if on page #1 there was a link or button that I could just click to generate a feature override module from my customizations? I'm not talking about the "Recreate" link, but rather a new link to make a feature override module out of just the things I changed.
Is this doable? If it was, it would take the step where the dragons lie, and make it simple for us (in theory). Thoughts? Reasons why this couldn't be done?
Thanks!
Comment #11
Argus commentedThe ability to save configuration in code is a recognised Drupal shortcoming, being addressed in Drupal 8. Features wasn't designed to be a configuration management tool, but "only" a way to export bundled functionality. For now Open Atrium uses Feature Overrides, and as it happens to be mpotter is the maintainer and lead developer of both, so the problem is in good hands...
For now in Open Atrium we have to manage with Feature Overrides, this will at least be for the next 2? (guestimated) years. Any feature requests for improvements to Feature Overrides should be posted in the Feature Overrides module's issue queue imho. You can point to this issue.
Let's keep this issue about the problems with Feature Overrides in Open Atrium, and solutions or at least best practices to deal with them. Documentation about it can be improved, this issue is a valuable source for that :-) Back on the issue then;
Best way to find out is simply try it. Settings often need Strongarm when they are variables. They can also be found as one of the Components on the Create Feature (
admin/structure/features/create)page.Comment #12
strings6 commentedHi Argus,
I see your point about keeping this issue focused. I will say though, being the original poster of this post, I am ok with any discussion that leads to a better experience when updating an OA site. I do not view my previous post as "off-topic" in any way, because I am simply trying to talk about the problems I am having with the features override approach, and talking about new ideas in this post for how to tackle the problem is fine by me. I do not want to limit people to talk only about how to make features override work for them as it is now. If there are better ideas out there, I say let's share them! My 10 cents on that matter.
Also, I agree with you about the capability and skill of Mike Potter. I should probably mention, that I in no way would want to suggest otherwise. When I share the issues I am having with anything Open Atrium, I am not in any way trying to say that anyone is incapable. On the contrary, I hope that Mike is following this topic, and the thought that something I say here might help him or anyone else, is an exciting thought to me.
---
But yes, back to what we were talking about before... so... Strongarm. I see that Strongarm section on the "Create a Feature" page. And you know what else-- I see some of those odd overrides there that I could not find in the Features Override section, such as the advagg stuff I listed above. So here's the next question in line I guess:
Should we not be so zeroed in on just what checkboxes are under the "Features Overrides" section of the "Create a Feature" page? For example, do you often check things under other sections when creating your Features Override modules?
Thanks!
Comment #13
Argus commentedWhat 'things under other sections' do you mean?
Comment #14
strings6 commentedWhat I mean by that is other checkboxes on the right hand side of the Create a Feature screen, besides the checkboxes under Freature Overrides. Things under Strongarm, or other sections on the right.
Thanks
Comment #15
strings6 commentedAn update on my progress:
I've figured out how to do some of these Feature Overrides, at least some of the easier ones. I've successfully stored my customizations to Features for:
- Open Atrium Appearance
- Open Atrium Command Buttons
- Open Atrium Config
When I enable my override feature modules for those, they no long say Overridden, but instead say Default, so hurray to that!
However, I have attempted a couple more and am getting yellow messages that say "Needs review" for these two:
- Open Atrium Core (Field Instances and Page Manager still says "Overridden" and Strongarm says "Needs review")
- Open Atrium Discussion (OG Permissions says Needs review).
I've looked at them very closely and nothing jumps out at me that I would have missed. What are some reasons that these Open Atrium Features would still say "Needs review" even after I enable my Override module?
Thanks.
Comment #16
Argus commentedFeatures remaining overridden is a known problem: for instance check https://www.drupal.org/node/986932. Even in the default OA installation on Pantheon some Features remain overwritten, e.g. the Worktracker Feature.
Don't be to worried about this. In your installation check manually if you can find any changes that aren't saved, I bet you can't find any. Do keep a log of anything that you change for future reference.
Comment #17
strings6 commentedWhat if a role that comes with OA is removed, and thus all kinds of features are showing as overridden because there is a DELETED section under Permissions and/or OG Permissions? Can a Feature Override be created to handle a deletion like that?
Comment #18
strings6 commentedI should clarify the previous post. I don't think I really deleted any stock roles from OA, but I see things like this in my overrides:
DELETIONS
unset($data['node:oa_group:write oa_discussion_reply to discussion post']);
unset($data['node:oa_space:add attachment to oa_discussion_post']);
unset($data['node:oa_space:create oa_discussion_post content']);
unset($data['node:oa_space:write oa_discussion_reply to discussion post']);
I found something for all 4 of those in the Feature Overrides checkboxes when creating my feature override module for "Open Atrium Discussions", but it said "Needs review" on the original Open Atrium Discussions feature when I enabled my module. What's going on there exactly?
Comment #19
Argus commentedWhy would you want to make big changes like removing a role that comes with OA... ?
That is like stepping into a car and expect you can drive without a steering wheel.
Comment #20
strings6 commentedAgain, I don't think that I deleted any stock roles. I'm not sure what the DELETIONS I posted above are referring to. It might be as simple as me unchecking a checkbox on the permissions page to take a permission away from one of the roles, and then maybe it shows up as some kind of deletion. At any rate, I'm still needing to know how to handle those.
Thanks.
Comment #21
strings6 commentedSomebody that will do an Open Atrium upgrade to newest version, please call me. I'm looking for someone to assist on a "per upgrade, as needed" basis, not a monthly or yearly contract.
Comment #22
Algeron commentedBecause customers don't always want/need more roles than necessary. By not removing them, we're obliging ourselves to configure and test their permissions.
Comment #23
mpotter commentedNot sure if you actually deleted a Role, but the OG Roles of "Member", "Non Member", and "Administrator" are absolutely requires for Open Atrium to work and should not be removed.
The "deletions" mentioned in the Feature Override just means that some options were "unchecked". In HTML checkboxes, an unchecked box has a "null" value, so it turns into an array deletion within Drupal's forms. So it sounds like you maybe just unchecked those permissions which is fine.
Changes to permissions will often cause Feature overrides. Permissions are one of those things in Drupal that just don't work with Features well. But as long as you have your permission changes saved to an override feature module then you should be fine.
Comment #24
mpotter commented