*DRAFT* Proposed product goals for Drupal 8.4/8.5(+) core

There are a few experimental modules that have 8.4.0-beta1 deadlines that aren't listed above (while some are). Is there a particular reason they aren't listed?

I don't really agree with #3.

also: super nasty migration path to rip it out

Ripping out DateTime Range would have exactly the same nasty migration path, and would screw contrib the same as moving Workflow back out. Moving Settings Tray into contrib would be far, far, far less disruptive.

These are just "top focus areas" from the product team, not the only things the community is going to work on. In past minors we listed three focus areas and then the community initiatives; this time, the product team came back with like 10. :P I think it has more to do with the scope of work and investment needed to complete the feature, and how critical it is to the ecosystem. I'm not a product manager, though.

In the slide deck, the last page does say "PHPUnit, DateTime Range, IFE, and all the things" so I'm adding that to the summary, because it should be there in the list too.

I'm a full time dev, and so my views are pretty opinionated here please correct me where I'm wrong, I don't do as much core work as I'd like. But I feel like one of the primary considerations here, for generating the priorities, is adoption, and that's fine. But do folks really believe that migrate ui is really what's killing adoption?

I think adoption is down because of rewriting half the framework. Drupal 8 is one of the most powerful, and COMPLICATED cms's out there (or it can be), and we made it more complicated (which as a dev I agree with), and that being said, migrate ui isn't slowing most folks down.

To me as a developer here is my experience with writing 3 sites now in d8 here are the issues I've run into and have been a pain. We're moving AWAY from drupal because we're struggling with these issues and it's become easier to write a laravel/node app.

1. The project config_install, should be part of core. Having to install an contrib profile to install a site using core config management seems to be a counter intuitive step.
1a. A 'nice to have' would be config_split. This makes more sense as a contrib module so it's would be nice to have.
2. Path auto needs work. When I can't change taxonomy paths from the taxonomy item, and we have aliases go missing for a module, which is *basically* part of core or used on every site that's bad.
3. A stable migrate path from media to the core. It seems as if this is a priority so I would keep it on there. We are VERY worried that sites built using media_entity are going to become very difficult to upgrade later on.
4. Paragraphs vs layouts. I'm a bit behind on the subtle differences, I know there was a blog post about this the other day, but we LOVE paragraphs. It solves a lot of problems and can make for very easy admin interfaces. That being said, there is an issue a few months old where we can't delete paragraphs. I had to resort to leaving orphaned data because a bug fix hasn't been figured out yet.
5. EntityQueries and the EntityTypeManager seem to be very buggy and results can't be trusted, consider https://www.drupal.org/node/2759757 . It's been open 8 months. As far as I can tell, I still can't use full joins on the same entity type without patching.

I'm listing issues that I'VE run into but not asking for these issues to be prioritized, they are just an example of stability issues that I've seen.

I realize that some of these are contrib modules, and such may be out of scope of prioritization, but if it comes down to, feature requests and more front end items vs fixing some of the long standing issues, I vote for the latter. It seems as if there is a focus on the more complex, front end gui work and site builder tools, which I think should come AFTER stability and fixing up some of the issues with highly used contrib modules.

These are suggested product priorities from the product team. I agree there are a lot of critical and major bugs to work on, but your list also includes some features already in the product list, like layouts and media. Generally the product team doesn't set priorities for bugfixing - unless it's bugfixing of not-yet-stable features.

Addressing a couple of these:

But do folks really believe that migrate ui is really what's killing adoption?

A stable migration path is preventing over a million sites from upgrading from Drupal 6 or 7 to Drupal 8. Migrate UI isn't the biggest blocker to migrations since a lot of sites will use drush, but obviously it blocks migrations for people who don't.

The project config_install, should be part of core. Having to install an contrib profile to install a site using core config management seems to be a counter intuitive step.

There's already an issue opened and needing review for this #1613424: [META] Allow a site to be installed from existing configuration, but no-one's updated it for six months. I agree config_install is very convenient when you don't want to sync default configuration and a custom module during initial development.

Ok that makes a ton of sense. Just seeing the doc silo'ed off, without the other conversation the team already had, it just seemed as if these product priorities were taking priority over a lot of stability work. Your response makes me a bit easier with that knowing that they they are not taking a back seat to features.

Adding the plan issue for #2859225: [policy, no patch] Mark Migrate (not Migrate Drupal) as Stable.

@richard.c.allen2386, for specific suggestions of features you want to add to core, you can submit them as new ideas in this queue (the Ideas queue), or for small improvements a core issue. :) This issue isn't to gather feedback on all the great ideas for all the features we could add for core, just the product team's priorities for the next year or so, slightly adjusted based on the release managers' feedback.

Regarding core bugs, yeah, I definitely agree that technical debt needs to be a priority. For me it's the top priority. The release managers have ongoing dialogue with the product team about how we can manage the outstanding bugs while still adding new features that keep contributors engaged in Drupal and meet users' top needs.

Also, this is specifically about what core's goals are feature-wise, so while fixing contrib issues is definitely also important, that's why it's not covered here. The contrib modules mentioned in the roadmap are mentioned only because they are related to a core module that's among the product goals -- for example, Panels, Panelizer, and Display Suite are all supported by a new experimental layout API in core, so to help the ecosystem and those modules, we want to make sure the new core API meets their needs and is stable as soon as possible.

As always, since Drupal is open source and still mostly a do-ocracy with a thin governance candy shell, what gets done is whatever people work on. :) There are a hundred or more commits to core every month, mostly bugfixes, sometimes small new features that would not make this list, and occasionally significant ones that do.

Thanks @richard.c.allen2386 for your feedback!

P.S. -- I tried to put a bullet about critical issues at the top of the list, but that didn't fly. ;) They're still the most important place to contribute though IMO, always.

Improve configuration management, the way it is right now is a major pain for large production enviroments. Various issues have been posted, two examples:
- new custom blocks break when you try to move them from dev to prod.
- if users make changes on prod, config imports will overwrite them.

There are currently only workarounds, at best, like "create the block on prod and then copy the DB to local". But try doing that with a 4GB DB.

I agree with making Migrate systems a high priority and found it pretty difficult to import Wordpress data into Drupal, which I think strategically would be a winning feature to polish up even if most of that is not in core.

Entity Reference Revisions is I think a strong contender to get included in core, after it's hammered out more because it enables more complex versioning to work correctly. When something is really formed from multiple entities, a common D8 pattern, then you can get proper versioning. Paragraphs is dependent on ERR. It would make sense to consider ERR as part of the workflow and perhaps migration areas.

richard.c.allen2386 Re paragraphs deletion problem see #2764681: Allow paragraph type deletion if content exists. I think Paragraphs should probably not be in core, but ERR should be because it is a key API for that module and potentially many others.

re #12 #2730605: Improve the SBX of Configuration in Block Module is opened to address that.

So far there are two mentions of config management improvements in this thread, and I'd like to add a third. CMI in core is currently really stable and awesome, but only solves what I'd say is a 10-20% use case. Mix in Config Filter and Config Split, and I think we'd solve the 80% case. Both of those modules are not yet stable, but if we're going to improve DX / SBX, having a practical Configuration Management tool/methodology (instead of every company writing up their own set of tools and practices) would be nice.

Otherwise, it's going to take 3-5 years for the community to finally settle on a CM strategy like we did very late in the D7 cycle (Features + Strongarm). For most sites larger than a brochure site, I've seen a lot of developers really struggle with how they manage configuration. Can a product feature be 'we have CM that's practical, built into core'?

In terms of building and maintaining enterprise level sites, config management is a critical aspect of Drupal. It's also very important for a good "out of the box experience" item 8, because without this, we're required to write custom modules and solutions in order to work with blocks. There are other shortfalls to core's CM at this point too. In short, I second the request in #15.

CMI: This external article was featured on Drupal's newsletter a few months back, and describes the challenges and workarounds exactly:

https://www.freelock.com/blog/john-locke/2016-12/drupal-8-configuration-...

I also second #15 and what others have said here about the adoption of Drupal and CMI, especially at the enterprise level.

Hi,

@hoporr, thanks for pointing to my blog post, and letting me know about this issue!

As I think about the question of what might be improved, I'm a little concerned that a solution might add even more complexity to the challenge of managing configuration. At the risk of doing that, I can think of a couple improvements:

• Some sort of automatic config sync, or at least reporting. Perhaps a cron job compares the installed config with the sync path? I'm thinking there could be settings to either just report "out of sync" on the status report page or automatic update of the files from the installed config... I know it's possible to use file-based configuration entirely, but this has a performance penalty as well as potential security ramifications (we don't allow the web server user to have write access to config files, for example).
• "Developer mode" config. It would be awesome to have a UI that would allow flagging specific configs as "development only" and stored separately with a toggle for whether this config is applied (possibly as an override) on a particular environment. I could see wanting to ignore these changes entirely for a particular environment, or allow it to be exported/committed without risking getting applied on a production environment. Particularly need the ability to specify modules enabled only on development copies.
• Provide easy mechanism to "install" a new content entity. I'm thinking of things like block entities, content pane entities that have their UUIDs exported/referenced in a config entity. I would think a config export should export the current state of any referenced content entities, and a config import would create these entities if they don't exist -- but ignore if they do.

Freelock's current methodology works very well for us, but it took a lot of work and a bunch of mistakes before we felt like we had it dialed in -- and I think the barriers to new shops adopting it are pretty large. These 3 improvements would make it a lot easier, and a lot more discoverable, for developers to figure out best practices.

Cheers,
John

@webchick - Two proposed configuration management user stories:

1. As a developer, I want to vary configuration between different environments (e.g. config specific to local, dev, stage, and prod).

2. As a site builder, I want to create blocks and contact forms (and potentially other entities) on production, but not have them removed when doing a full config import that doesn't include them in the configuration (e.g. Create and manage Webforms on prod, not have to copy back database and export to preserve them).

Those are probably the two most common annoyances.

Perhaps to add one story to that list.

As a translator i wan't to be able to translate config on the production site, and not have them removed or overridden on a full config import.

For geerlingguys second story, I'd like to add on (or create a new story) a piece.

As a site builder, I want to create blocks and contact forms (and potentially other entities) on production, and have the option to not have them removed when doing a full config import that doesn't include them in the configuration (e.g. Create and manage Webforms on prod, not have to copy back database and export to preserve them).

Basically I would think that this is a user decision which would vary by case. This would have probably been covered but I think it's important to have it defined in the story.

Additionally my own user story:

Given I have a sites configuration stored in my project
and the user has provided a database connection string in drush/console/settings.*.php
and I choose the option to install a site with pre-exsisting configuration.
I see a site with all configuration entities created (I think config entity is the term I'm looking for here)
and the site is installed with the same install profile.

The last part, may be written better but the point was to avoid a config-import conflict because it installed the incorrect profile. We've had a few issues with config_install and minimal getting confused during the import then have issues like not being able to disable dblog because it's defined as on in the profile.

Adding "configuration management improvements" to the list in the issue summary as was added to the slides as well.

Thinking further on the config management side of things, as we're starting to switch some early D8 sites from "building it new" to "maintaining and updating it", another huuuuge issue has been configuration that's provided by modules.

• How do I add new configuration entities/schema/settings (see #1398040: Detect if default configuration of a module has been changed, and allow to restore to the original?
• How do I allow for overrides without having really weird things happen (e.g. #2682369: Fix problems with overridden config entities)?
• How do I revert a module's configuration or reinstall it so I can get new config (e.g. for migration-related modules where you want to add a new migration or test a change migration's mappings - see #1497268: Add revert functionality to Config entities)?

@geerlingguy It would be great if you could help flesh out the details on that topic in #2869915: Configuration management improvements for 8.4. I created a stub issue there because config management is not my main area of expertise, but I do know that we're aiming to have all the plans/meta issues for these product priorities inside the ideas queue. Thanks!

I updated #2757967: API-first initiative, #2836165: New experimental module: JSON API and #2834718: New experimental module: OAuth2 to reflect that adding more experimental modules won't happen in 8.4.

i thought there would be an isolation between core views and my custom views in drupal 8.4 but there seems to be no update on that ..

no site admin want to mess core views. unless its necessary..