Allow marking a config schema types to be marked as "internal" (to avoid exposing them via HTTP APIs)

a internal flag (name TBD)

In #3037039-17: Create a public API for indicating resource types should not be exposed, @gabesullice proposes distinguishing the semantics of "makes sense to be consumed via an API" vs. "is (un)prepared to be consumed via an API". I don't know if there's agreement about that but #3040354: Introduce entity type annotation key for indicating that an entity type is *safe* for HTTP API usage is also exploring the possibility of adding a different annotation for the latter.

However, we don't currently distinguish those two meanings, so as long as that's the case, a single internal flag (to match TypedData's isInternal()) might be the best way to proceed. And if we eventually agree on adding a different flag to TypedData, then we can add it to config schemas as well.

I think a proof of concept patch would be a good next step. I'm curious if it would make config normalization much slower, since we'd need to walk the entire tree and load and process config schema along the way.

It appears quite a lot of decoupled sites are reading display configuration stored in EntityViewDisplay

Really? Where have you noticed that?

Personally, I consider what you're describing to be a complicated anti-pattern. It's unfortunate people feel forced into it.

My hope is that with JSON:API Hypermedia stabilized, some enterprising individual will write a "JSON:API Displays" module which adds a custom link to resources in much the same was that JSON:API Schema does, like so:

links: {
  displays:full: {
    href: https://example.jsonapi.dev/jsonapi/node/article/display/full.json
  },
  displays:teaser: {
    href: https://example.jsonapi.dev/jsonapi/node/article/display/teaser.json
  }
}

These endpoints would aggregate information across entity view displays and field configs into a single, logical JSON object. There's no good reason to expose the Drupal implementation detail that is config entities and view displays other than the fact that it's expedient. This module could also incorporate the Layout Builder information you're talking about without exposing its storage format.

Why do you consider it an anti-pattern? It enables site builders to have some control over the front end just like they are able to do in fully coupled Drupal, specifically to control the order in which fields are rendered.

Allowing site builders to have some over the front end is desirable, but deriving that display logic from disparate and complex config entities (read: implementation details) is an anti-pattern. Consider this: Twig templates do not load and parse config entities, they receive render arrays which encode display logic through structure and semantics.

I've really struggled to come up with a case where direct access to config entities is really the best pattern for a decoupled applications. Perhaps you can help me there.

This issue's title, intent and scope are generic. If it's possible to mark subtrees of information as internal for content entities (by marking fields or field properties as internal), then the same should also be possible for config entities. Do you disagree with that?

Whether I agree with that totally depends on the conclusion to the above. If we really think config entities represent valuable content to a decoupled consumer, then yes. I think there should be a facility to mark some things internal. OTOH, if we can't come up with any practical use cases for exposing config entities that shouldn't be done in another, more elegant way then I think the point is moot. Config entities are by nature completely internal.

I've really struggled to come up with a case where direct access to config entities is really the best pattern for a decoupled applications. Perhaps you can help me there....There's no good reason to expose the Drupal implementation detail that is config entities and view displays other than the fact that it's expedient.

You might be correct that decoupled applications accessing config entity resources directly is always a case of expediency rather than ideal architecture, but let's not underestimate the value of expediency here. In a lot of cases, teams who are building decoupled applications might consist entirely of front-end developers and no back-end developers. Let's say that such an application wants to present a UI that lets you select from an image style, or a date format, or a Vocabulary, or any number of contrib config entities, such as a commerce currency, or shipping option, or ...

It might be that in each such case, there ought to be a JSON:API resource with a well-thought-out API that presents these things in a way that is abstracted from the underlying config entities, but let's face it: most contrib modules won't ship with such resources. A team with a back-end developer could build these resources, but what about a team consisting of just JS developers? Maybe config entity resources aren't the best representation that's theoretically possible, but they're already there, for free, for every contrib module's config entity type, and that counts for a lot.

@effulgentsia, good points. I think the selection UI you mention in your first paragraph could already work by utilizing JSON:API Schema and reading the allowed enum values. The rest could be automatically derived from field and display config entities if someone wrote a module for that generically. So it wouldn't have to be custom to every module or project (read, it wouldn't need a BE dev).

I do hear your point about the here and now though. What I'm describing is a long way off.

FWIW, my proposal isn't (and wasn't) to close this issue. I said we should leave it on this list of issues, but at the lowest of priorities.

I think you both would agree that most sites don't need to read config entities. Once we complete #3040372: Allow for API-First content without requiring all configuration to also be API-first and #3040354: Introduce entity type annotation key for indicating that an entity type is *safe* for HTTP API usage, which I believe @Wim Leers agreed were higher priorities, most config entities won't be accessible by default. Thus, internalizing sub-properties of the very few remaining accessible config entities, is not a very high priority in my mind.

@Wim Leers or @effulgentsia, do you think this issue should be prioritized above anything else?

I started working on this, and assumed that a good approach would be to remove LayoutEntityDisplayNormalizer and add an internal flag to the layout_builder.section config schema, then add support for checking the internal flag to \Drupal\serialization\Normalizer\ConfigEntityNormalizer::getDataWithoutInternals.

After removing LayoutEntityDisplayNormalizer, I still couldn't see "sections" when viewing a entity view display that used Layout Builder. This is making manual testing difficult - what's the expected behavior when LayoutEntityDisplayNormalizer is removed?

@wim-leers Thanks for digging - I'll do my testing with a test config schema instead of Layout Builder then.

Here's what I have so far - test coverage is implicit based on the changes to the config_test entity type. Since config entities can't be edited via HTTP APIs yet, POST/PATCH haven't been tested.

Should fix some of the test failures.

Working on fixes.

Adding onto config_test, even when I tried adding a new entity type within config_test, led to too much complication. I ended up splitting test coverage into a completely separate test submodule which should avoid those complications.

Worked on test fixes and added unit test coverage for \Drupal\serialization\Normalizer\ConfigEntityNormalizer.

Forgot to update the config normalizer service arguments.

This sounds like a good idea, but needs a review from someone with a little more knowlegde of the config system than me i think.

The last patch failed to apply on 9.4, so re-rolled it for 9.4.

Seems your reroll didn't work. Any chance you can try again?

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

I couldn't figure out a good way to reroll this, so I copied all the code from the patch to a new install and tried to get that to work, attached are the results.

Actually uploading a patch that contains what I wanted it to

rebased on 11.x