Problem/Motivation

#2300677: JSON:API POST/PATCH support for fully validatable config entities intends to allow config entities to opt-in into typed data validation, something that only content entities were able to do until now.

Proposed resolution

Allow config entities to support typed data validation by implementing the new ValidatableInterface.

Remaining tasks

Review.

User interface changes

Nope.

API changes

Not really.

Data model changes

Nope.

CommentFileSizeAuthor
#2 2907163.patch2.53 KBamateescu

Comments

amateescu created an issue. See original summary.

amateescu’s picture

Status: Active » Needs review
StatusFileSize
new2.53 KB

This should be enough I think.

dawehner’s picture

What I still don't understand, how can entities opt out of validation with this patch? How can rest module stop exposing content_moderation entities ... just this patch doesn't solve that bit :)

amateescu’s picture

@dawehner, like I said in #2300677-190: JSON:API POST/PATCH support for fully validatable config entities and again in #194, I don't think we should introduce the ability for content entities to opt out of validation because it would be an API change. And even though the ContentModerationState entity type is internal, why wouldn't it support validation?

REST module should exclude that internal entity type in \Drupal\rest\Plugin\Deriver\EntityDeriver::getDerivativeDefinitions().

dawehner’s picture

And even though the ContentModerationState entity type is internal, why wouldn't it support validation?

We had this discussion before in #2843753: Prevent ContentModerationState from being exposed by REST's EntityResource ... its a purely internal entity.

REST module should exclude that internal entity type in \Drupal\rest\Plugin\Deriver\EntityDeriver::getDerivativeDefinitions().

Fair, I guess content_moderation could exclude itself using some alter hook, if really needed.

One question about \Drupal\Core\Entity\FieldableEntityInterface::isValidationRequired ... why is this not used at all in Drupal core? Is it just me that this smells like another bug?

amateescu’s picture

We had this discussion before in #2843753: Prevent ContentModerationState from being exposed by REST's EntityResource ... its a purely internal entity.

And purely internal entities should also be validatable before saving (potentially invalid) data to the storage, no?

One question about \Drupal\Core\Entity\FieldableEntityInterface::isValidationRequired ... why is this not used at all in Drupal core? Is it just me that this smells like another bug?

That's because \Drupal\Core\Entity\ContentEntityBase::preSave() decided to access the validationRequired property directly and no other outside use came up so far.. I guess.

dawehner’s picture

Assigned: Unassigned » tim.plunkett

Thank you @amateesc for the explanation this makes sense.

I'm calling out to @tim.plunkett whether he believes that adding these 3 methods to ConfigEntityInterface makes sense.

At least for me personally I like that config entities are free of typed data and everything typed data related is orthogonal.

amateescu’s picture

@dawehner, who said anything about adding these 3 methods to ConfigEntityInterface? This issue proposes to introduce an interface that *can* (not should, nor must) be implemented by a config entity type. For example:

class NodeType extends ConfigEntityBundleBase implements NodeTypeInterface, ValidatableInterface

I really don't understand what's so confusing about this issue :)

tedbow’s picture

I made a patch on #2300677-200: JSON:API POST/PATCH support for fully validatable config entities that takes this patch and applies it to that issue.

I like the way it simplifies that logic. I add the trait there to the ConfigTest entity and made a trait for basic validation that implements the methods in ValidatableInterface that was basically does what we were doing in ConfigEntityResource. So basically if an ConfigEntity want the basic validation of typed data they can just implement the interface and use the trait.

Also before this idea in #2300677: JSON:API POST/PATCH support for fully validatable config entities we were only validating in the REST resource so if so it would not be obvious for a developer how to validate a config entity if they were making one programmatically.

wim leers’s picture

Assigned: tim.plunkett » Unassigned
Status: Needs review » Reviewed & tested by the community

Fair, I guess content_moderation could exclude itself using some alter hook, if really needed.

It already does this: content_moderation_rest_resource_alter().

why is this not used at all in Drupal core? Is it just me that this smells like another bug?

😱

I'm calling out to @tim.plunkett whether he believes that adding these 3 methods to ConfigEntityInterface makes sense.

@amateescu said in the other issue that we'd add this on a per-config entity type basis. In other words: to those config entity types that add support for validation. So for example class NodeType extends ConfigEntityBundleBase implements NodeTypeInterface would change to class NodeType extends ConfigEntityBundleBase implements NodeTypeInterface, ValidatableInterface (EDIT d'oh that's the example @amateescu gave in #8, sorry :P

Also before this idea in #2300677: JSON:API POST/PATCH support for fully validatable config entities for config entity types that support validation we were only validating in the REST resource so if so it would not be obvious for a developer how to validate a config entity if they were making one programmatically.

Great point! This enhances the PHP DX.

wim leers’s picture

Issue tags: +API-First Initiative, +blocker
dawehner’s picture

Well I'm still not convinced of putting typed data level stuff on config entities, but it seems to be nobody gives a shit.

wim leers’s picture

Assigned: Unassigned » tim.plunkett
Status: Reviewed & tested by the community » Needs review

I didn't mean to bypass you, Daniel. I thought you had just misinterpreted. Probably means I'm missing something. Moving back to prior state.

wim leers’s picture

Priority: Normal » Major

#2300677: JSON:API POST/PATCH support for fully validatable config entities is major, this is a blocker, so bumping this to major too.

dawehner’s picture

I mostly try to keep things maintainable.

As of now config schema for example is not required for config entities to work, but rather are departed completely.
At least for me validation is just another level of config schema, also based upon typed data, but this time instead based upon constraints.

Don't get me wrong, there are advantages of putting stuff onto this interface as well.

tedbow’s picture

@dawehner is you concern that with #2300677-184: JSON:API POST/PATCH support for fully validatable config entities #184 and before the typed data validation was something that happened on the Config REST resource level, i.e if you were using REST then typed data validation is not connected to Config entities....

But with this patch for any Config Entity that implements that new typed data validation will directly become part of the Entity itself?

dawehner’s picture

But with this patch for any Config Entity that implements that new typed data validation will directly become part of the Entity itself?

But with this patch for any Config Entity that implements that new typed data validation will directly become part of the Entity itself?

Yeah basically. I think asking some of the configentity maintainers would be fair, given they can judge this better.

Maybe the answer will be: Its okay to add this one but limit to any further change.

One general question, is there a reason why we could not do this issue after the rest config issue? Could we introduce some other temporary way to flag config entites as validatable, but don't make this a public flag yet?

amateescu’s picture

Well I'm still not convinced of putting typed data level stuff on config entities, but it seems to be nobody gives a shit.

How is this patch putting any kind of typed data level stuff on config entities?

+++ b/core/lib/Drupal/Core/Entity/ValidatableInterface.php
@@ -0,0 +1,39 @@
+  /**
+   * Validates the currently set values.
+   *
+   * @return \Drupal\Core\Entity\EntityConstraintViolationListInterface
+   *   A list of constraint violations. If the list is empty, validation
+   *   succeeded.
+   */
+  public function validate();
+
+  /**
+   * Checks whether entity validation is required before saving the entity.
+   *
+   * @return bool
+   *   TRUE if validation is required, FALSE if not.
+   */
+  public function isValidationRequired();
+
+  /**
+   * Sets whether entity validation is required before saving the entity.
+   *
+   * @param bool $required
+   *   TRUE if validation is required, FALSE otherwise.
+   *
+   * @return $this
+   */
+  public function setValidationRequired($required);

If we look at the documentation of these three methods, they have *nothing* to do with typed data.

Arguably, the only "problem" might be the return value documented by the validate() method, which is an object that implements EntityConstraintViolationListInterface, and that interface mentions "fields" in the signature of some of its methods, but those can simply be changed to mention "property" instead.

So.. I really don't understand all the hate for this proposal.

amateescu’s picture

StatusFileSize
new3.14 KB
new628 bytes

We also need a small correction to EntityConstraintViolationListInterface because of the new ValidatableInterface that we're adding here.

tim.plunkett’s picture

I read through #7, and didn't have to comment. But my question to Wim in slack was "why isn't config schema sufficient?"
Great to see @dawehner with similar thoughts.

amateescu’s picture

@tim.plunkett, have you also read #8?

@dawehner in #7 was somehow under the impression that we're trying to add the validation methods to ConfigEntityInterface, which is not the case.

amateescu’s picture

And the new concern is that we're trying to shove typed data stuff into config entities, which is also not the case, as explained in #18 :)

tim.plunkett’s picture

I misunderstood the scope of this issue. But mostly because I kept reading the last patch in #2300677: JSON:API POST/PATCH support for fully validatable config entities, which then starts applying it to config entities.

So, in a vacuum this patch is fine? But I'm very wary of the other issue.

amateescu’s picture

Ok, then let's take it out of the vacuum and ask some real world questions.

The intention of #2300677: JSON:API POST/PATCH support for fully validatable config entities is pretty clear, it wants to extend rest.module's support for config entities, and in order to do that it needs config entities to be validatable.

Now, the real question is: how can we make config entities validatable with an API that is not REST-specific? I think the first step is what the current patch does.

What should be the next step? Implement one-off validation code for each config entity?

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

What should be the next step? Implement one-off validation code for each config entity?

This is a weird question ... I hope we don't suggest people to validate on their own, instead of using our systems.

Let's just get this in, people seemed to not be convinced by my argument, cool.

xjm’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +Needs framework manager review

#25 isn't really convincing as an RTBC; please only RTBC things that you actually are signing off on as a peer reviewer. :)

So both @dawehner and @tim.plunkett have raised concerns about the longterm roadmap, that this might be used to add additional validation to config entity types other than their schemata. I understand that this is not being added to ConfigEntityInterface or anything and not added to any config entity types yet, that it's only an option, etc. However, I think the concerns are legitimate.

The concerns are:

  1. That config schema is already the way we validate config, and so implementing other validation could be superfluous or even dangerous.
  2. That config is not currently coupled to typed data and we want to keep it that way.
  3. That implementing such other validation inconsistently for some config entity types on a case-by-case basis or as a gradual API addition might actually be worse.

I had all the same thoughts as I looked at this issue, and I'm not convinced by the arguments that they shouldn't be concerns. Like @tim.plunkett, I think this patch is fine as an improvement for the content entity API. Nice, even, as @tedbow says. So I think for all of us, the concerns are with #2300677: JSON:API POST/PATCH support for fully validatable config entities and not with this issue;. I'd like to see the relevant discussion of those concerns added to #2300677: JSON:API POST/PATCH support for fully validatable config entities before we proceed here. I'll also tag that issue for subsystem and framework manager review.

Also tagging this one for FM review since that's where we escalate when subsystem maintainers disagree, but let's please have an actual RTBC and attempt at consensus (or documenting the lack of consensus and counterpoints in the IS) before we do. :)

dawehner’s picture

In case we decide to do this, we should file a follow up which does the following: It implements the validation inside a trait. This way we don't add the validation onto ConfigEntityBase, given it would show the wrong signal.

amateescu’s picture

@dawehner, I think we can use #2912626: Configuration validation for that?

andypost’s picture

IMO ValidatableInterface is too broad better to prepend it kinda EntityFieldValidatationInterface

amateescu’s picture

@andypost, the interface name is intended to be broad :) It also covers validation of entities as a whole (composite constraints), so it's not tied to fields.

dawehner’s picture

I really think #1818574: Support config entities in typed data EntityAdapter should be added before adding an interface and pretending to support customisability, where we actually don't support some.

wim leers’s picture

Title: Split validation-related methods from FieldableEntityInterface into a separate ValidatableInterface » [PP-1] Split validation-related methods from FieldableEntityInterface into a separate ValidatableInterface
Status: Needs review » Postponed
Related issues: +#1818574: Support config entities in typed data EntityAdapter

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

wim leers’s picture

Issue tags: -API-First Initiative, -blocker +API-First Initiativer
effulgentsia’s picture

Issue tags: -API-First Initiativer +API-First Initiative

Restoring what I think was an unintentional tag name change in #34.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

tim.plunkett’s picture

Assigned: tim.plunkett » Unassigned
Status: Postponed » Needs work

Should have been unpostponed 5 years ago, whoops

andypost’s picture

Title: [PP-1] Split validation-related methods from FieldableEntityInterface into a separate ValidatableInterface » Split validation-related methods from FieldableEntityInterface into a separate ValidatableInterface
wim leers’s picture

This blocks #2300677: JSON:API POST/PATCH support for fully validatable config entities.

IMHO we should also do #2327883-108: Field [storage] config have incomplete settings until they are saved here, because without it, we won't be able to actually validate some config entities! 😱 (Which is the goal stated in the issue summary.)

Quoting #2327883-108:

One thing I definitely hadn't thought of yet so far is how config entity objects may have different values in their runtime state vs their storage state, thanks to \Drupal\Core\Config\Entity\ConfigEntityStorage::mapToStorageRecord()! 😳 Fortunately, only 3 classes in core override it:

  1. \Drupal\Core\Field\FieldConfigStorageBase::mapToStorageRecord()
  2. \Drupal\field\FieldStorageConfigStorage::mapToStorageRecord()
  3. \Drupal\layout_builder\Entity\LayoutBuilderEntityViewDisplayStorage::mapToStorageRecord()

… plus 6 in contrib, 5 of which are virtually identical to core's Layout Builder one.

This means that for config entity validation to work, the infrastructure that provides validation for config entities must also be able to call mapToStorageRecord() not just prior to save, but prior to validation. But that method is currently protected 😅 So we need some way to get config entities into the same state as just before saving. We may want to do that as part of #2907163: Split validation-related methods from FieldableEntityInterface into a separate ValidatableInterface or in another issue.

@amateescu, do you have thoughts on how to do that? 🤓🙏

wim leers’s picture

Also, #3324150: Add validation constraints to config_entity.dependencies introduced \Drupal\KernelTests\Core\Config\ConfigEntityValidationTestBase::assertValidationErrors() which does:

  protected function assertValidationErrors(array $expected_messages): void {
    /** @var \Drupal\Core\TypedData\TypedDataManagerInterface $typed_data */
    $typed_data = $this->container->get('typed_data_manager');
    $definition = $typed_data->createDataDefinition('entity:' . $this->entity->getEntityTypeId());
    $violations = $typed_data->create($definition, $this->entity)->validate();

We will may need to update that for #48.

wim leers’s picture

Priority: Major » Normal
Issue tags: -blocker

Do we really still need this if #1818574: Support config entities in typed data EntityAdapter already made this possible? This code was committed to Drupal core in 2018:

  public function testValidate() {
    $adapter = ConfigEntityAdapter::createFromEntity($this->entity);
    $violations = $adapter->validate();

Seems this is just a nice-to-have now?

Not to mention: not every config entity type will actually be validatable (we can guarantee this in core, not in contrib), so the existing ::isValidationRequired() is insufficient. We'd also need ::isValidationSupported() or something like that?

Demoting priority.

joachim’s picture

It would still be really useful and a DX improvement if all entities, whether config or content, had the same API for validation.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.