Problem/Motivation

Currently in ContentEntityForm we build the entity once in the validation and once in the submit, however if the validation passed without errors then the submit will be called and will have to build the entity again.

This has two problems ;
1. In case of nested entity forms building the entity is expensive.
2. Because of entity builders doing changes to form state etc.. the entity build in validate and later in submit again might differ which means it might happen that we save an entity that we haven't validated and also might not be valid at all.

There is even an existing problem in core where we are saving an entity without completely validating it - #2834030: Currently we are saving an user entity with some not validated fields on the registration form and it just proves that it is so easy to oversee such things and how important this issue is in order to prevent saving invalid entities - in core, custom and contrib.

Proposed resolution

Store the entity build in the validation phase in $this->intermediateEntity and use this later in in submit instead of building the entity again.

Remaining tasks

Write the patch plus test ensuring we are saving the same entity that we have validated.

User interface changes

None.

API changes

None.

Data model changes

None.

CommentFileSizeAuthor
#51 2833682-nr-bot.txt144 bytesneeds-review-queue-bot
#43 interdiff-41-43.txt1.82 KBgease
#43 2833682-43.patch10.84 KBgease
#41 interdiff-39-41.txt746 bytesgease
#41 2833682-41.patch10.63 KBgease
#39 interdiff-36-39.txt784 bytesgease
#39 2833682-39.patch9.72 KBgease
#36 2833682-36.patch9.58 KBtstoeckler
#32 interdiff-29-32.txt4.09 KBhchonov
#32 2833682-32.patch9.57 KBhchonov
#29 interdiff-27-29.txt577 byteshchonov
#29 2833682-29.patch7.52 KBhchonov
#27 interdiff-25-27.txt2.86 KBhchonov
#27 2833682-27.patch7.49 KBhchonov
#25 2833682-25.patch5.92 KBhchonov
#20 interdiff-19-20.txt849 byteshchonov
#20 2833682-20.patch5.91 KBhchonov
#19 interdiff-16-19.txt3.09 KBhchonov
#19 2833682-19.patch5.92 KBhchonov
#16 2833682-16.patch5.37 KBhchonov
#12 2833682-12.patch2.36 KBvprocessor
#11 interdiff-3-11.txt961 byteshchonov
#11 2833682-11.patch5.37 KBhchonov
#8 2833682-7.patch2.41 KBvprocessor
#4 2833682-3-fix-with-test.patch4.34 KBhchonov
#4 2833682-3-test-only.patch2.34 KBhchonov
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

hchonov created an issue. See original summary.

hchonov’s picture

Title: In submit use the entity build during the validation instead of building it again for performance and consistency ensuring we are saving the entity that has been validated » In submit use the entity built during the validation instead of building it again - for performance and consistency ensuring we are saving the entity that has been validated
vprocessor’s picture

Assigned: Unassigned » vprocessor
hchonov’s picture

Assigned: vprocessor » Unassigned
Status: Active » Needs review
FileSize
2.34 KB
4.34 KB

I've implemented the proposed solution from the issue summary.

@vprocessor sorry ;)

The last submitted patch, 4: 2833682-3-test-only.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 4: 2833682-3-fix-with-test.patch, failed testing.

vprocessor’s picture

Assigned: Unassigned » vprocessor
vprocessor’s picture

Status: Needs work » Needs review
FileSize
2.41 KB
vprocessor’s picture

Assigned: vprocessor » Unassigned

Status: Needs review » Needs work

The last submitted patch, 8: 2833682-7.patch, failed testing.

hchonov’s picture

Status: Needs work » Needs review
FileSize
5.37 KB
961 bytes

So the problem was that the user registration form was relying on building the entity twice and has done stuff with the form values which has to be done as part of ::buildEntity instead of ::submitForm.

vprocessor’s picture

hchonov’s picture

@vprocessor when you are posting a patch could you please provide interdiffs and a little explanation what you are doing? Is there anything you do not agree with regarding the last patch I've posted?

Status: Needs review » Needs work

The last submitted patch, 12: 2833682-12.patch, failed testing.

hchonov’s picture

Status: Needs work » Needs review

Re-uploading the patch from #11 as the currently working active patch to be reviewed.

hchonov’s picture

Oups.. there it is...

Berdir’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/EntityForm.php
    @@ -428,4 +435,15 @@ public function setEntityTypeManager(EntityTypeManagerInterface $entity_type_man
    +   * {@inheritdoc}
    +   */
    +  function __sleep() {
    +    // Do not serialize the intermediate entity as it is only needed if the
    +    // form is submitted directly after the form is validated, but in case of
    +    // steps on multi-step forms.
    +    unset($this->intermediateEntity);
    +    return parent::__sleep();
    +  }
    

    end of that sentence doesn't make sense, a not missing?

  2. +++ b/core/modules/user/src/RegisterForm.php
    @@ -59,7 +59,7 @@ protected function actions(array $form, FormStateInterface $form_state) {
       /**
        * {@inheritdoc}
        */
    -  public function submitForm(array &$form, FormStateInterface $form_state) {
    +  public function buildEntity(array $form, FormStateInterface $form_state) {
         $admin = $form_state->getValue('administer_users');
     
         if (!\Drupal::config('user.settings')->get('verify_mail') || $admin) {
    

    I think this is exactly why I gave up on doing something like this in #2799637: Document that #entity_builders and overrides of EntityForm::copyFormValuesToEntity() must be idempotent.

    Any change that requires changing something else means that it is an API change and breaks BC, unless it is in areas where we say that's ok, which is I think not true for this.

hchonov’s picture

end of that sentence doesn't make sense, a not missing?

Yes :). Will update in the next patch version :).

Any change that requires changing something else means that it is an API change and breaks BC, unless it is in areas where we say that's ok, which is I think not true for this.

Yes I know that this is a change, but it is there for two reasons - one is performance and the second is to ensure that exactly the entity we have validated is saved and not any other one. Take a look at the paragraphs module which actually is a famous one and there is an inconsistency that when the langcode of the language widget is altered and the form is submitted the entities built during the validation phase will have the language code they had during initially building the form but then in the submit the generated entities will differ than the ones generated during the validation and will have the new language code. This leads into saving entities which are not validated and under circumstances might even lead to some constraints not validating what is being saved. Consider e.g. a validation constraint on the paragraph entity that the language "test_language" is not a valid translation, but currently they would pass the validation with the previous langcode and be saved with the new one...

That is why I think that this API change is an important and needed one as everybody makes mistakes and this would prevent such ones - I guess we all want that when we start using a contrib or also a custom module in our system that this module is not saving not-validated entities.

hchonov’s picture

Addressing #17.1 and moving all knowledge of the intermediate entity to the content entity form only.

hchonov’s picture

Just a small improvement for a shorter statement.

hchonov’s picture

Issue summary: View changes

The part where I had to adjust the RegisterForm is actually an issue on itself - #2834030: Currently we are saving an user entity with some not validated fields on the registration form as because of the current implementation not all user entity fields are correctly validated. #2834030: Currently we are saving an user entity with some not validated fields on the registration form just proves that it is so easy to oversee such things and how important this issue is in order to prevent saving invalid entities - in core, custom and contrib.

hchonov’s picture

Issue summary: View changes
hchonov’s picture

Issue tags: +Needs change record

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

hchonov’s picture

Re-roll + instead of unsetting the intermediateEntity property in __sleep it is better to set it to NULL as in some issues we've seen problems by doing so on PHP 5.6.

Berdir’s picture

+++ b/core/lib/Drupal/Core/Entity/ContentEntityForm.php
@@ -134,7 +141,13 @@ public function form(array $form, FormStateInterface $form_state) {
+    // Let ::buildEntity return the entity generated in ::validateForm instead
+    // of building it again to ensure the validated entity is the one that is
+    // saved.
+    $this->returnIntermediateEntity = TRUE;
     parent::submitForm($form, $form_state);
+    unset($this->returnIntermediateEntity);
+

I don't understand why you don't properly define this with default value FALSE and then set back to FALSE instead of unsetting, which is IMHO very weird and would simplify the code a lot below.

hchonov’s picture

You are right .. I've addressed this and also found a little mistake in buildEntity having logic still running twice, so I've covered that too.

Status: Needs review » Needs work

The last submitted patch, 27: 2833682-27.patch, failed testing.

hchonov’s picture

Would it be better/simpler if instead of having these two new properties on the content entity form just to put the validated entity as a temporary value in the form state and then retrieve it in the submit?
In this case we don't have to worry about the serialization of the form, form object and the form state as temporary values are being thrown away and we don't serialize between validation and submit.

What I mean is:
$form_state->setTemporaryValue('validated_entity', $entity);

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

hchonov’s picture

Adding getter and setter to ContentEntityFormInterface for the intermediate entity.

tim.plunkett’s picture

+++ b/core/lib/Drupal/Core/Entity/ContentEntityFormInterface.php
@@ -75,4 +75,24 @@ public function isDefaultFormLangcode(FormStateInterface $form_state);
+  public function setIntermediateEntity(ContentEntityInterface $entity);
...
+  public function getIntermediateEntity();

I think these should explain when to use these instead of ::getEntity()/::setEntity()

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

tstoeckler’s picture

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

gease’s picture

Fixed Drupal\Tests\system\Functional\Entity\EntityFormTest failure for 8.9. All other failures are on layout builder, and, truly, layout builder doesn't work with this patch. Entity built on validation doesn't contain layout elements, while entity built on submit contains them. The solution may be to introduce "entity_rebuild" flag and rebuild entity on submit only when this flag is set.

Status: Needs review » Needs work

The last submitted patch, 39: 2833682-39.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gease’s picture

Status: Needs work » Needs review
FileSize
10.63 KB
746 bytes

Fixed layout builder by removing condition which was meant to be removed with this patch.

hchonov’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/layout_builder/src/Plugin/Field/FieldWidget/LayoutBuilderWidget.php
    @@ -39,12 +39,6 @@ public function formElement(FieldItemListInterface $items, $delta, array $elemen
    -    // @todo This isn't resilient to being set twice, during validation and
    -    //   save https://www.drupal.org/project/drupal/issues/2833682.
    -    if (!$form_state->isValidationComplete()) {
    -      return;
    -    }
    

    Fixed layout builder by removing condition which was meant to be removed with this patch.

    This is just amazing. The current approach has been just validated as needed for core!

  2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityForm.php
    @@ -146,7 +166,13 @@ public function form(array $form, FormStateInterface $form_state) {
    +    // Let ::buildEntity return the entity generated in ::validateForm instead
    

    When referring to methods in a comment we should append braces.

    Also #33 still has to be addressed.

gease’s picture

Status: Needs work » Needs review
FileSize
10.84 KB
1.82 KB

Tried to fix the comments in response to #33 and #42 p2.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

rp7’s picture

I have a use-case where I need the built entity (intermediate entity as it's called in this issue) in a #element_validate callback for a specific form element. I believe this would be a solution for this as well?

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
FileSize
144 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.