Problem/Motivation
The new method FieldItemList::equals() introduced by #2297817: Do not attempt field storage write when field content did not change is not reliable.
Due to the nature of PHP value types get converted into various types when dealing with form values or database result sets. So the final check return $value1 === $value2;
fails on comparing on these values:
timestamps:
array(1) {
[0] =>
array(1) {
'value' =>
int(1429624933)
}
}
array(1) {
[0] =>
array(1) {
'value' =>
string(10) "1429624933"
}
}
booleans:
array(1) {
[0] =>
array(1) {
'value' =>
bool(false)
}
}
array(1) {
[0] =>
array(1) {
'value' =>
int(0)
}
}
Proposed resolution
Replace the type safe comparison in FieldItemList::equals() by a non type safe one.
Remaining tasks
Think carefully if the propsed solution can cause any problem ;-)
User interface changes
none
API changes
none
Comment | File | Size | Author |
---|---|---|---|
#23 | 22-23-interdiff.txt | 823 bytes | mkalkbrenner |
#23 | 2475237_23.patch | 1.31 KB | mkalkbrenner |
#22 | 2475237_22.patch | 2.3 KB | mkalkbrenner |
#20 | 2475237_test_only_20.patch | 1.84 KB | mkalkbrenner |
#19 | 2475237_19.patch | 1.45 KB | mkalkbrenner |
Comments
Comment #1
BerdirDepends on your understanding of reliable ;)
What if the type matters and those two should be considered something different?
For storing values, we definitely want to rather save too much than lose data. It's a best effort implementation.
That might be very different for your use case, but I'm not sure how to unify those two different use cases.
Comment #2
mkalkbrennerI agree!
But FieldItemList::equals() became part of the field API and can therefor used elsewhere and not just to decide if we run an update statement on the storage. In this case it should do what it's documentation states - in a reliable way.
Personally I really appreciate this new API functionality and therefor it would be bad if we turn it into a somehow "private" function.
Let me think about it ...
Comment #3
hchonovWhat about if the form_state values are type casted to the type set by their fields before they land in values?
In this case the entity values and the ones from form state should always be of the same type and so there should be no case where this function is going to check the same values but of different types.
Comment #4
cspitzlay#3 Sounds like a good idea to me.
One might consider it a bug if one of the values to compare has the wrong type and is only equal if it is implicitely typecast. The data with the wrong type should never get this far.
Just one thought though: Will this force authors of custom code or contrib modules that manipulate form data to always get the type right, too? This may lead to lots of "My field is not saved. WTF?" bugs. Or would this typecast be done after any other alteration of the data?
Comment #5
cspitzlay@mkalkbrenner: Did you get these type differences when just loading and manipulating the entity or is a form submission involved?
Comment #6
mkalkbrenner@hchonov: I thought about a kind of normalization of field values, too. But this will cause a bigger change.
What about a simple change that allows to "select" the desired behavior:
Comment #7
alexpottRe-titling to describe the issue.
Comment #8
alexpottWhere does the type casting occur? Does it occur on the way back out of the database? If it does then shouldn't
==
be good enough because if we're relying anywhere on the variable type this shouldn't be down to the storage.Channeling my inner @yched - I'm not a huge fan of #6
Comment #9
mkalkbrenner@alexpott: Have a look at Drupal\system\Tests\Entity\ContentEntityChangedTest::testChanged() in the patch posted at https://www.drupal.org/node/2428795#comment-9852123.
This test failed and the examples in this issue's description are var_dump() during this test.
The timestamp is form the "created" field, the boolean from "language_default" and the same happens for "user_id" with an integer type:
In this test, no form handling is included, just the entity API and the entity SQL storage.
The arguments in #4 seems to be in the same direction. As well as https://www.drupal.org/node/2297817#comment-9852673
Let's see what the testbot says to such a simple change ...
Comment #10
mkalkbrennerComment #11
joelpittetWhoops, sorry re-posting here:
PHP is weird sometimes:
Comment #12
mkalkbrenner@joelpittet: you're right with that example. The string 'all' gets converted to integer in this context. That's PHP.
But I think that in our case we're just comparing "compatible" types like
0 == '0'
or0 == FALSE
that could be converted into each other. And the array keys matter as well.Comment #13
mkalkbrennerThe patch in #9 passed. How good is the test coverage? ;-)
Comment #14
joelpittetI'm not sure of this part of the system that well, it really does depend on where that data is coming from for the comparison. As long as you've thought of those strange bits.
Comment #15
mkalkbrenner@alexpott asked me to check where these type conversions are coming from. Basically they're caused by PDOStatement which returns every value as a string. That's an example of a result for a simple article:
These values are directly passed to entities' __construct() and cached internally in $this->values as strings without any type conversion.
From my point of view a type conversion within PHP makes no sense. Even if you assign a "PHP type" to each field the amount of places where you have to ask the field definition for the type and you have to convert the value accordingly seems to be endless.
Therefor the simple patch from #9 seems to be the right solution.
Comment #16
alexpottI agree - if the entity loaded from the db doesn't have typed properties then this makes sense. Let's improve the tests to cover this.
Aside: The temptation to say "doesn't have typed data" above was almost overwhelming.
Comment #17
mkalkbrennerHere's a test that covers the SQL storage load issue as described in #15. This one should fail.
Comment #19
mkalkbrennerAnd now the patches from #9 and #17 combined.
Comment #20
mkalkbrennerI extended the FieldItemListTest unit test In addition to the FieldSqlStorageTest from #17.
So we now have two different test that should fail without the patch from #9.
Comment #22
mkalkbrennerAnd now again the patch from #9 in combination with both tests.
Comment #23
mkalkbrennerI removed the kernel test as requested by @alexpott in IRC, because that will be covered by related issues.
Comment #24
hchonovJust to give a real example from contrib, where this patch will solve problems in the Relation entity.
From the preSave() of the Relation entity:
As we do not know if the number of the endpoints have changed, we always count them again and set the arity. In this case the arity is set as an integer, but the one loaded from the storage is a string.
The arity is an integer base field on the relation entity.
So just running
causes equals to return false for the arity field.
Comment #25
Fabianx CreditAttribution: Fabianx commentedRTBC, looks great to me.
Comment #26
alexpottThis issue addresses a major bug with test coverage and is allowed per https://www.drupal.org/core/beta-changes. Committed d560b48 and pushed to 8.0.x. Thanks!