The function addressfield_field_map_filter() is used as an array filter callback in order to prevent some code duplication. The problem is that this function gets called in multiple loops each time an entity's tokens are loaded. For example, on a single request to index search content this function was called 245,586 times for 50 entities on a site that has 122 fields. This number may be due to entities that reference other entities, regardless it is a significant performance issue.
The attached patch pulls these filter calls out into a helper function that is statically cached. This reduces the number of calls to 245,586~ to 122 in my testing. This took off approximately 3 seconds of processing time in my test scenario. It also significantly reduced the amount of memory (15mb -> 6kb).
We do not use addressfield tokens on the site, so I have not tested if this negatively impacts any token functionality. Is there a need for the call to get fields to be dynamic every time a token hook is fired during a request?
Comment | File | Size | Author |
---|---|---|---|
#20 | addressfield-fields-list-in-cache-2416997-20.patch | 4.26 KB | jsacksick |
#20 | tokens-with-token-module-enabled.png | 348.97 KB | jsacksick |
#20 | tokens-without-token-module-enabled.png | 245.49 KB | jsacksick |
#15 | 2416997-15.address_field_list_cache.patch | 2.9 KB | rszrama |
#13 | 2416997-13.address_field_list_cache.patch | 2.62 KB | rszrama |
Comments
Comment #1
bojanz CreditAttribution: bojanz commentedComment #2
SocialNicheGuru CreditAttribution: SocialNicheGuru commentedI have found this too while profiling with xhprof
Comment #3
SteffenRLooks fine for me too - i could drop the number of calls from of addressfield_field_map_filter from 37,008 to 144.
Comment #4
joelpittetI've been using this for a month now. I can't see any downsides to this approach.
Extra whitespace lines can be fixed on commit.
Comment #5
bojanz CreditAttribution: bojanz commentedWe have access to the entity type in $data['entity_type'], we can pass that to addressfield_get_address_fields() to further restrict the returned data.
Will save us from iterating on a bunch of non-needed fields.
Comment #6
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commentedAdded $entity_type as an optional argument to addressfield_get_address_fields(). It needs to be optional since entity_type is not always available in the caller function.
Comment #7
joelpittetThank you @czigor:)
Just some nit picks, but I think this looks pretty good as well.
Could you add this param to the docblock above?
Is it ok to just have the static's function name with an underscore when it's defaulting to empty string?
We ususally use !isset() it seems to 5% faster(but both are fast it's mostly for consistency). Also could just return early to avoid the extra nesting. (this of course was from the first patch)
Comment #8
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commentedThanks for the review!
1 and 3 have been fixed.
I can sleep with leaving 2 as it is, but if it's a problem, I can fix it as well. It's just one more line of code which I thought was not worth it.
Comment #9
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commentedA ! was missing from the previous patch.
@joelpittet Where should I return early?
Comment #10
joelpittet@czigor I was thinking like:
To avoid the overly nested code.
Comment #11
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commented@joelpittet Ah yes, that makes sense.
Comment #12
joelpittetThank you for your persistence @czigor, this should do the trick.
Comment #13
rszrama CreditAttribution: rszrama at Centarro commentedLightly updated patch attached. (Documentation fixes / a variable name.)
Comment #14
rszrama CreditAttribution: rszrama at Centarro commentedOk ... so, this isn't the fault of this patch. In theory, it's right. The problem is that the $field_name value isn't actually going to match anything in the token data array because field names there use hyphens in place of underscores. Arguably, we could commit this improvement without fixing that, but I'd prefer to kill two birds with one stone ... otherwise the solution to this issue could've just as well have been to remove this logic entirely. : P
Comment #15
rszrama CreditAttribution: rszrama at Centarro commentedUpdated patch attached that uses strtr() to convert underscores to hyphens in what I believe to be the appropriate places needs review.
Also, I realized my patch changes the logic from previous patches. Previously, filtering fields worked like so:
But my patch added an additional
!empty($entity_type)
to that if statement. My read on the earlier version is that if $entity_type were empty, every field would have been unset. Am I reading that wrong? In any event, review the approach in the current patch to see if it's screwing something up now, because I do think it's the intended behavior.Comment #16
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commentedWe've been using the patch by steven.wichers for more than half a year on freitag.ch without a problem. I gave the patch in #15 a try on my local machine, I could see no problems. The checkout process was fast, the sent emails (using addressfield tokens) were right.
The code looks good to me. As per the strstr() part: I'm no expert here but isn't the token module way (_) more general? Not all entities are based on entity api (-).
Comment #17
joelpittet@rszrama doesn't the token string replacement changing the API in this patch. It may be right but may be scope creep for this specific issue. I'm not sure if it would break a site's existing tokens, maybe @czigor tested that as it sounds like it. Dropping it back to NR for that question
Comment #18
czigor CreditAttribution: czigor at Liip for FREITAG lab. AG commented@joelpittet In my limited understanding if the addressfield is on an entity using the API of the entity API contrib module, then field tokens with '-' work. Otherwise '_' should be used. But this is just a gossip-level knowledge, would be nice if someone could confirm it.
Comment #19
apotek CreditAttribution: apotek commentedAccording to comments here https://www.drupal.org/project/addressfield/issues/2935041, this issue is blocking release 1.3 (which would include a fair number of improvements and fixes since last release). Can we either push this issue forward or attempt to release 1.3 without this issue? <-- @bojanz
Given the point at which the discussion is on this issue, it would seem best to leave this issue behind in a 1.3 release, and allow the other fixes to go in.
Comment #20
jsacksick CreditAttribution: jsacksick at Centarro commentedI've been troubleshooting this issue and testing the patch from comment #15 today and found out the following:
First of all, here's the list of tokens when the token module is NOT enabled:
When enabled, the list is slightly different:
When the token module is not enabled, only the token with "dashes" exists.
So with the patch in #15, printing the following would work:
But
[commerce-order:commerce-customer-billing:commerce_customer_address:country]
wouldn't.The attached patch supports both (I tested it locally by printing the 2 tokens in the order confirmation email and it worked as expected).
Comment #22
rszrama CreditAttribution: rszrama at Centarro commentedGood deal, thanks for the reroll / improvements. I've confirmed what you've said, that the address field tokens with hyphens or underscores will still be present / identified as address fields before and after the patch. Committed!
Comment #24
gregori.goossens CreditAttribution: gregori.goossens commentedBe aware,
previously working token do not work anymore : [message:message-commerce-order:commerce-customer-billing:commerce-customer-address:first_name] instead need to use [message:message-commerce-order:commerce-customer-billing:commerce-customer-address:first-name].
Unlike, specified in generate token list help (exemple for message type) :
Comment #25
ndobromirov CreditAttribution: ndobromirov at FFW commentedIs this somehow affecting 8?