Performance problems on node save for nodes with a lot of field collections

Hey dpolant,

Thanks for reporting. Eight minutes is definitely a long time. Also thanks for your patch; I'll apply it and take a look.

Hello again dpolant,

I wanted to ask what performance increase you've seen applying your patch. I also wanted to make sure I understand the problem correctly; I haven't seen a significant performance increase within my local site. I created a content type with 30 text fields and 5 field collection fields. I added 40 fields to each field collection, 20 text fields, and 20 long text fields. Then I enabled all fields for translation within Lingotek. See node_fields.png and fc_fields.png for screenshots. Here are my test times.

Without the patch:
6.13 seconds
5.62 seconds
5.88 seconds
5.46 seconds
5.51 seconds
5.41 seconds
5.64 seconds
5.59 seconds
5.35 seconds
5.80 seconds
Average: 5.6 seconds

With patch:
5.73 seconds
5.82 seconds
5.69 seconds
5.57 seconds
5.56 seconds
5.29 seconds
5.62 seconds
5.23 seconds
5.37 seconds
5.23 seconds
Average: 5.5 seconds

I also used xhprof to profile the page creation. See lingotek_node_load_default.png. My xhprof report had less calls to lingotek_node_load_default (only 6 without the patch applied), but I may not have it setup correctly.

Your patch contains good logic, but I want to make sure I'm replicating the use case accurately.

@jbhovek: Out of interest, with your test scenario what is the average page request time if the Lingotek functionality is disabled entirely?

dpolant,

I wrote a script that programmatically creates 100 field collections containing 10 text fields, equaling 1000 fields. I'm seeing some performance increase with this scenario.

Without patch:
31.96 seconds
31.35 seconds
31.13 seconds
31.58 seconds
31.18 seconds
30.89 seconds
31.29 seconds
31.30 seconds
30.77 seconds
32.09 seconds

With patch:
17.66 seconds
15.95 seconds
15.55 seconds
15.51 seconds
15.41 seconds
14.95 seconds
15.54 seconds
15.29 seconds
15.27 seconds
15.56 seconds

DamienMcKenna, the times are better without Lingotek.

Without Lingotek:
14.92 seconds
14.16 seconds
14.06 seconds
13.71 seconds
13.20 seconds
13.75 seconds
13.26 seconds
13.70 seconds
13.43 seconds
13.34 seconds

@jbhovik: Thanks for the extra stats. It seems clear that for complicated sites the module could cause performance problems.

Would you mind updating the script? We could help expand it to also test Paragraphs, etc.

@dpolant and @DamienMcKenna,

The other place Lingotek makes a lot of queries is in the lingotek_entity_load() function. This may account for the extra time added by Lingotek. These queries are only run if it's an admin page, drush, or cron.

I wrote a script that adds 10 text fields to an existing paragraph bundle ("my_paragraph_bundle"). I wrote a second script that adds paragraph fields to a content type (paragraph_content_type). Here are the times.

Without patch:
4.90 seconds
5.23 seconds
5.38 seconds
5.98 seconds
5.13 seconds

Average: 5.32

With patch:
4.56 seconds
5.73 seconds
4.96 seconds
5.35 seconds
5.45 seconds

Average: 5.21

The two scenarios are pretty close. I didn't perform a "without Lingotek" test because each test took quite a while since you have to click the "Add new paragraph" buttons. I bet the times would be similar without Lingotek.

I'm attaching the scripts (ran in devel) I used to create the test content. For the paragraphs, make sure you create a paragraph bundle named "my_paragraph_bundle" so addFieldsToParagraphBundle.txt and addParagraphsToNodes.txt have something to run against. The "delete" scripts remove the created fields if you're so inclined. Thanks again for your patch--we're currently QAing it.

@dpolant and @DamienMcKenna,

As an update, we don't see any unwanted effects from applying your patch. Feel free to use it in environments. We would still recommend testing it out on a test environment first. We'll include the patch in our next release.

@dpolant and @DamienMcKenna,

Just so you know, we did find a problem with the patch in QA. The node_presave hook code was causing edited nodes to not register as edited, so we moved that logic into lingotek_entity_changed(), since this function is where the Lingotek module checks for edits to content. Thanks again for your contribution--we've released your code in 7.23.

@jbhovik: That's awesome, thanks!