From original issue #2173371: out of memory on large CSV exports of submissions

The problem occurs because entityform_views_pre_view() is executed each batch segment rather than once per export.

_entityform_view_add_all_fields() is then fired off for each batch and adds the fields without checking if they're already present.

Comments

tedbow’s picture

Here is patch that checks if fields have already been added.

tedbow’s picture

Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 1: entityform-autofield_duplicates-2340989-1.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 1: entityform-autofield_duplicates-2340989-1.patch, failed testing.

tedbow’s picture

Status: Needs work » Needs review
StatusFileSize
new3.64 KB
imclean’s picture

Thanks tedbow, patch applies cleanly and works well. I've successfully exported 500+ submissions with around 120 fields each in a minute with no duplicates.

Just a note that the first problem was the PHP process running out of memory.

This was circumvented by using paging (offset + specified number of items) and smaller segment sizes of 1 to 3. Only then were we able export the CSV and see the duplicate fields.

  • tedbow committed 93984b2 on 7.x-2.x
    Issue #2340989 by tedbow: Fixed Fields added multiple times when using...
tedbow’s picture

Status: Needs review » Fixed

@imclean thanks for your help on this. Committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.