Problem/Motivation

doExportToArray() caches each exported entity, but on a cache hit it returns the whole payload again, so an entity reached N times in the reference graph is serialised N times.

Anything shared across the graph multiplies: the node author, the stores a product belongs to, a vocabulary's terms. A handful of entities referenced from everywhere can account for most of the file. On content with a wide reference graph, the export grows large enough that the download exhausts the PHP memory limit and the entity cannot be exported at all through the UI.

Proposed resolution

On a cache hit, export a reference instead of the payload: drop custom_fields and it becomes the stub shape that the importer already resolves by uuid. EntityReference.php emits exactly this when isReferenceCached() is true. Every entity is still exported in full exactly once so no data is lost.

User interface changes

None.

API changes

None. The emitted shape is already produced elsewhere in the module.

Data model changes

None.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

alex.bukach created an issue. See original summary.

alex.bukach’s picture

Assigned: alex.bukach » Unassigned
Status: Active » Needs review
anybody’s picture

Issue tags: +Performance

VERY good finding @alex.bukach! That will help a lot with performance.

anybody’s picture

PS: Failing tests seem unrelated, I created #3624230: Fix failing tests.

Maybe you'd like to ask for co-maintainership on the module to help the maintainers pushing things forward?

ayoub.elmansouri’s picture

I have investigated and fixed the general CI pipeline/job configuration issues reported in thes ticket "3624230". However, PHPUnit was not one of them—the PHPUnit job has actually been passing successfully (green) from the very beginning.

nginex’s picture

Assigned: Unassigned » nginex

Let me take a look

nginex changed the visibility of the branch 3622552-export-entity-once to hidden.

nginex changed the visibility of the branch 3622552-export-entity-once to active.

  • nginex committed 6fe974e4 on 1.4.x authored by alex.bukach
    Issue #3622552 by Alex Bukach: Referenced entities are exported once per...
nginex’s picture

Assigned: nginex » Unassigned
Status: Needs review » Fixed

Thanks for the patch and everyone involved, will put it into a new release soon

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

nginex’s picture

There were a few issues during my testing, everything was fixed under https://git.drupalcode.org/project/single_content_sync/-/merge_requests/...

and

https://git.drupalcode.org/project/single_content_sync/-/merge_requests/...

it will be included to the next release as well