Problem/Motivation
doExportToArray() caches each exported entity, but on a cache hit it returns the whole payload again, so an entity reached N times in the reference graph is serialised N times.
Anything shared across the graph multiplies: the node author, the stores a product belongs to, a vocabulary's terms. A handful of entities referenced from everywhere can account for most of the file. On content with a wide reference graph, the export grows large enough that the download exhausts the PHP memory limit and the entity cannot be exported at all through the UI.
Proposed resolution
On a cache hit, export a reference instead of the payload: drop custom_fields and it becomes the stub shape that the importer already resolves by uuid. EntityReference.php emits exactly this when isReferenceCached() is true. Every entity is still exported in full exactly once so no data is lost.
User interface changes
None.
API changes
None. The emitted shape is already produced elsewhere in the module.
Data model changes
None.
Issue fork single_content_sync-3622552
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
alex.bukach commentedComment #4
anybodyVERY good finding @alex.bukach! That will help a lot with performance.
Comment #5
anybodyPS: Failing tests seem unrelated, I created #3624230: Fix failing tests.
Maybe you'd like to ask for co-maintainership on the module to help the maintainers pushing things forward?
Comment #6
ayoub.elmansouri commentedI have investigated and fixed the general CI pipeline/job configuration issues reported in thes ticket "3624230". However, PHPUnit was not one of them—the PHPUnit job has actually been passing successfully (green) from the very beginning.
Comment #7
nginex commentedLet me take a look
Comment #11
nginex commentedThanks for the patch and everyone involved, will put it into a new release soon
Comment #13
nginex commentedThere were a few issues during my testing, everything was fixed under https://git.drupalcode.org/project/single_content_sync/-/merge_requests/...
and
https://git.drupalcode.org/project/single_content_sync/-/merge_requests/...
it will be included to the next release as well