Problem/Motivation
I'm profiling a drush script that exports a large number of entities to CSV.
A lot of time is pent in normalizing the entities, even though only a subset of fields are required.
JSON:API's ResourceObjectNormalizer supports $context['sparse_fields'] to restrict the number of fields that need to be normalized.
However when normalizing to csv or other formats via drush or similar, there is no such equivalent. We can adopt the same approach.
Steps to reproduce
Proposed resolution
Support $context['sparse_fieldset'] in EntityNormalizer and ContentEntityNormalizer.
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
Comments
Comment #2
catchComment #4
catchThis doesn't work in ComplexDataNormalizer because it's recursive, but it can work one level up in EntityNormalizer.
Comment #5
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #6
catchComment #7
catchCoving both EntityNormalizer and ContentEntityNormalizer now, also added test coverage to their respective unit tests.
Comment #8
catchPushed a commit for an additional optimization.
When we have a sparse fieldset, we can only get the field items within that fieldset, instead of iterating over all properties of the entity and checking it against the list.
Comment #9
catchFound another closely related optimisation which I'm adding here.
In ComplexDataNormalizer we first check if there are fields, then if there are non-internal fields - we can skip that and only check if there are non-internal fields in the first place. If there aren't any, then there won't be any fields to normalize.
Comment #10
catchBefore/after xhprof screenshots for that last change.