Problem/Motivation

I'm profiling a drush script that exports a large number of entities to CSV.

A lot of time is pent in normalizing the entities, even though only a subset of fields are required.

JSON:API's ResourceObjectNormalizer supports $context['sparse_fields'] to restrict the number of fields that need to be normalized.

However when normalizing to csv or other formats via drush or similar, there is no such equivalent. We can adopt the same approach.

Steps to reproduce

Proposed resolution

Support $context['sparse_fieldset'] in EntityNormalizer and ContentEntityNormalizer.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3599844

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

catch created an issue. See original summary.

catch’s picture

Status: Active » Needs review

catch’s picture

Title: ComplexDataNormalizer should allow normalizing a subset of fields » EntityNormalizer should allow normalizing a subset of fields
Issue summary: View changes

This doesn't work in ComplexDataNormalizer because it's recursive, but it can work one level up in EntityNormalizer.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new2.21 KB

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

catch’s picture

Status: Needs work » Needs review
catch’s picture

Issue summary: View changes

Coving both EntityNormalizer and ContentEntityNormalizer now, also added test coverage to their respective unit tests.

catch’s picture

Pushed a commit for an additional optimization.

When we have a sparse fieldset, we can only get the field items within that fieldset, instead of iterating over all properties of the entity and checking it against the list.

catch’s picture

Found another closely related optimisation which I'm adding here.

In ComplexDataNormalizer we first check if there are fields, then if there are non-internal fields - we can skip that and only check if there are non-internal fields in the first place. If there aren't any, then there won't be any fields to normalize.

catch’s picture

Before/after xhprof screenshots for that last change.