Problem/Motivation

Obtaining accurate source counts may be time-consuming in some instances, slowing dependency checking and status reporting, particularly for non-SQL sources. Usually one is not migrating directly from live data, so these counts do not change often. Source plugins should provide the option of caching the counts.

Proposed resolution

If caching is enabled and a cached count is available, return that count. Otherwise, do a full count of the source (saving to cache if caching is enabled). A refresh option should be available to force a full count even when a cached count is present.

Remaining tasks

All of them.

User interface changes

N/A to core (Migrate Plus will make use of it in contrib).

API changes

Right now SqlBase::count() does a direct count query. To avoid implementing caching in each concrete source plugin, the D7 approach was for the base plugin to implement caching in count(), invoking computeCount() (implemented by the specific source plugin) only when necessary.

Comments

benjy’s picture

benjy commented

I notice we have a caching implementation for count in Drupal\migrate\Source i'm guessing that came from D7? I can't even tell if it's used anyway at this point.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Active » Closed (duplicate)

Ah, I was looking at the source plugin level, didn't see it was implemented at Source. Move along, nothing to see here...

benjy’s picture

benjy commented

You might be interested in #2427335: Combine legacy Source class into SourcePluginBase where i'm combing the legacy Source class into SourcePluginBase so count will be in the source plugin as you expected.