Problem/Motivation
Currently, the 'get' process plugin is the only place that contains the logic needed to process row keys by unescaping '@' and determining if the key represents a source or destination property. As such, any process plugin that might take in a row key as a configuration value must instantiate a copy of the get plugin to determine the value.
Proposed resolution
Currently Row has a getSourceProperty method and a getDestinationProperty method. Lets add a generic get method which performs the same logic as the plugin, and implement the magic __get method as well to access a property and have the Row object determine if it is a source or destination property and return the correct one.
Remaining tasks
Add a get, getMultiple, and __get method to the Row Class
Write Tests
Update MigrationLookup to use this instead of Get
User interface changes
None
API changes
Add \Drupal\migrate\Row::get($property)
Add \Drupal\migrate\Row::getMultiple($properties)
Add \Drupal\migrate\Row::__get($property)
Remove plugin.manager.migrate.process dependency from MigrationLookup
Data model changes
None
Comments
Comment #2
mikelutzComment #3
mikelutzThis adds support for retrieving multiple values at once from the row.
Comment #4
mikelutzCompletely uninteresting coding standard fixes.
Comment #5
mikelutzComment #6
heddnDiscussed briefly in the migrate maintainers meeting w/ @mikelutz @maxocub @heddn and discussed removing the row interface. Also discussed the possibility of later, down the road removing entirely the Get plugin. For now leaving it.
Comment #7
mikelutzRemoved the new interface and re-rolled this patch.
Comment #8
phenaproximaBig +1 for this patch. I find nothing actually wrong with it; just minor bits and bobs.
This can just be mixed|null, and the description should say that NULL is returned if the property doesn't exist.
reset() returns FALSE if the array is empty. So we should probably translate that to NULL, like so:
return $properties ? reset($properties) : NULL
.If you cast a string to array, you get [$string]. So this can just be
$values = $this->getMultiple((array) $properties)
.Comment #9
mikelutz#8.1 - I updated the documentation.
#8.2 - getMultiple wouldn't ever return an empty array unless it were called with one, (which it can't be from the magic method). If it or ::get() were to be called with an empty array, a return value of an empty array would be appropriate.
Either way, the lookup might result in an actual boolean FALSE value, so we couldn't blanket convert boolean false to NULL anyway.Edit - Seems I didn't read your code fully, your solution would not have converted a boolean false value to null. I still think it's unnecessary though, since getMultiple will always return an array of the same length as the one it received.#8.3 - We use the if(!is_array($foo)) $foo=[$foo] pattern several places (see Get, StaticMap, and MigrationLookup). I'm not sure typecasting is simpler, (array) $foo vs array($foo) vs [$foo], but I left it as [$foo] for consistency with the other plugins for now. I will change it if there is an overwhelming desire to.
Comment #10
mikelutzBah, I was perfectly happy with this patch until phenaproxima made me look at it again. I just realized that moving the get plugin instantiation out of migration lookup means the migrate plugin manager is injected and never used, so I need to address that when I get to work.
Comment #11
mikelutzGoing to need crs too, I suppose.
Comment #12
mikelutzI expect this will need additional test updates...
EDIT - I was wrong, I got them all, woohoo!
Comment #13
mikelutzComment #14
phenaproximaOnly a few small things, and then this is ready to go. I really love the test coverage -- seems quite thorough!
Constructors (and plugin implementations in general) are not considered part of the API, so I think we can actually remove this parameter entirely.
We can remove the final sentence here, as it is repeated in the @return documentation. Also -- super nitpick time! -- can these lines be nicely formatted and wrapped to 80 characters?
Nitpick, but is there any reason not to have __get() essentially be an alias of get()? So the whole body of __get() could be
return $this->get($property)
.Should be
@param string|string[]
, since AFAIK this cannot be an array of anything but strings.Can this be string[]?
I'd say we should use ===, but I'm not sure if that, unlikely as it is, will break existing code somewhere. I guess we'll leave it for now.
This should mention that $is_stub defaults to FALSE.
Comment #15
mikelutzThe only reason for 14.3 is that ::get() will take an array or a string, while ::__get() will only ever be called with a string, so it didn't need the extra is_array check.
for 14.6, the logic was directly copied over from the plugin, I didn't want to risk changing the behavior as part of this exercise.
Comment #16
mikelutzThere you go, addressed everything, but left 14.6 alone. We can add it as a novice issue if we want.
Comment #17
phenaproximaThe tiniest thing -- "the" needs to be capitalized after the period.
No need to go back to "needs review"; this is RTBC from me once green on all backends. Awesome!
Comment #18
mikelutzThere you go.
Comment #19
alexpottWhy do do we need this magic?
Why do we need a get that supports an array when we have getMultiple()?
Comment #20
mikelutzI don't think we particularly need either to make this patch worth while. The magic method was just for the DX, as was the flexibility in the get method. Both I thought might be used in contrib, but I'm happy to tighten it up.
Comment #21
mikelutzComment #22
heddnFeedback addressed. Back to RTBC.
Comment #24
mikelutzTestbot hiccup..
Comment #25
alexpottOne thought I have is after this patch what is the point in getSourceProperty() / getDestinationProperty()? It would be handy if the document described when you should use the new methods over the existing ones.
Also the place where the methods have been added to the Row class seems inconsistent. I would have thought the two new methods would come before or after any getSource* or getDestination*
Not used.
Comment #26
mikelutzI've added some additional documentation as to when to use the methods, and I can add it to the CR as well. The new method should typically be used in process plugins to pull a property defined in a migration. These are usually source properties, but sometimes you want to process a source property and save it in the destination and use it. The use case in this patch is in MigrationLookup, where you specify the property to use as a lookup when looking up multiple migraitons. This is generally a source id, but you might want to run it through a process first like a map or default value and use it as a destination property. This was previously done by creating a copy of the get plugin and using that to retrieve the value. The patch makes that unnecessary, however destination and source plugins that need to access properties generally know whether they want a source or destination property, and they don't want to have to worry about adding a '@' to a destination property name, or escaping '@' symbols in general, so they would still use the respective retrieval methods. I think they still have use and should stay part of the API for that reason.
Comment #27
alexpottSo I guess that leads me to wondering how useful will this really be for contrib and if the flip case is true - in that it might cause some contrib / custom plugins to use the ->get() method when they should be using one of the others?
At least the new document makes it clearer about when to use the different things.
Comment #28
mikelutzThis all started with working on some refactoring around the migration lookup plugin, which has lots of issues, and high churn. There are a few places in core, and more in contrib where copies of the migration_lookup plugin is created inside of other process plugins.
I started to think that if the lookup/stub creation logic is so useful in other processes, it should probably be available as a service, or somewhere else in the api so that you aren't creating a plugin inside a plugin. That made me realize that migration_lookup was doing that with get and that I've done that with get as well in custom processes, and it just seemed logical to move the logic of that directly to the row class.
This will also support some future refactoring around migration lookup and migration executable to avoid creating get plugins when they aren't really needed.
Comment #29
heddnre #27: after reviewing the patch, the CR and the docs on the new methods, it does make sense to add this to Row. Because we use the 'get' logic all over the place in custom and contrib process plugins. And to make it generic, we need a way to get those values from either source or destination. We already had that code duplication, just it was stuffed down in Get process plugin. And in retrospect, we probably don't need the getSource or getDestination methods as public. We might even want to eventually deprecate them or see if we can move them to protected. Where the data is stored is an implementation details irrelevant to callers.
Plenty of test coverage. CR is good. And since this is to help us improve migration_lookup, let's get this in.
Comment #30
quietone CreditAttribution: quietone as a volunteer commented+1 This is really nice work and the two CRs are spot on.
I have one personal reservation and that is the potential loss of getSourceProperty and getDestinationProperty which, I think, are easier to understand than get('name') or get('@name'). Just have to wait and see on that one.
Comment #31
alexpottCommitted a9f1973 and pushed to 8.7.x. Thanks!
Fixed unused use on commit.