Problem/Motivation
Currently the way we handle exceptions on configuration errors is inconsistent in our process plugins. Sometimes an exception is thrown in the constructor, sometimes we throw exceptions for invalid configurations in the constructor, sometimes we don't do it until the transform method is called. There is some inconsistency in which exceptions get thrown as well.
All plugins should do the following:
If the plugin configuration is invalid, and this can be determined from the configuration alone with no value to transform, the plugins should throw a consistent exception in the constructor, specific exception to be determined.
If the configuration is valid, but certain inputs are not, then the plugin should determine this in the transform method and throw a consistant exception there, exception tbd
In most cases, moving or changing the exception will be a breaking change, so we will likely have to add @trigger_errors in all the places we want to put new exceptions warning people that we will be adding the new exceptions in the next major version, Drupal 10 currently if we can get this into Drupal 9.4. We should have a version of the patch ready for D10 as well that actually adds the excpetions.
Steps to reproduce
Proposed resolution
Modify these process plugins to use InvalidArgumentException.
ProcessField
Explode
FormatDate
MachineName
MakeUniqueBase
StaticMap - Breaking change, add @trigger_error in the constructor if bypass and default value are set.
Remaining tasks
Review
Commit
Update Change record
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #34 | 3247950-34.patch | 17.21 KB | quietone |
| #34 | interdiff-31-34.txt | 1.89 KB | quietone |
| #31 | 3247950-31.patch | 17.42 KB | quietone |
| #31 | interdiff-29-31.txt | 1.61 KB | quietone |
| #30 | interdiff_27-29.txt | 1.58 KB | murilohp |
Issue fork drupal-3247950
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3247950-callback-process-plugin
changes, plain diff MR !1389
Comments
Comment #3
mikelutzComment #4
quietone commentedNice cleanup! and as a bonus this simplifies testCallbackExceptions
I guess this needs a CR so people know to catch a different exception?
Comment #5
mikelutzI've added a change record, I've also moved the exception from an overridden constructor into the transform method. While all of this, (throwing a different exception and throwing the exception in a different place is technically a behavior change, it brings this plugin in line with the other migration plugins. During any normal migration workflow, whether through the core ui or drush or with the common contrib migrate modules, this change should be completely non-disruptive. I can't think of any practical way that this change could cause a problem. You would have to be running a custom executable, specifically handling this plugin separately for some reason which I can't fathom. Even so, the new exception would still be caught and handled correctly along with the other plugins now, and I consider the fact that it sent out the wrong exception in the wrong place to be a bug that this is fixing.
Comment #6
quietone commented@mikelutz, thanks!
I applied the MR locally, read the code and ran the tests. It all looks good to me.
Then I read the CR and thought it could use a little improvement in the way it explains what Callback does that is different than other process plugins.
All together I think this is good to go.
Comment #7
benjifisherWe have discussed these exceptions before. The way I remember it, @heddn convinced me that we should throw different exceptions in different cases.
If you can tell from the configuration alone that there is a problem, then throw an
\InvalidArgumentExceptionfrom the constructor. This has to be fixed in the code. It will fail for every row, and there is no point in throwing an exception (and logging a message) for every row.If it depends on the value passed to the
transform()method, then throw aMigrateExceptionthere. Some rows may fail, others may not.Found it: #2882276-71: Extended callback process plugin to call functions with multiple parameters is pretty close to what I just said. Search that issue for "InvalidArg" and you will see it in a few other comments, including #69, which refers to #2936463-27: Throw exception when "callable" plugin is called with undefined on invalid callback where @heddn asked for
InvalidArgumentExceptionin theCallbackconstructor.On #2882276 we also discussed whether to use something other than
InvalidArgumentException. @heddn suggested "invalid plugin exception", but the closest I could find wasInvalidPluginDefinitionException. No one made a follow-up issue.If you agree with this approach, then instead of using
MigrateExceptioninCallback, we should move the first test inExplodeto the constructor and change the exception.Maybe also
FormatDate. I do not know why thetransform()method returns an empty string in some cases before validating the configuration.MachineNamethrowsMigrateExceptionfrom the constructor.MakeUniqueBaseis the first one where I have noticed an@throwcomment in the doc block. I guess we should clean that up. The validation could be moved to the constructor, and we could use whatever exception we agree on for constructors.Substris similar.MigrationLookuphas a couple of@throwcomments after{@inheritdoc}. We are not supposed to do that, right?Comment #9
quietone commented@benjifisher, thanks for doing the research on this. On reflection, what heddn says seems right to me.
With that in mind I started working on a patch, updating the process plugins mentioned in #7. I've used InvalidArgumentExcpetion because I haven't put any thought into if it should be that or InvalidPluginDefinitionException. I also searched for all process plugins in core that have an throws statement. There were 12 to check and of those only ProcessField is a candidate for a change. The patch does not do that one or MakeUniqueBase.
If anyone has a custom process plugin that creates one of these process plugins and is catching the MigrateException, this change would cause it to break.
Regarding the @throws after @inheritdoc, the coding standards for inheritdoc do not explicitly say this should not be done.
Comment #10
danflanagan8@quietone, that patch looks really good.
I wonder if
static_mapshould be updated too. It throws an exception ifdefault_valueandby_passare both set...but only if the value does not get mapped. I would argue that the exception should get thrown in the constructor since it would make it easier to nip this problem in the bud.Other than that it seems like we still have to move exceptions to the constructor in:
substrMakeUniqueBaseI'll set to NW.
Also tagging for IS update, since the direction of this issue has changed dramatically.
Comment #11
danflanagan8Comment #12
mikelutzI think to move some of these we are going to have to ease into it. static_map, in particular. we need to @trigger_error in the constructor if bypass and default value are set, warning that having both is unsupported and will throw whatever exception in Drupal 10, and then add the exception in D10. Because danflangan8 is right, we should throw an exception earlier, but if all your values map, you can currently get away with it, so adding the exception is a breaking change.
Will need Change Record(s) too.
Comment #13
mikelutzComment #14
quietone commentedAdd in core/modules/field/src/Plugin/migrate/process/ProcessField.php and the related test.
I think that leaves static_map and MakeUniqueBase to resolve.
Comment #16
quietone commentedRetesting because the test failure was due to #3255836: Test fails due to Composer 2.2 which has been fixed.
Comment #17
danflanagan8The changes for
core/modules/field/src/Plugin/migrate/process/ProcessField.phplook great. Thanks, @quietone!Setting back to NW to address these.
Comment #18
quietone commentedHere is a first attempt at static_map and make_unique_entity_field.
Comment #19
danflanagan8Getting close!
In addition to the "custom command failure", there are a couple things that caught my eye.
I don't think we want to remove this since the current behavior is to throw an exception. It would make more sense to change
MigrateExceptiontoInvalidArgumentExceptionnow and then remove this in 10.0.0. when the constructor can start throwing the exception.Should this go in the constructor of
MakeUniqueBase? There might be tricky BC implications with this one since it's a base class.Comment #20
murilohp commentedHere's a new patch fixing the coding standards from #18, I'll leave this issue in NW since we have to fix #19.
Thanks!
Comment #21
murilohp commentedAddressing the changes from #19 on this new patch, and moving back to NR thanks!
Comment #22
danflanagan8The
static_mapstuff in #21 looks good to me.As far as the
MakeUniqueBasestuff, in #19 I was hoping to get a little discussion going as opposed to asserting that the change should be made. From #19:Currently, the
InvalidArgumentExceptionsget thrown by thetransformmethod ofMakeUniqueBase. The intention certainly appears to be that extensions ofMakeUniqueBasedeclare theexists()method and let the base class handle thetransformmethod. Buttransformis notfinal, so there's nothing stopping anyone from overridingtransformtoo. Even if that doesn't make much sense.I *think* from a BC standpoint, I'd like to see deprecation notices in the
MakeUniqueBase::__construct()and leaveMakeUniqueBase::transformuntouched until10.0.0. Maybe we could even maketransformfinalin10.0.0. I have no idea how that would work though.Also, let it be know that I'm happy to be wrong about any of this! Leaving as NR to get additional input.
Comment #23
quietone commented@danflanagan8, I think your right.
#22. This makes the suggested changes. Tests pass locally but I am not running the tests because the deprecation notice for MakeUniqueBase needs work. I've had a long day and just not up to thinking up some text.
Comment #24
quietone commentedComment #25
danflanagan8Setting back to NW since @quietone indicated a desire to update the exception message. Currently it is:
Having a non integer length position is deprecated in drupal:9.4.0 and will throw an InvalidArgumentException in drupal:10.0.0.I guess I would propose (with little conviction mind you!):
Using a non-integer value for length is deprecated in drupal:9.4.0 and will throw an InvalidArgumentException in drupal:10.0.0. See https://www.drupal.org/node/3248473It looks like the deprecation notice in the
static_mapconstructor needs the "See" added too.Comment #26
quietone commented@danflanagan8, thanks! I forgot to come back to this.
This patch uses the text from #25 and add the link to the change record to the deprecation message in StaticMap.
Comment #27
murilohp commentedHey, just fixing the code standards here.
Comment #28
danflanagan8Very close to the finish line here! I'm having second thoughts about the following change to
static_mapthough:I'm the one that proposed this back in #19, but now I'm pretty sure this line should not be changed prior to 10.0.0. MigrateExceptions are handled gracefully, while InvalidArgumentExceptions cause the whole migration to quit immediately. We are changing MigrateException to InvalidArgumentException in a number of other places where we know that every single row will fail. So at that point there's arguably no behavior change.
In this
static_mapcase though, this exception will not (likely) happen on every single row. That's because the configuration currently only gets validated if the source value is empty, not on every single row. Therefore there is a significant behavior change possible if we change from MigrateException to InvalidArgumentException before 10.0.0. We should continue throwing a MigrateException until 10.0.0.So sorry about my bad advice in #19. But I think this is going to be the last change required.
Comment #29
murilohp commentedNo worries @danflanagan8! Here's a new patch.
Comment #30
murilohp commentedOoops forgot the interdiff
Comment #31
quietone commentedI was reviewing this and saw that I left the changed exceptions in MakeUniqueBase::transform. See #22. The exceptions will be changed in D10.
I did a little work on the IS as well.
Comment #32
quietone commentedI've updated the change record, it could use reviews.
Comment #34
quietone commentedArg, I got called away and forgot I hadn't adjusted the test.
Comment #35
danflanagan8I think the patch is ready.
I checked out the change record and made a couple three little edits. But I have a question about this statement from the change record:
Is that true? I thought that an InvalidArgumentException brought everything to a grinding halt. I see this block of code in
MigrateExcecutable::import():It looks to me like InvalidArgumentException is not caught when thrown by
MigrateExecutable::processPipeline. I don't have any "real" migrations at the ready though.Comment #36
joachim commentedI'm not really following why we're throwing InvalidArgumentException instead of InvalidPluginDefinitionException.
Is it because InvalidPluginDefinitionException is for cases where the plugin definition in the annotation is incorrect, as opposed to the configuration that's passed to a plugin instance?
Comment #37
quietone commented@joachim, I think the answer is in comment #7 of this issue, the links in the issue point to original point being made by heddn.
Comment #38
joachim commentedI read that, but what it says about the exception class is this:
> On #2882276 we also discussed whether to use something other than InvalidArgumentException. @heddn suggested "invalid plugin exception", but the closest I could find was InvalidPluginDefinitionException. No one made a follow-up issue.
and I don't really understand. If InvalidPluginDefinitionException isn't correct because this is about plugin *configuration* rather than *definition*, it seems to me we need an InvalidPluginConfigurationException in the core plugin component. What does the comment about a follow-up mean?
Comment #39
joachim commentedJust spotted that Libraries module has an exception class exactly for this purpose: https://api.druphelp.com/api/libraries/src%21Plugin%21MissingPluginConfi...
Comment #40
mikelutzI swear I didn't make this up, but looking at it, I have no idea where I got this from..
If we are going to throw invalidArgumentExceptions in the constructor, we should probably catch that when we create the plugins, and gracefully reset the migration status, and bow out nicely...