Problem/Motivation
Whever I try the upgrade, it hangs at 38%. Checking the error log, I see multiple entries with the message "Missing filter plugin: filter_null."
I know that the PHP plugin was enabled on the old site, so not sure if it's related to that. I did try enabling the contrib version of the D8 PHP filter, but php still showed up as a missing dependency, and there was no change in the import.
Proposed resolution
When replacing a filter with filter_null, be sure we are stripping out any settings that may fail a validation check.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
In previous versions of Drupal 8, if a corresponding D8 filter was not found for a D6/D7 filter, it would get mapped to filter_null, but the settings would be retained. This "mixed-mapping" was a bug, which could later lead to configuration schema validation errors. The new behaviour is to instead drop the filter settings, with a message directing users to either re-configure the format by hand, or attempt the migration again with a D8 version of the dropped filter in question.
Comment | File | Size | Author |
---|---|---|---|
#52 | 2946889-52.patch | 4.25 KB | jofitz |
#52 | interdiff-2946889-46-52.txt | 2.27 KB | jofitz |
#46 | 2946889-46.patch | 4.17 KB | jfussion |
#46 | interdiff-41-46.txt | 1.1 KB | jfussion |
#43 | 2946889-43.patch | 1.21 KB | jfussion |
Issue fork drupal-2946889
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
jcnventura CreditAttribution: jcnventura at Wunder commentedDuplicate of #2618332: Better handle replacement of missing filters with filter_null
Comment #3
jcnventura CreditAttribution: jcnventura at Wunder commentedComment #4
NiklasBr CreditAttribution: NiklasBr commented#2618332 didn't fix the issue as described. Which is why I opened a new one.
Comment #5
heddnReviewed this in the migrate maintainers meeting. While this is important, it isn't migrate critical. Removing that tag.
Comment #6
NiklasBr CreditAttribution: NiklasBr commentedQuestion.
The original issue (#2618332) was/is tagged Migrate critical. This issue is a 1:1 copy of that issue because that issue did not resolve the reported bug, why is it no longer critical?
Comment #7
heddnThat issue was about how to handle null filters so it didn't skip importing any when it encountered a bad one or two. Additional logging was added as well. But it doesn't mean that null filters were all be fixed. There will still be the case were filters won't exist on the d8 site and it will fallback on the default null filter.
Rather than copy/pasting the IS from that previous issue, can you explain what you are seeing and what you were hoping to see?
Comment #8
NiklasBr CreditAttribution: NiklasBr commentedI am seing exactly the same error message as the issue summary says, everything as far as I can see is identical to the previous issue. I see no reason not to copy it. I see:
And then the migration fails. I was hoping to see no error messages.
Comment #9
heddnThose are probably coming from shortcode, markdown or codefilter.
Comment #10
NiklasBr CreditAttribution: NiklasBr commentedMaybe. I don't know. How would the message "Missing filter plugin: filter_null." be related to those modules and how would we know and why does it cancel all migrations?
We do not use them in Drupal 6 and do not plan on using them in Drupal 8.
Comment #11
heddnIt shouldn't cancel all migrations. I'm pretty sure the canceling is another issue unrelated to these messages in the logs.
Comment #12
NiklasBr CreditAttribution: NiklasBr commentedThis is the only error message, then it stops. Maybe there is some other issue, but the log does not show any other error.
Comment #13
NiklasBr CreditAttribution: NiklasBr commentedIssue still in D8.6.x-dev.
Comment #15
jcnventura CreditAttribution: jcnventura at Wunder commented@NiklasBr, what exactly do you mean with 'migration fails'? Can you provide the output from this 'select * from migrate_message_d6_filter_format;'.
Comment #16
jcnventura CreditAttribution: jcnventura at Wunder commented@heddn, if I can hijack this issue a bit, since it clearly seems to be the best one. My scenario is the following:
data-picture-mapping='__id__'
to a responsive format.[error] Missing filter plugin: filter_null.
And in migrate_message_d7_filter_format:
Filter picture could not be mapped to an existing filter plugin; defaulting to filter_null.
The issue here is that since picture is now in core, it should be core's task to do a proper filter migration. I know that at one point this will be handled by #2822389: Allow responsive image style to be selected in Text Editor's image dialog (necessary for structured content), but until that point, there should be a 'safe' filter that would not remove all text, but simply show it as is. This would probably start by using a mapping between 'picture' and 'filter_none' (as opposed to 'filter_null').
Comment #17
aiphesI get this Alert on dblog : Plugin de filtre manquant : filter_null. On node edit mode and on this kind of ting:
quickedit/attachments?_wrapper_format=drupal_ajax
Then it break Coffee with message inside it "Could not load data, please refresh the page" (see https://www.drupal.org/project/coffee/issues/3000611).
Using Drush 8.1.6 and Drupal 8.6.1.UPgrading from D6 with drush.D8 PHP filter enabled.
Remove PHP filter from the incrimined filter doesn't change things, Coffee work randomly.
Other information, unable to edit a field settings in admin/structure/types/manage/XXX/form-display
In DB, migrate_message_d6_filter_format and migrate_message_upgrade_d6_filter_format tables:
Comment #18
siramsay CreditAttribution: siramsay as a volunteer commentedI was getting this same error "Missing filter plugin: filter_null."
#16
"Only after saving the format without any changes in /admin/config/content/formats/manage/full_html... ". worked form me.
+ removing/disabling some other filters too such as PHP filter.
Comment #20
drupalninja99 CreditAttribution: drupalninja99 commentedIs there a way to change the error from "error" to "warning"? Right now it stops the migration, I think this should be more of the "warning" category that won't interrupt migrations.
Comment #22
Wim LeersThis is not yet perfect, but it does do what the issue title says: it improves the situation.
The reason that some filters can cause a migration to completely fail is that there may be config schema violation errors when a D6/D7 filter's "settings" are migrated when that filter does not exist on the destination D8 site. The D8 site will apply config schema validation, and the
filter_null
filter does not have any settings, therefore any settings being present causes it to fail completely.So the only feasible solution is quite simple: drop those settings and warn the user explicitly (which should've already been the case, but alas), like so:
This still results in body fields (or any other field using formatted/processed text) that uses a text format with at least one
filter_null
to show up as empty. But at least now the migration is successful, and you can go and edit those Drupal 8 text formats.Comment #24
Wim LeersComment #25
mikelutzThis seems a sane and obviously correct issue and fix.
Setting to NW for tests.
Comment #26
Wim LeersComment #28
quietone CreditAttribution: quietone as a volunteer commentedYes, nice fix. I like the more informative error message.
Filter @plugin_id could not be mapped to an existing filter plugin; defaulting to @fallback and dropping all settings. Either redo the migration with a viable successor, or modify the text format after the migration to remove this filter if it is no longer necessary.
I'd prefer if the comment didn't use the future perfect tense but that is preference. The comment is correct.
I found only one nit, which I fixed.
Needs trailing comma.
Comment #30
quietone CreditAttribution: quietone as a volunteer commentedSorry, bad patch. Will fix later today.
Comment #31
quietone CreditAttribution: quietone as a volunteer commentedRight, try this again.
Comment #32
Wim LeersGiven you're a Migrate maintainer, and you only fixed a nit, I think you're eligible to RTBC this 🤓
Comment #33
quietone CreditAttribution: quietone as a volunteer commented@Wim Leers, thanks. I was just waiting to make sure I hadn't make another mistake.
This is good to do. My review is in #28
Comment #34
Wim Leers🥳
Comment #35
heddnSorry to kick this back on pedantics.
Nit (and yes I know that's how we did this before...) But we don't typically translate error messages. Can we use sprintf instead?
Instead of
viable successor
, could we say "format that exists on the destination site", or whatever we want them to do? I'm not sure we are explicit enough for folks to know what to do when we just say "viable successor". Alternatively, add a docs page and refer to it via a URL.Comment #36
Wim Leers#35++ 🤓😊
How about:
Filter @plugin_id could not be mapped to an existing filter plugin; defaulting to @fallback and dropping all settings. Either redo the migration with the module installed that provides an equivalent filter, or modify the text format after the migration to remove this filter if it is no longer necessary.
Comment #37
heddn#36++ Let's go with that wording, unless someone speaks up.
Comment #38
Wim Leers#37 👍 🙂 — tagging to let somebody new to core update the patch!
Comment #39
quietone CreditAttribution: quietone as a volunteer commentedAdding DrupalSouth tag.
Comment #40
jfussion CreditAttribution: jfussion as a volunteer commentedI'm working on this one. (DrupalSouth 2019)
Comment #41
jfussion CreditAttribution: jfussion as a volunteer and at Pantheon commentedI have made changes from #36.
Thanks @jibran for helping me with creating the patch!
Comment #43
jfussion CreditAttribution: jfussion as a volunteer and at Pantheon commentedUpdated the patch from #41 to fix the failing test.
Thanks @dpi for the help.
Comment #44
jfussion CreditAttribution: jfussion as a volunteer and at Pantheon commentedComment #45
quietone CreditAttribution: quietone as a volunteer commented@jfussion, thanks for the patch. I noticed the size of the patch in #43 was much smaller than in #41. And after taking a look it seems that the interdiff is the patch and the patch is the interdiff.
Comment #46
jfussion CreditAttribution: jfussion as a volunteer and at Pantheon commented@quietone, thanks for the review. Here are the update interdiff and patch.
Comment #47
jfussion CreditAttribution: jfussion as a volunteer and at Pantheon commentedComment #48
heddnNit: no need to cast to string.
This could probably also use the same sprintf pattern as aboove. As I don't think we want to just hard code filter_null when we have the option to make this dynamic based on the data provider.
Comment #49
kostyashupenkoComment #50
Wim LeersNeeds another reroll, for the additional nits described in #48.
Comment #51
NiklasBr CreditAttribution: NiklasBr commented@jcnventura I gave up and tried now again with 8.8.0 available, but migrations still fails/choke/stop/crash on null filters. It stops.
Comment #52
jofitz CreditAttribution: jofitz at jofitz commentedAddressed @heddn's code review comments from #48.
Comment #53
heddnAll feedback now addressed. Thanks for picking up the nits.
Comment #57
webchickOooh, I love the error message improvements here very much!!
This bug fix represents a behaviour change from previous versions, so I'm tagging for release notes and making my best attempt at a release notes snippet.
@NiklasBr, could you try testing the 8.8.x-dev release (in about 12 hours) and see if it now resolves your issue? And if not, open a specific support request that we can try to dig in on?
Committed and pushed to 9.0.x; 8.9.x; 8.8.x. Thanks!
Comment #58
webchickComment #60
donm CreditAttribution: donm as a volunteer commentedIn case it helps others: Cleaning up log messages re "Missing filter plugin: filter_null." Context: Currently Drupal 8.8.1, with a database migrated out of D7 into D8 in whatever versions were current 2 years ago. In my case, I was getting 4 such log entries, and it seems that they were generated from 2 separate sources:
Comment #61
brooke_heaton CreditAttribution: brooke_heaton as a volunteer commentedI'm getting this error on a Drupal 7 to Drupal 8 upgrade:
Is there a way to get around this issue? Can a default filter be set?
Comment #62
brooke_heaton CreditAttribution: brooke_heaton as a volunteer commentedStill seeing this error on 8.9.16.
Comment #64
ressa CreditAttribution: ressa at Ardea commentedI saw "Missing filter plugin: filter_null" alerts in the log after a migration from Drupal 7 to Drupal 9, and saving all text formats seems to have solved it. I switched
fallback_format
from the old ('4') to new ('plain_text') with this:Comment #65
Grevil CreditAttribution: Grevil as a volunteer and at DROWL.de commentedSmall hint if anyone is still having this issue during a Drupal 7 to Drupal 9 Migration:
You can skip the empty filters through migration, by adding the "skip_on_empty" plugin in your "upgrade_d7_filter_format" migration yaml.
Either by directly inserting it in the process pipeline:
Or through the usage of hook_migrations_plugins_alter():