When testing various migrations. It is too easy to jam up when the file attachments are not present at the URLs in the XML file. (whether 404, server is missing, etc)
This occurs when the import imagefield is selected but images cannot be loaded by url.
in consideration of #2742279: Rewrite local link/image references in content, switch image content domains we should be able to replace these remote URL domains with a new field in the initial config for optional replacement domain (eg. wordpress staging domains reachable from our box)
Try to import anything with file attachment / featured image with unavailable URL
skip on error for missing file attachment / image URLs.
include a replacement domain URL field in the config.
i think this should be configurable ideally, have timeout and so on.
Not certain. Maybe, if the image field is not selected, do not create the attachments migration at all, there is no if statement here. Unlike the tag_vocabulary right below it. If so, then it would need to not be a dependency for content post and content page.
// Set up the attachment migration.
$this->attachmentID = $this->configuration['prefix'] . 'wordpress_attachments';
$migration = $this->createEntityFromPlugin('wordpress_attachments', $this->attachmentID);
$migration->set('migration_group', $this->configuration['group_id']);
$process = $migration->get('process');
$process['uid'] = $this->uidMapping;
$migration->set('process', $process);
$migration->save();
Improve the UI message. perhaps as a placeholder until this is improved.
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
hongpong commentedComment #3
ressaYes, I ran into this during migration testing with FakerPress generated content, which served its purpose, generating different variations of non-importable image data. It would be nice if dirty data does not cause the migration to a full stop, but alerts the user that there's a problem, whatever is the easiest and simplest method.
3 out of 11 items failed due to malformed data, and after deleting those three items, the import completed. Posts with this kind of image data failed, stopping the migration hard (copied out from each
<item>):The error reports in the migration:
Comment #5
hongpong commentedI have a possible code for processing these fetches better. Can you run with roughly the same setup as before? Will try this MR47 .. https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/47 Got this spun up w Claude sonnet.
Comment #6
hongpong commentedokay this will also help for logging a great deal, it now tells missing hostnames etc in watchdog, may also help #2742279: Rewrite local link/image references in content, switch image content domains and fatal date error #3516516: DatabaseExceptionWrapper: Invalid datetime format
https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/47... i would say give that a try and see what you get including in the logs (watchdog etc). the config for the module has been updated, in the new /docs/ file i have the "drush cim" command to add the new config settings that control logging level.
i have not tried it in drush but its good in the wizard UI. I need to head out for a while but I hope this helps move things.
Comment #7
ressaThank you so much, this is a very nice improvement, and I get great feedback now with Drush.
I can warmly recommend creating a lot of test content with FakerPress (with more or less dirty data) for testing purposes and combine it with the amazing DDEV snapshot, to allow very speedily restoring a fresh instance within seconds, and then run the whole shebang again with Drush :)
If I run a migration with dirty data, I get actionable feedback, like below, and I can fix these three instances:
<wp:meta_value><![CDATA[a:1:{s:17:"svg-not-supported" ...<wp:meta_value><![CDATA[raw]]></wp:meta_value><wp:meta_value><![CDATA[a:0:{}]]></wp:meta_value>The migration still stops hard though, if there's a problem ... would it be possible to simply log these problems, but let the migration continue? Or maybe that's in the pipeline for the solution here?
...
Also, about the improved feedback: When I run the import I see these notices, about inline images I think. For some reason, I need to run
drush migrate:import --alltwice to get attachments import to complete:Also, it seems like a new hard blocker has been introduced. I am not sure what the cause is, since the source data has previously completed a migration:
The attachment post challenge above blocks progress from here on, to finalise the import of the comments, and completing the migration:
Status:
Comment #8
hongpong commentedIn theory yes that should be what it does, it should just skip. I think it is possible that 'error' level alerts will stop things cold, that is how the configuration validator works but I haven't tried with the attachments yet. I just tested this MR to make sure that the logging was coming thru. thanks for taking it for a spin and updating the documentation carefully.
I feel we may want to commit this MR 47 if it seems like it doesn't make things worse - but if it is halting just because an image can't be retrieved we do not want that to be happening. They should just be "Warning" not "error" in PSR terms.
in addition re the domains can you try looking at replacement_domain: '' in wordpress_attachments.yml? i think its not implemented in the wordpress migrate UI GUI but it could be relevant here.. (this was auto generated with claude and i have not tested yet).
replacement_domain "constants/replacement_domain" could not be parsed.this seems like something was incorrect.oh looking at this more carefully. i think these should not be in quotation marks. in the wordpress_attachments.yml.
I will also mention that this test XML - the attachments are all valid there - https://gitlab.com/HongPong/wordpress-test-imports/-/blob/v1.0.0/wp-impo...
I just tried reworking this stuff but no dice (re the constants/replacement domain issue, it clearly needs to be redone. the variable is not passing thru correctly). I also tried adding debug reporting and that didn't work either . byahhh
Comment #9
ressaThat sounds fantastic, I'll be ready to try it out, after you have had time to look at it, and if something is ready for testing at some point.
And once we get it working, and it handles dirty data more gracefully, and logs and reports missing images while continuing the import, it's probably worth considering adding such dirty data -- like a few variants of faulty image attachments like
<wp:meta_value><![CDATA[a:0:{}]]></wp:meta_value>-- to the test XML, to make sure they are handled well.About committing this #47 MR now or wait, I'll leave that up to you to decide. But if a halting migration due to dirty data is not acceptable, then it probably needs some more tweaking.
PS. About LLM's, have you seen #3574093: Ban slop issue summaries and comments? :)
Comment #10
hongpong commentedI think I should hold on the MR or perhaps subdivide it. I think it should not halt, but instead it should clearly notify the user what has happened.
Thank you for linking the issue, I had not seen it! Dries has a new one - https://dri.es/never-submit-code-you-do-not-understand - which I should try to stick by. Our situation with this module includes so many dirty inputs possible, more than a usual challenge. I agree adding the dirty ones is helpful for tests. I am very sympathetic to the argument that these vibe coding giant PRs are harmful to code reviewers. Ironically when I've been to the DrupalCamp NJ last couple years those were the most forward on AI usage especially on content review.
Zivtech warned that LLM are bad at using existing frameworks to solve problems, they tend to re implement existing systems (as noted in the thread as well). I have noticed that the LLM will tend to regenerate checker / helper functions duplicatively and in poorly organized ways. Zivtech created a system called drupal critic which is intended to put these things in check somewhat https://lobehub.com/skills/zivtech-drupal-critic-drupal-critic - https://www.zivtech.com/blog/ai-driven-code-reviews-transforming-efficiency - some of the niftiest stuff I have yet heard of. More here - https://www.zivtech.com/blog/claude-skills-and-mcp-drupal-development-dr... - slides https://www.drupalcampnj.org/sites/default/files/2026-03/Skills-MCPs-Dru...
I have also noticed that the logger I added (which I intended to apply in all places of the module) does not have the same signature as the logger in the MR. The already committed logger was mainly LLM generated and I think it is already improving our situation - I would like to just keep adapting it. However now I have not implemented it correctly using the LLM in this MR, and that should be a blocker on the MR I now think.
Comment #11
ressaYes, Dries's article was shared on Planet Drupal, so I already read it. Creating Drupal code with LLM is an extremely interesting subject, and divisive as can be seen in the issue with the hot debate taking place right now :)
I am impressed by the strong positions against LLM code for Drupal, from who I consider the elite of Drupal coders and core contributors. @phenaproxima mentioned banning users who don't disclose LLM-created MR's in #3568936: Embrace the chaos, add a couple of AGENTS.md file to core, and I floated the idea of banning LLM code contributions in Drupal, or perhaps allow it, but with VERY strong guard rails.
Several coders, like _nod almost got addicted to using LLM's, and this guy even feels he lost the ability to code: I was a 10x engineer. Now I'm useless.
Thanks for the Zivtech links! Interesting tools and articles by Zivtech, but like you noticed, LLM-generated code is not systematic at all, and how can it be? LLM's are not system architects, we are! An LLM is in essence just a glorified "guess the next word"-machine ... So it may be a useful tool for very small, confined tasks, but surely not for large chunks of code, where all kinds of hallucinations can sneak in. Interesting times!