Drupal 11 compatibility fixes for wordpress_migrate

I've updated the module to D11 and kept D9 and D10 compatibility. I did one of the backwards compatibility issues differently than rector and found another similar issue. I lightly tested it on D9/D10/D11. The two methods I added at the end of SourceSelectForm can be changed once D9 is no longer supported.

I and someone in the Slack channel (https://drupal.slack.com/archives/C226VLXBP/p1726971471014629) had a problem where the protocol of the uploaded file was set to 'public://', but that wasn't resolved to 'http://'. I assume it was resolved in the past but behavior changed somewhere. I didn't look into that, I just changed the code to resolve the URL.

I also made a factory to create instances of WordPressMigrationGenerator and made that factory a service.

@tolstoydotcom I was reviewing this work since this module is failing automated D11 testing.

FYI there is a related issue on the public protocol problem.

Your patch seems to be missing the definition of WordPressMigrationGeneratorInterface as well as a wordpress_migrate.services.yml

That's odd, I'll update it tomorrow. I'll also test out #5 on the other thread.

An abridged def is:
namespace Drupal\wordpress_migrate;

interface WordPressMigrationGeneratorInterface {
public function createMigrations(array $configuration): void;
}

and the services file:
services:
wordpress_migration_generator_factory:
class: Drupal\wordpress_migrate\WordpressMigrationGeneratorFactory
arguments: ['@entity_field.manager', '@plugin.manager.migration', '@file_url_generator']

Thank you tolstoydotcom , thanks to msielski we should be clear of public protocol issue. #3484883: Protocol "public" not supported .

Open to advice about best move here w D11.

I can rework my patch to have the missing parts, it does work on D11.

However: the rector patch uses the backwards compatibility layer. My patch instead tests if things are available. That's not an issue if you don't want to support D9.

Should I post a new patch?

I am pretty unfamiliar with the backwards compatibility layer to be honest. I think we should probably hold onto Drupal 9 support if possible. Thank you.

Thanks everyone for working on this. "hold onto" means keep doing something, which slightly contradicts the first part of the sentence ... But I assume the decision is "No D9 support"?

I think that will be fine. After all, Drupal 9 reached end of life November 2023 https://www.drupal.org/about/announcements/blog/drupal-9-is-end-of-life.

That is fair enough ressa regarding D9 EOL. Maybe the best move is, tag another release as "the last D9 supported release", and after that, include this patch.

Here's an updated patch. It should work with D9/D10 but I only tested D11 this time.

@tolstoydotcom I tried applying the patch against wordpress_migrate 3.x-dev@dev but it failed to apply. Do you have other patches applied?

Composer gives me the error:

Gathering patches for dependencies. This might take a minute.
  - Installing drupal/wordpress_migrate (dev-3.x b0ebd6d): Cloning b0ebd6d586 from cache
  - Applying patches for drupal/wordpress_migrate
    patches/d11update-3435726-13.patch (D11)
   Could not apply patch! Skipping. The error was: Cannot apply patch patches/d11update-3435726-13.patch

A src/WordPressMigrationGenerator.php.rej file was generated with the conflicts:

--- src/WordPressMigrationGenerator.php
+++ src/WordPressMigrationGenerator.php
@@ -196,12 +203,12 @@ class WordPressMigrationGenerator {
   protected function createContentMigration($wordpress_type) {
     $dependencies = [];
     $content_id   = $this->configuration['prefix'] . 'wordpress_content_' . $wordpress_type;
-    $migration    = static::createEntityFromPlugin('wordpress_content', $content_id);
+    $migration    = $this->createEntityFromPlugin('wordpress_content', $content_id);
     $migration->set('migration_group', $this->configuration['group_id']);
-    $source                   = $migration->get('source');
+    $source = $migration->get('source');
     $source['item_selector'] .= '[wp:post_type="' . $wordpress_type . '"]';
     $migration->set('source', $source);
-    $process                = $migration->get('process');
+    $process = $migration->get('process');
     // Path alias generator.
     $process['path/alias'][] = [
       'plugin' => 'str_replace',
@@ -224,12 +231,12 @@ class WordPressMigrationGenerator {
       'replace' => '',
       'regex' => TRUE,
     ];
-    $process['uid']         = $this->uidMapping;
+    $process['uid'] = $this->uidMapping;
     $process['body/format'] = [
       'plugin' => 'default_value',
       'default_value' => $this->configuration[$wordpress_type]['text_format'],
     ];
-    $process['type']        = [
+    $process['type'] = [
       'plugin' => 'default_value',
       'default_value' => $this->configuration[$wordpress_type]['type'],
     ];

If you clone https://git.drupalcode.org/project/wordpress_migrate the patch should apply cleanly.

Maybe use a Gitlab fork, to streamline the process?

AFAIK, the link in my last comment is where the latest code is. If anyone wants to test it, just:

git clone git@git.drupal.org:project/wordpress_migrate.git
cd wordpress_migrate
patch -p1 < d11update-3435726-13.patch

@tolstoydotcom, thank you, I was testing against the dev branch. Patch applied fine for me against 8.x-3.x and everything is working well on Drupal 10.3.6.

Okay I fumbled around a bit (issue forks are a little tricky) on MR 19 but I think this encompasses your changes tolstoydotcom and a few small small type-safety changes. I think we can roll this in unless someone catches an issue that needs attention.

One flag still in there : (link)

\Drupal calls should be avoided in classes, use dependency injection instead
File wordpress_migrate_ui/src/Wizard/ImportWizard.php:83

It will need a re-roll against 8.x-3.x head and ideally cleaning up a few of those flags, which may need some more dependency injection added in constructors to get rid of \Drupal calls. There is currently a merge conflict popping up.

I count five instances of \Drupal:
- in README.md. I don't think that should be replaced
- in a drush function. Unless that's changed to use classes, that can't be injected.
- in SourceSelectForm. I didn't inject that because getFileValidators will be reworked so the Drupal call is no longer needed once the module doesn't support D9
- in ImportWizard. I don't know if there will be a conflict with how ctools does things. If there is, this might have to remain.
- in ImageSelectForm. I can replace that.

I'll post another patch over the weekend that deals with the last two items.

Increasing Priority, since it is placed under "Major issues" header in #2904990: Plan for wordpress_migrate 8.x-3.x beta release, so I am also adding that as parent issue.

@tolstoydotcom and @ressa thank you for digging around this weekend. It was suggested to me by @dinarcon that we could split up the patch (MR19) to simplify matters somewhat by moving the new interfaces / service / factory / validator over to that issue (unless they are absolutely necessary to get Drupal 11 working).

I have filed a new issue #3489412: Refactor / split up Wordpress generator (service, factory and validators) to cover these things. If that is objectionable or duplicative I would say, alright let's proceed as we have been - and I would change the new issue to cover a smaller refactoring. However I think this will be helpful to consider.

Regarding dropping Drupal 9 and the file API issues you mention maybe this is an indicator we should actually tag a final D9 compatible release before adding this D11 patch. I am not against that.

Also dinarcon pointed out that we have broken Drush 8 code inside 8.x-3.x currently and I think at a minimum we should comment out the code, probably on this issue / patch, because it is inoperative and throwing CI notices for being so wildly out of date. Then drush 12+ coverage could be re added, dinarcon suggested in a new issue which makes sense to me since that is years different than the existing drush thread.

I based this on the latest dev version. I injected services in the last two places I listed in #23. I didn't deal with the drush code. It'd be great if we could freeze changes to dev for a while, test the patch on D9/D10/D11 (I only tested in on D11), and then deal with drush etc.

To use:

git clone git@git.drupal.org:project/wordpress_migrate.git
cd wordpress_migrate
patch -p1 < ../d11update-3435726-26.patch

Oh this is really looking look great. Thank you for this elegant workaround for D9. I think we can just tag the final D9 release before applying your patch. I can test it on D10 and D9 tomorrow as well. (not tested yet)

Hoping we can get a fix on duplicate users (#3123393: Users are duplicated (Integrity constraint violation) Duplicate entry) before tagging the next release.

on SourceSelectForm.php . the IDE doesn't like !empty but I think 'isset' is a little better.
if (isset($constraint_definitions['FileExtension'])) {
Also I was able to get a TypedDataManager in the __construct() here to replace the \Drupal notice in getFileValidators.

on ImportWizard.php $tempstore_id needs a parameter type.

Thank you for working through this as I reshuffle a bunch of stuff and try to get nearby code fixes in there. Also I removed the broken drush code tonight earlier so it's out of the way. I think we can freeze for a day or 2 unless someone has a quick fix for duplicate users. Really appreciate all this work.

here is the patch with a few adjustments noted above. MR is now updated. Awesome stuff thank you!

I pushed a commit to fix coding standard issue, please review.

Thank you baltowen I was also wondering about that one and not sure if it is a string but, if it works it is probably fine.

I am going to cherry pick some of the smaller changes out of this patch to apply before the larger patch on D11 so we can get some of it into the next tagged release this week alpha7 - things which ware not part of the 'big reworking' but just involve smaller changes, and dependency injection, and are still Drupal 9 compatible. New issue for these: #3489725: Dependency injection and access check

Also I have updated the "plan" again to clarify we will drop D9 + D10.1 support going into alpha8, which will make this patch a lot easier to deal with juggling the APIs and so on.

I will also try and get some more testing. We are nearly there folks!

Yes, $tempstore_id is a string See https://git.drupalcode.org/project/ctools/-/blob/8.x-3.x/src/Wizard/Form...

Okay I have been able to finish the cherrypicking of several items from this issue into 8.x-3.x, and then rebased the MR to match the new 8.x-3.x, which has scaled down the size of the rest of the patch at least a good ways. Thanks for bearing with me, I won't do any more cherry picking from here. Excellent work everyone!!

re SourceSelectForm typed data manager dependency injection. the IDE does not like something about it, I didn't really understand so I didn't cherrypick that item.

The last pipeline was failing. It looks like during a merge @baltowen's phpcs fix was lost. I cherry-picked c636be41 and pushed. This should make phpcs pass.

Tested the latest code and was getting a fatal error when starting the wizard at /admin/structure/migrate/wordpress_migrate

The website encountered an unexpected error. Try again later.

ArgumentCountError: Too few arguments to function Drupal\wordpress_migrate_ui\Form\SourceSelectForm::__construct(), 0 passed in /var/www/html/web/core/lib/Drupal/Core/Form/FormBase.php on line 82 and exactly 1 expected in Drupal\wordpress_migrate_ui\Form\SourceSelectForm->__construct() (line 25 of modules/contrib/wordpress_migrate/wordpress_migrate_ui/src/Form/SourceSelectForm.php). 

I pushed a new commit passing the missing parameter.

Tested this with Drupal CMS 1.0.2 and the posts titles are imported but the content is ignored. Tried this branch on Drupal 10.4 and had the same issue.

I think we can bring the patch in. I am testing in Drupal 10.4 11.0 11.1 to put it to bed to this point.

However re xpath note I believe we need a better body field picker in Drupal 11.1 (similar to the other field pickers in the wizard) . New issue added #3516531: Add body field picker w default to wizard & processing. It works in Drupal 10.4 and 11.1 with the default 'body' field but it doesn't warn the user if this situation isn't connected.

Also I've seen the migration paginator skip back to the migrate machine names setter page. It still seems to be saving fine but there is a kink there imo (in 11.0 and 11.1 i think). [could be a form cache issue also] I think that is probably #3489999: Subtle wizard pagination bug posts/pages, remove "empty".

Everything else appears to be solid so I am pulling this in, then we can move to other refactoring and the plugin architecture, & so on.

I think we could still put changing logger statements into injected service, into this issue. (and then change the required drupal core version to >=10.3

I tested the 8.x-3.x branch again with version Drupal 10.4.6 and Drupal CMS 1.1.0. It's fixed on version 10.4.6 but with Drupal CMS the Blog and Page fields are not visible on the body field picker so the body still can't be imported.

Thank you for testing xpete! can you confirm if it is a "text_with_summary" type of body field? (screenshot if needed).

Also i created a Drupal CMS "recipe" module here last week but not tested it yet - i don't have a good drupal CMS test process https://www.drupal.org/project/wordpress_migrate_recipe

Drupal CMS don't use text_with_summary. It uses regular text fields. You can read a discussion about it here https://www.drupal.org/project/drupal/issues/3477043
You should follow the instructions to setup a local Drupal CMS installation.

OK xpete I will re open the other issue #3516531: Add body field picker w default to wizard & processing to address the body field issue. I would still like to add some exception handling here on this issue before closing it

re what else goes in next alph tagging the release: assuming nobody is going to nudge ahead on #3173973: Pluggable extension, support contrib + custom WordPress, Yoast SEO which is close but I am trying to extend its functionality to preprocess -- and that is not a 11.x blocker at all so it should not hold up tagging for 11.x. i think this body field thing ought to be done before tagging the release tho.

Screenshot

Hi, I'm here just for this issues.

Tomorrow, if can be useful, I can provide more info. Ty.

Ok, thanks to Claude Ai i maked a custom module to fix this issue and now I can import as expected both body and featured image, but featured image are present in filesystems, but not present in Drupal media gallery ATM.

Is there any guide to import as best as possible.
I need to import in some specific order?

Eg: post, tag, images, comments can be ok?

Thanks.

Great that you got it working!

Did you already check "Documentation" and "Readme" under Resources on https://www.drupal.org/project/wordpress_migrate?

@ressa, yep,.I searched in faq page. But cannot found anything about import order. Ty.

Thanks for a fast response. I skimmed the README, and the only mention of order seems to be images before Posts, under https://git.drupalcode.org/project/wordpress_migrate/-/blob/8.x-3.x/READ....

There will probably be more who wonder about this, so perhaps you can create a new issue to clarify this, and then add the answer in the FAQ page, or README, possibly under a new "Recommended order of import" section?

Ty too ressa. Today I try again. ✌️

I tried, but when I import images, I can see in FTP (not in Drupal media center). Then, when I import post all are without images.

Launched a fix to update media from folder, now I can see (but I just need to set again in post).

Also, comments seems to be not imported at all.

So, isn't a great migration 😥

Thanks for testing this patch, too bad it seems to not work ... in that case, it seems like the status should be changed to "Needs work".

It would be valuable if you shared your steps, to make it clearer where and how it fails. Like:

• I downloaded the latest version of 3.x-dev
• I applied the patch from comment #49
• I updated the file a_file_in_the module.php with ...
• etc.

But the best would be some feedback from the other developers here who supplied patches (@dinarcon, @xpete, @hongpong, etc.) since they know best what to look out for.

To be honest, my WP contain just about 50 articles, only a member (me) as admin, and some post with a lot of comments. I hope to can make a clean migration asap. Write all by hands require a lot of work and I don't have time for this 😅

ATM I just imported articles and comments, but all comments are with my name (so, I still need to edit all).

Anyway, this module ATM isn't ready for Drupal CMS 11.

This is a small extra modules fix.

Anyway, migration still need a lot of work.
ATM I've a custom php script and can import also description, fix image URL, clean html.

I think this is resolved now? I applied the formatByteSize auto patch it wanted. Let's open more narrowly tailored new issues if things continue to be a problem. Thank you salvatoren for throwing in an alternative system.

As we discussed earlier support for Drupal 9 and 10.1.x is now removed in both the info.yml files which will let us tailor efforts better. (meant to do this on the release yesterday but forgot!)