Import as media

Updated the patch according to the current version of the module

Rerolled patch from #10

Can anyone else check this one out and give it RTBC? Thank you!

None of the latest patches apply anymore. Trying to apply them by hand ends in deprecation warnings.

I added a few type safety updates flagged in the IDE. Otherwise should be a clean re-roll against today's 8.x-3.x. i saw the WordPressMigrationGenerator "path alias generator" part, I think the reroll patch here is correct but should be double checked.

(the MR !21 is the same as this patchfile.)

I applied it using patch -p1 < 3173916-11.patch from the base dir of the module as working directory. This helps deal with stuck hunk patch application. I did not test this but wanted to give people a clean spot to start again with this issue.

Oops - my local 8.x-3.x was missing the latest "path alias generator" commit. re rolled again.

some more notices. Per !MR21 reroll . https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/21

Major - Access to an undefined property Drupal\wordpress_migrate\WordPressMigrationGenerator::$mediaAudioID.
in src/WordPressMigrationGenerator.php:167

Major - Access to an undefined property Drupal\wordpress_migrate\WordPressMigrationGenerator::$mediaDocumentID.
in src/WordPressMigrationGenerator.php:167

Major - Access to an undefined property Drupal\wordpress_migrate\WordPressMigrationGenerator::$mediaImageID.
in src/WordPressMigrationGenerator.php:167

And additional phpstan and phpcs notices.

Some upstream changes in wordpress_migrate_ui/src/Form/ImageSelectForm.php - the merge conflict is only there, basically got similar modernization by other folks. I'm not actively going to tangle with this one more for now, but if someone else wants to hammer away on it, go ahead.

Created a new MR with branch 3173916-import-as-media rebased to 8.x-3.x.

Thanks so much abarrio for rebasing. Any ideas what we can do to fix the "Access to an undefined property" stuff flagged above? That seems like the biggest flagged snag currently.

Uploaded a fixing for the undefined property error and fixing some other coding standards on WordPressMigrationGenerator.

Uploaded one fix for an undefined index url error when executing drush ms and changed xpath expressions on media migrations with a simpler one.

excellent stuff abarrio. do you think it is RTBC yet? I don't have a test xml with audio attachment to test.

No, I think there is more work remaining to have it as RTBC.
I think we still need to work on:

• Test each media type to check that is completely working automatically without changing yml
• Fix coding standard of all new code
• Maybe be able to check if xml file is completely correct to avoid problems when importing (Maybe could be done on a follow up issue)
• Create some unit tests
• Fix all problems that appears on merges

I think with this points resolved we can proceed with this as RTBC.

regarding checking the xml file see #3248042: Add XML Lint to check the WordPress exports xml lint could be added to composer but i figured that would intensify requirements. as well as #3348775: WP import xml entities & security considerations for a note on other considerations.

regarding testing media types I suppose this would require a new test site altogether with the types of assets in question.

if you think we can subdivide out any portion of the patch as well (in other words if you think some standalone portion would actually be RTBC as it is), let me know. Thanks for all this!

Makes sense to me.

Maybe we can merge and complete this issue as a first approach and then continue working on some other issues to finish this big development part.

I think we need a meta issue with a complete list of tasks needed to finish it. If I have time this days I'll create it.

one small issue, unless i'm mistaken there are duplicated migration yml files in the root dir of the project on the Merge request?

that's completely true!!
The only yml files that needs to be there are the ones from migrations folder.

Removed unnecessary yml files from MR.
As I said on #26 I think we should work on those points but maybe we can proceed with this as a first approach and them complete the development in a follow up issue.

Regardless of the 'nice to have' like xml check, unit tests and certain code standard flags, I think we need to be assured that the migration is working correctly for most use cases.
Forgive my silly question but is "document" like PDF files and things of that nature? In other words if someone has added a plugin that allows the PDF uploads in the media library, this would cover importing those. I'm a little more familiar with the built in audio content type.
[In late Jan or early Feb i'm going to take at least a couple days to focus on this module again -- which would involve composing a new site in WP with these content types and trying to run the migrate in the patch. we could roll in before then with more clear test results but i am down to make that effort a little later.]

Actually on each media is getting the default type and checking the extensions allowed there to get default value when configuring migration:

it is not different between types of attachment files, it is just getting them to drupal files and then converting it into media.

Need a static patch of the current state of MR 35. Thank you for your work so far 🤗

Great! I did have to change the selector syntax from:

item_selector: '/rss/channel/item[wp:post_type="attachment" and wp:attachment_url[contains(text(), "gif") or wp:attachment_url[contains(text(), "jpg") or wp:attachment_url[contains(text(), "jpeg"))]'

To:

item_selector: '/rss/channel/item[wp:post_type="attachment" and (contains(wp:attachment_url, "png") or contains(wp:attachment_url, "jpg") or contains(wp:attachment_url, "jpeg"))]'

@wotts can you say where you changed this? was it in wordpress_attachments.yml at :

source:
  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: xml
  urls: []
  item_selector: '/rss/channel/item[wp:post_type="attachment"]'

Thank you.

I muddled things and needed to put in a rebase on latest 8.x-3.x HEAD after Drupal 11 support added and refactored WordPressMigrationGenerator. going to refactor a bit to make these functions smaller. Pulled the previous MR into MR39: https://git.drupalcode.org/project/wordpress_migrate/-/merge_requests/39

Re upping what abarrio suggested if anyone wants to get in:

• Test each media type to check that is completely working automatically without changing yml
• Fix coding standard of all new code
• Maybe be able to check if xml file is completely correct to avoid problems when importing in recent days ( I started a new module to cover this - https://gitlab.com/HongPong/xml_vuln_detector ) - not needed to do this now imo
• Create some unit tests
• Fix all problems that appears on merges

K I tried to extract the media field loop however I think I missed the mark. I also extracted the MediaTypes array to the properties of the class.

Warning: Undefined array key "plugin" in Drupal\migrate\Plugin\Migration->findMigrationDependencies() (line 685 of core/modules/migrate/src/Plugin/Migration.php). (and line 688). Appears after specifying all the fields at urls like - : /admin/structure/migrate/manage/my_wp_firstmedia1/migrations - the imports are "no data found".

Drupal\migrate\Plugin\Migration->findMigrationDependencies() (Line: 655)
Drupal\migrate\Plugin\Migration->getMigrationDependencies() (Line: 124)
Drupal\migrate\Plugin\MigrationPluginManager->createInstances() (Line: 93)
Drupal\migrate\Plugin\MigrationPluginManager->createInstance() (Line: 132)
Drupal\migrate_tools\Controller\MigrationListBuilder->buildRow() (Line: 272)
Drupal\Core\Entity\EntityListBuilder->render() (Line: 23)
Drupal\Core\Entity\Controller\EntityListController->listing()
call_user_func_array() (Line: 123)

On Drupal 11.1.5. Tried to extract as.

foreach ($media_types as $media_type_id => $media_type) {
        $config_key = $media_type_id . '_media_type';
        if ($this->configuration[$config_key]) {
          $this->CreateSingleMediaTypeMigration($media_type['migration_id'], $media_type_id, $process, $config_key);
        }
      }

and new function

  public function CreateSingleMediaTypeMigration($migration_id, string $media_type_id, $process, string $config_key): void {
    $this->migrateInfo[$migration_id] = $this->configuration['prefix'] . 'wordpress_media_' . $media_type_id;
    $migration = $this->createEntityFromPlugin('wordpress_media_' . $media_type_id, $this->migrateInfo[$migration_id]);
    $migration->set('migration_group', $this->configuration['group_id']);
   ...
    $migration->save();
    unset($process, $migration);

Added the unset, trying to make a more clean function.

@hongpong Which is the status of merge? It needs more work to do or just revision?

@abarrio I haven't had the time / ability to move this along further in recent months. Hoping people can test.