Problem/Motivation

When d7_file migration is reused, in the source section it is not possible to filter the type of file to migrate.

Steps to reproduce

Reuse d7_file migration, it is not possible to filter by type.

Proposed resolution

Add code to Drupal\file\Plugin\migrate\source\d7 File.php

In the first comment I post a patch that I have been using for some time.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#2 migration-file-filters-3263393-1.patch1.5 KBarturopanetta

Issue fork drupal-3263393

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

arturopanetta created an issue. See original summary.

arturopanetta’s picture

arturopanetta
Primary language Italian
Location Grotteria (RC)
commented
StatusFileSize
new migration-file-filters-3263393-1.patch1.5 KB

I submit the patch that I use to filter the files to migrate.

arturopanetta opened merge request !1798

quietone’s picture

quietone commented
Title: [Migration d7_file] Add a field that filters the files to migrate » Allow file source plugint to filter by mime type
Component: file.module » migration system
Issue tags: +migrate-d6-d8, +migrate-d7-d8

@arturopanetta, thanks for the patch.

Only have time to move this to the migration system component where the migrate maintainers work and to add tags.

jonathan_hunt’s picture

jonathan_hunt commented

I like this enhancement but it's probably achievable via #3069776 SQL source plugins: allow defining conditions and join in migration yml, e.g.

source:
  plugin: d7_file
  scheme: public
  conditions:
    -
      field: filename
      value: ['thumbnail', 'preview', 'mini']
      operator: "NOT IN"
    -
      field: f.filemime
      value: 'image/%'
      operator: LIKE
arturopanetta’s picture

arturopanetta
Primary language Italian
Location Grotteria (RC)
commented

Thanks jonathan_hunt, I didn't know sql source existed.
Since sql source does the same thing, I think this PR is no longer needed, it would be duplicate code to keep.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mikelutz’s picture

mikelutz
He/Him
Primary language English
Location Michigan, USA
commented
Status: Active » Closed (works as designed)

The core migration sources are designed to support the core migration UI, which itself is designed to migrate all content/users from a d7 site. There is no need to limit files by mimetype, as they are all migrated at once in the core file migrations. You will want to have a custom source plugin to support your custom migration here, but it is not appropriate to include custom migration sources in Drupal core. The migration system is designed to be extensible to allow you to create targeted source plugins like this as needed.

mikelutz’s picture

mikelutz
He/Him
Primary language English
Location Michigan, USA
commented
Status: Closed (works as designed) » Closed (duplicate)
Related issues: +#3069776: SQL source plugins: allow defining conditions and join in migration yml

Note: we do have an issue #3069776: SQL source plugins: allow defining conditions and join in migration yml to allow arbitrary sql filtering on ANY plugin extending SqlBase. This is something that we would make use of in core to reduce the number of source plugins needed. It would also be a path to solving your issue. In retrospect, I'm going to close this as a duplicate of that one.