Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues

Video meeting 2022-10-13 2100Z (tentative)

The hope is that most or all of the maintainers will attend. We will try to focus on longer-term goals than in the weekly meeting.

0️⃣ Who is here today?

quietone Hi
benjifisher Hello!
greg-boggs I am also here async (at work)
alison Helloooooo  :wave:

1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today’s meeting.

quietone What is the progress on the migration lookup changes?
benjifisher Earlier today, there was a discussion on an old Slack thread about migrating views from D7 to D9+. Can we continue that here? https://www.drupal.org/project/views_migration
benjifisher Was RTBC, re-rolled (by me), currently NR: #3266491: Add content in migrate UI functional tests only when needed
quietone Issues related to extensions removed from core
mikelutz (he/him) Are there issue numbers for those, @quietone?
mikelutz (he/him) Or you just want a generic discussion?
quietone A generic discussion

2️⃣ Action items. To be added later.

3️⃣ Statistics

mikelutz (he/him) NR: 13, 1 major
mikelutz (he/him) RTBC: 6, 1 major
mikelutz (he/him) Fixed since last week, only 1, but it was a biggie.

4️⃣ Comment in this thread if you are looking for ways to help. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

5️⃣ Previous minutes.

mikelutz (he/him) #3307393: [meeting] Migrate Meeting 2022-09-08 2100Z
benjifisher That one is fixed, but the meeting from 2022-09-01 still NR: #3307392: [meeting] Migrate Meeting 2022-09-01 2100Z

6️⃣ Announcements

7️⃣ Expose full set of debugging data in migrate_message table (filterable/searchable)

mikelutz (he/him) #3063856: Add ability to view migrate_message table data
benjifisher The latest patch adds 3 lines to SqlBase.php:
+    catch (\PDOException $e) {
+      throw new RequirementsException("No database connection available for source plugin " . $this->pluginId, [], 0, $e);
+    }

It is a little surprising that a change to a base class like that does not break any tests. Maybe that is a problem: should we add test coverage for this change?

quietone Yes, it should. I just didn't get to it.
quietone I do wonder why it was only catching that one exception. Was that by design?
benjifisher It was added in #2830036: MigrationPluginManager::getDefinitions() blows up in node derivers
benjifisher Looking at that file, we could do some cleanup
benjifisher
  • Simplify with the ?? operator.
  • Should we use a service instead of calling static methods (addConnectionInfo, getConnection) on the Database class?
benjifisher I think the snippet pulled in by Slack answers the question of why they checked only that exception.
quietone So, at the time, it was in response to a particular problem.
benjifisher a SQL source plugin (any source plugin that extends SqlBase) is pretty useless if the database is not configured OR if it cannot connect to the database.

Is there any reason not to throw a RequirementsException in both cases?

quietone As an aside, I wonder if the proposed change to SqlBase should be a separate issue?
benjifisher It is already kind of odd that we re-throw a ConnectionNotDefinedException unchanged, unless we are using the fallback $key = 'migrate', in which case we re-throw it as a RequirementsException.

I think that fallback is more trouble than it is worth.

It gets even more odd if we re-throw a PDOException as a RequiirementsException.

benjifisher I was also thinking that we should fix SqlBase and postpone this issue on it. Then we could have a separate issue for the current discussion.
benjifisher I meant to continue: a SQL source plugin is useless if it cannot connect, but a migration plugin is not useless without its source plugin. As this issue shows, we want to have access to the map and message tables even if the source is not working.
quietone Yes, separate issue is the way to go.
benjifisher Where else is RequirementsException caught? Here is a partial list:
  • MigrateExecutable::import()
  • tests
  • several places in the migrate_drupal module

Where is the RequirementsException caught? (edited)

8️⃣ Views Migrations

benjifisher @greg-boggs was saying earlier that what we really need is some testing of the current version of https://www.drupal.org/project/views_migration.
  • Start with a D7 site with lots of views
  • See what breaks
  • Create issues for the module
quietone Is there a database dump with the data?
benjifisher Not that I know of.
quietone I have said that I am willing to help any contrib if they provide a database dump.  #2906878: [Meta] Support for D7 -> D9 contrib migrate#comment-13995567
quietone But, I recall, only two have taken me up on that offer.
greg-boggs Do we have database dumps that were used for the rest of migrate? I don't really remember how that was done. Chx just helped me.
quietone In core, there are two fixtures used for testing, drupal6.php and drupal7.php. However, there is no reason to tie views to them. In fact, the drupal7 one has its own problems when used on a Drupal 7 site.
quietone Off the top of my head, if it were me I would probably start with 3 views, two very basic ones and a third that uses many features.
quietone The dump should be as minimal as possible but usable via the UI.
greg-boggs Yea, the current module was made by importing the core provided views. Going beyond that, I can test myself with my work sites, but obviously can't share dumps.
greg-boggs Doesn't need to be much in the test databse, just needs content types with fields and things  and blocks placed in a theme so you can load the views and see if they work
quietone And I don't care about the format of the dump. I just need functional data.
greg-boggs @hestenet (he/him) do you think the DA has a D7 website you can share a database from for views migration testing? Could truncate all the content from it. I ask because I know a lot of the DA websites are open source (edited)
greg-boggs Bonus points, if the site is used as the test site for views migration, the sites views will migrate to D9 :slightly_smiling_face: (edited)

9️⃣ Deprecate migrate_lookup and replace with cleaner process plugin

mikelutz (he/him) #3246666: Deprecate migrate_lookup and replace with cleaner process plugin
quietone Ah, still early days.

🔟  Add content in tests only when needed

mikelutz (he/him) #3266491: Add content in migrate UI functional tests only when needed
benjifisher Was RTBC, re-rolled (by me), currently NR
quietone There is a test failure. I've set it back to NW.
benjifisher Oh, you triggered a re-test?

It looks like more of the same: entity counts are changing because we are moving modules to contrib.

benjifisher I can tweak the tests again. Are we done moving modules to contrib? If not, then maybe we should just postpone this issue.
quietone No, I didn't retest. The test was run on 9 Sep and failed but the status was not changed.
benjifisher ... and I just missed it?

The test passes locally. :thinking_face:

quietone Lots of changes in HEAD recently?
benjifisher I have a few uncommitted files lying around. That explains it.
benjifisher I just uploaded a new patch.

Maybe the testbot left it at NR because the issue targets 10.0.x and the failing test was on 9.5.x.

benjifisher The new patch passes tests. :white_check_mark:

1️⃣ 1️⃣ Issues related to modules removed from core.

quietone Last night I found two migration issues, one about the RDF migration and one about Color.
quietone Shall we move them to the contrib project?
quietone And any others for removed extenstions?
quietone I guess the exception would be for some critical bug.
quietone The ones I found are #3187334: Drupal 7 color settings migration assumes that theme machine names do not contain underscores
benjifisher If those issues are going to be fixed, where will the fix be?If the answer is "in the code that has been moved to the contrib modules", then I think the issues should be moved.
quietone Considering that the issues for most removed extensions (some are still works in progress) have been moved to contrib, the answer is 'yes'.
quietone Right then. I'll move them.

1️⃣2️⃣ Wrap up

mikelutz (he/him) See you all next week!
quietone @mikelutz (he/him) Thank you!

Comments

benjifisher created an issue. See original summary.

benjifisher’s picture

benjifisher’s picture

Status: Active » Needs review

edit: when I first added this comment, it had the transcript. That belongs in the issue summary.

benjifisher’s picture

Issue summary: View changes
quietone’s picture

Issue summary: View changes

The minutes and credit are correct, I have only made formatting changes.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

benjifisher’s picture

Status: Needs review » Fixed

@quietone:

Thanks for cleaning up my copy/paste.

I confirmed that the only changes were formatting. Based on that and #8, I think we can close this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.