Hello all, it’s time for the biweekly migration subsystem meeting. The meeting will take place in slack in various threads. This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every second Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. #3456078: [meeting] Migrate Meeting 2024-07-18 2100Z)See the parent issue for an idea of the typical agenda.
➤Transcript will be exported and posted to the agenda issue. For anonymous comments, start with a 👤 emoji. To take a comment or thread off the record, start with a 🚫 emoji.
| Migrate Initiative Meeting |
https://www.drupal.org/project/drupal/issues/3421014 (edited) |
| Migrate Initiative Meeting |
(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1739455578919459?thread_ts=... (edited) |
| benjifisher |
I reviewed and tested over the weekend. I think it is close, but I asked for some minor changes. |
| benjifisher |
@quietone made the most important change, but there are a few still pending. @godotislate, can you take a look? |
| benjifisher |
Feel free to discuss "perhaps" and "probably" in my comment. |
| benjifisher |
This issue was unblocked when[#3258581] was fixed. |
| godotislate |
@godotislate, can you take a look?maybe on the holiday Monday (edited) |
| Migrate Initiative Meeting |
https://www.drupal.org/project/drupal/issues/3069776 (edited) |
| Migrate Initiative Meeting |
(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1739456239488759?thread_ts=... (edited) |
| benjifisher |
This issue is unblocked because[#2833060] was recently Fixed. |
| benjifisher |
I have not looked at it recently. How close is it to being ready? |
| benjifisher |
We agreed to work on[#3500698] as a follow-up issue. That decision made it possible to finish #2833060. |
| benjifisher |
@heddn, do you know how close this issue is to being ready? |
| heddn |
@benjifisher which issue? 3069776 is pretty close |
| benjifisher |
Yes, that is the one I meant. |
| Migrate Initiative Meeting |
Do we have any examples of this in our documentation? |
| Migrate Initiative Meeting |
(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1739456449824939?thread_ts=... (edited) |
| benjifisher |
I asked about this in the previous meeting, but I did not get any answers. @mikelutz (he/him), @dinarcon? |
| dinarcon |
Link to thread in previous meeting for reference https://drupal.slack.com/archives/C226VLXBP/p1738273717740989 |
| dinarcon |
I use this heavily when debugging migrations... It lets me see the source query.
use Drupal\Core\Database\Query\AlterableInterface;
/**
* Implements hook_query_TAG_alter().
*/
function udrupalcom_query_migrate_alter(AlterableInterface $query) {
/** @var \Drupal\migrate\Plugin\MigrationInterface $migration */
$migration = $query->getMetaData('migration');
if (in_array('Drupal 7', $migration->getMigrationTags())) {
var_dump((string) $query);
}
}
(edited) |
| dinarcon |
Reading the original thread, the request is for an example of altering the query... |
| dinarcon |
I sometimes do the following to only import a subset of records... That way I have some sample data for my migrations without having to wait for all records to be imported.
use Drupal\Core\Database\Query\AlterableInterface;
/**
* Implements hook_query_TAG_alter().
*/
function udrupalcom_query_migrate_alter(AlterableInterface $query) {
/** @var \Drupal\migrate\Plugin\MigrationInterface $migration */
$migration = $query->getMetaData('migration');
if (array_key_exists('high_water_property', $migration->getSourceConfiguration())) {
$query->range(0, 100);
}
}
(edited) |
| benjifisher |
This is related to the issues in 8️⃣. The SqlBase base class adds tags to the query, and we want to clean up that class. But in the mean time, we should make sure that we have examples of what those tags are good for. (edited) |
| benjifisher |
@dinarcon, do you mention that debugging code in any of your articles? |
| dinarcon |
I updated my code snippets to note that, in fact, I am using the migrate tag added in \Drupal\migrate\Plugin\migrate\source\SqlBase::prepareQuery My examples are mostly for debugging purposes. I also use the migration object added as metadata. (edited) |
| benjifisher |
I was beginning to question my short-term memory ... |
| mikelutz (he/him) |
I mean, the documentation for using the hook is here: https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Database%... adds the tag “migrate” and “migrate_” by default to queries that extend it and don’t override it in prepareQuery(). I don’t know that we document that we add that query tag. It’s not in the SqlBase file documentation. |
| mikelutz (he/him) |
I don’t think it’s necessary to document how to use the hook, that is documented elsewhere and not really a migrate thing. What should be documented is simply the fact that SqlBase adds those tags by default to migration queries. |
| benjifisher |
I think it would be helpful to add under https://www.drupal.org/docs/8/api/migrate-api/migrate-source-plugins, not as API documentation. |
| dinarcon |
I have not included those snippets in articles written so far, but I plan to include debugging tips in the last article of the current D7 to D10 migration series. |
| benjifisher |
Maybe add an @see comment to the doc block for prepareQuery(). |
| dinarcon |
The one to see the query as a string might be useful in general, but the one to limit the records to process in the presence of high_water_property might not be good advise. I mostly use it to get sample content, but it does not take into account relationships among entities so things could break or not function as expected. (edited) |
| benjifisher |
New documentation page: https://www.drupal.org/docs/8/api/migrate-api/migrate-source-plugins/mod..., corrections, and other comments are welcome! |
| benjifisher |
I tested using an OOP hook. As I hoped, it lets you target a tag like migrate_d7_node:page directly. You cannot do that with hook_query_TAG_alter() since : is not allowed as part of a function name. |
| benjifisher |
I also took the opportunity to mention the batch_size option. I have long thought that this option needs to be more widely known. |
| benjifisher |
@nicxvan: ^^ |
| benjifisher |
@mikelutz (he/him): What is the best practice for the parameter type declaration? If you want to use hasTag(), then you should specify AlterableInterface (as in database.api.php). But if you want to use condition(), then it should be SelectInterface. |
| benjifisher |
I guess AlterableInterface&SelectInterface. I have never used an intersection type before (and I had to look up the syntax). I guess there is a first time for everything. |
| Migrate Initiative Meeting |
We already moved the content_entity source plugin. |
| Migrate Initiative Meeting |
(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1739458332693849?thread_ts=... (edited) |
| benjifisher |
I agree. Let's create an issue for it. |
| dinarcon |
FWIW, it was mentioned at[#3315257]#comment-15126074 |
| benjifisher |
I would like to finish converting source plugins to attributes before working on this. We almost ended up reverting content_entity to source_module:migrate_drupal in that issue. |
| benjifisher |
Other than that conflict, moving content_entity was pretty easy. One thing we decided was to keep the version in migrate_drupal intact (but deprecate it) instead of making it a wrapper for the new version in the migrate module. That way, we could add parameter and return type declarations to the new version without breaking BC. |
| benjifisher |
I added[#3506605] and already added a couple of comments. We can continue the discussion there. |
Comments
Comment #7
benjifisherComment #8
smustgrave commentedSeems all threads were captured