Problem/Motivation

The migrations in l10n_migrate and the site repository copy the Drupal 7 tables column by column. Comparing them with the 3.0.x data model and with what the site does after the migration turned up transforms that are missing, data with no migration, and steps the cutover needs. This issue lists them so each can become a child issue with a fixture based test, as #3397394 asks for.

Transforms the migrations lack

  • Connector module ids. The project migration copies connector_module verbatim, so migrated projects reference l10n_drupal_rest_restapi while 3.0.x expects the plugin id drupal_rest:restapi. Parsing from the queue, drush and the connector pages fail for every migrated project. Needs a static map, after checking which values production holds.
  • Release titles. Drupal 7 titles are the version only, the 3.0.x scan writes "Project version". Migrated and new releases read differently in every list unless the migration builds the title.
  • String context. 3.0.x stores no context as NULL, Drupal 7 as an empty string. The filters accept both since #3621196, but the migration should turn the empty string into NULL so there is one spelling.
  • Strings with a falsy value ('' or "0") are skipped by the source plugin. If production has any, their lines and translations lose their string. "0" is a legitimate source string.
  • Plural formulas. Languages outside l10n_pconfig's list get the raw Drupal 7 formula, without nplurals and with $n, in the l10n_pconfig setting. The exporter writes that setting into Plural-Forms as is, so those languages export invalid headers. Build the full formula from the Drupal 7 plurals and formula columns for every language.
  • Disabled languages migrate as active translation targets.
  • Group memberships. The membership state (pending, blocked) is not mapped, so everyone becomes active, and the role map uses OG role ids 1, 2, 10, 12, 14 without bypass, so a membership with any other role id is dropped. Same for the user migration with role ids 6, 23, 38, 33, 28. Both need the production ids verified first.
  • Packaged files. The file migration deliberately skips .po files (they are synced, not copied), but the packager file rows keep their fid. Every packaged file row then points at a missing file entity, and the downloads page and blocks, which join the file table, show nothing until each release is repackaged. The .po rows need file entities without a copy, or the packager file migration creates them.

Data with no migration

  • Config schema store (#3621250 added the tables): the Drupal 7 rows are serialized PHP, the 3.0.x columns JSON. Without them, config strings that depend on another module's schema stay missing for releases that never get parsed again.
  • Settings. No variable migration exists and the site config carries development values: packager update URL under the site's files, the fixture release list as refresh URL, cron off. Production values belong in the site configuration for the live environment, starting from the live variable table.
  • Statistics pages read l10n_server_translation_history from the migrate database connection, the Drupal 7 database. After the cutover that database is gone, while the data is in the local table. Small separate fix.
  • Scan state: covered by the runbook step from the #3563318 follow-up.
  • OG role permissions: the site config roles match the live ones except that the community manager lacks start over packages and member management.

Cutover steps

  1. Verification queries on the production snapshot (connector values, role ids, membership states, falsy strings, languages, variables, packaged file counts), then fix the static maps.
  2. Production settings and group roles in the site config.
  3. The migrations, then the scan state seed, one scan, and a check that only newer releases were created.
  4. Parse what Drupal 7 left unparsed, never repackage everything: the files stay on ftp.drupal.org.
  5. Count comparison per table and spot checks.

Proposed resolution

One child issue per item above, each with a fixture based kernel test in l10n_migrate following #3397394. The full list with priorities, evidence and the verification queries is kept in the working notes and will be attached.

LLM disclosure

LLM was used to find, diagnose explain and fix this issue. With human review.

Comments

gábor hojtsy created an issue.

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

One item from the summary can be dropped: the string context.

Drupal 7 stores '' for strings without a context. On 3.0.x the parser writes '' as well (see the "Context cannot be null" line in L10nHelper), but a string base field holding '' counts as empty for the entity API and gets stored as NULL. Verified on a 3.0.x site: a string entity created with context '' comes back from the database with a NULL context. The migration destination is the same entity API, so migrated rows end up as NULL without any transform in the migration.

There is no index effect either way: strings are looked up by the hashkey, an md5 of value and context concatenated, which is the same for '' and NULL, and nothing indexes the context column itself.

What has to hold is that every comparison against an empty context on 3.0.x also accepts NULL. The translate page filter does that since #3621196: Declining suggestions and several translate page filters are broken by leftover Drupal 7 column names and the exporter checks for emptiness, so no migration work is needed here.

LLM was used to find, diagnose explain and fix this issue. With human review.