[Meta] Improve the general approach of handling module reinstallation

Problem/Motivation

First and foremost, the challenge revolves around the fact that “we” (aka. Acquia DAM) do not own the media entities (in terms of their entity type is not defined by us) but we still have some responsibility over their future existence. We only practice ownership over the source plugins the media items rely on.

When the module gets uninstalled, thus our source plugins are gone, then all the media items that once relied on them, still remain in the system.

In such a divided situation, when the module gets re-installed for any reason (debatable how realistic case it is), then we face a somewhat different landscape compared to a vanilla-fresh installation. For example, various problems stem from getting the media__acquia_dam_asset_id DB table dropped during uninstallation of the module.

Problem I. – Inconsistent listing results after module reinstallation

After reinstalling the module existing media items once created with DAM-related media types still appear on Drupal core's media listing page but they all disappear from the module's own “DAM assets” listing page. Administrators may find this discrepancy disturbing as it confuses them about whether these media items still exist or have been gone.

Steps to reproduce

1. Install the module, authenticate the site-wide client and your user account too with Widen.
2. Create any media items through DAM. Ensure that in the media__acquia_dam_asset_id table, a record representing it exists.
3. Visit the /admin/content/media page. Ensure that the media item is listed.
4. Visit the /admin/content/dam-media page. Ensure that the media item is listed here too.
5. Uninstall the module either via the GUI or by the Drush command. Ensure that the media__acquia_dam_asset_id table gets dropped in the DB.
6. Install the module again by any means mentioned above. Ensure that the media__acquia_dam_asset_id table got created fully empty.
7. Authenticate the site-wide client and your user account too with Widen.
8. Visit the /admin/content/media page. Ensure that the media item is still listed.
9. Visit the /admin/content/dam-media page.
   • Expected: the media item should be listed because Core's media table still holds a record of it.
   • Actual: the list is empty.

This is because the current filtering logic implemented as a Views filter (source code) strictly checks the existence of records within the media__acquia_dam_asset_id table. However, when the module uninstalls, this table – among others – gets dropped, but the media items created remain. Next time, when the module is installed again the still existing media items have no mention in the examined DB table, therefore the Views listing returns empty results.

Proposed resolutions

1. Add exclusion of the media__acquia_dam_asset_id table on module uninstallation. Arguments:
   • Pro: probably the easiest solution
   • Contra: residue remains in the DB which is against the general best-practice of the Drupal ecosystem.

   
   OR
   

2. Alter the module process to let the administrator decide when uninstalling the module whether they want to preserve this table or not. Explain them the reasoning why is it asked. If opted to preserve, then see option #1. Suggest a default option for this decision (probably to not preserve).
   • Pro: best of both worlds, responsibility given to human instead of false but hardcoded assumptions.
   • Contra: research needed on how easy to alter the module uninstallation process.

   
   OR
   

3. Change the filtering logic to check different values to tell whether a DAM-related media item still exists or not.
   • Pro: no user interface change, only the information content gets improved.
   • Contra: needs to figure out what other values we can check (for example the existence of acquia_dam_managed_file/acquia_dam_managed_image fields? Obviously thius can happen )

Problem II. – Misleading the asset update checking logic

The asset update checking logic (added in #3443055: Handling media items when their asset gets unavailable in Widen) believes that a media item existing from earlier installation is still DAM-related, but actually it is not anymore after it looses all its connection to their asset ID.

Problem III. – Previously saved local files blocks the URI from fresh data being stored

In #3490413: Prepare our file handling logic to residue left behind from previous installations, some further logic has been outlined regarding how we could free up the file system before saving the freshly fetched data: thumbnail images + asset files (if on-site storing is enabled).

Reminder: we cannot simply erase the two DAM-related directories in {webroot}/sites/default/files, because media entities might still exist and reference their associated managed file entities, which those files on disk belong to.

Remaining tasks

User interface changes

API changes

Data model changes