Problem/Motivation

After updating Lightning to 4.1.9 and updating BLT to 11.6.0, database updates for Lightning Media fail.

Steps to reproduce

Update Lightning and BLT via Composer to above-referenced versions, and run drush updb. I don't think the BLT update is related, but including it here for context. Database updates then fail as follows:

vagrant@local:/var/www/mysite$ drush updb
 [warning] Invalid json in modules/contrib/migrate_devel/composer.json
 ------------- -------------------- --------------- ---------------------------
  Module        Update ID            Type            Description
 ------------- -------------------- --------------- ---------------------------
  lightning_m   8002                 hook_update_n   Creates the media_library
  edia_docume                                        form display for
  nt                                                 documents.
  lightning_m   8007                 hook_update_n   Creates the media_library
  edia_image                                         form display for images.
  lightning_m   8003                 hook_update_n   Creates the media_library
  edia_twitte                                        form display for tweets.
  r
  lightning_m   8004                 hook_update_n   Creates the media_library
  edia_video                                         form display for videos.
  media_entit   change_source_name   post-update     Rename source from
  y_instagram                                        "instagram" to
                                                     "oembed:instagram".
 ------------- -------------------- --------------- ---------------------------


 Do you wish to run the specified pending updates? (yes/no) [yes]:
 > y

>  [warning] Invalid json in modules/contrib/migrate_devel/composer.json
>  [notice] Update started: lightning_media_document_update_8002
>  [error]  The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, audio_file, video_file, oembed:video, image, oembed:instagram, twitter, video_embed_field
>  [error]  Update failed: lightning_media_document_update_8002
>  [notice] Update started: lightning_media_image_update_8007
>  [error]  The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, audio_file, video_file, oembed:video, image, oembed:instagram, twitter, video_embed_field
>  [error]  Update failed: lightning_media_image_update_8007
>  [notice] Update started: lightning_media_twitter_update_8003
>  [error]  The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, audio_file, video_file, oembed:video, image, oembed:instagram, twitter, video_embed_field
>  [error]  Update failed: lightning_media_twitter_update_8003
>  [notice] Update started: lightning_media_video_update_8004
>  [error]  The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, audio_file, video_file, oembed:video, image, oembed:instagram, twitter, video_embed_field
>  [error]  Update failed: lightning_media_video_update_8004
>  [warning] Invalid json in modules/contrib/migrate_devel/composer.json
>  [warning] Invalid json in modules/contrib/migrate_devel/composer.json
>  [warning] Invalid json in modules/contrib/migrate_devel/composer.json
 [error]  Update aborted by: lightning_media_document_update_8002, lightning_media_image_update_8007, lightning_media_twitter_update_8003, lightning_media_video_update_8004
 [error]  Finished performing updates.

The result of these updates failing, specifically 8004, is a fatal error that leaves the site in an un-usable state:

Drupal\Component\Plugin\Exception\PluginNotFoundException: The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, audio_file, video_file, oembed:video, image, oembed:instagram, twitter, video_embed_field in Drupal\Core\Plugin\DefaultPluginManager->doGetDefinition() (line 53 of core/lib/Drupal/Component/Plugin/Discovery/DiscoveryTrait.php).

I'm entering this with a priority of Major as it leaves the site in an un-usable state following the update failure.

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Comments

dbrownmt created an issue. See original summary.

bemarlan’s picture

No patch, but I was at least able to work around this issue by:

1. Reverting the database (to before drush updb was ran.
2. Manually modify media.type.instagram.yml to change source: instagram to source: oembed:instagram
3. drush updb
4. drush cr (this was required)
5. drush cim

dbrownmt’s picture

It makes sense that #2 would work, but drush updb continues to fail after rolling back the database.

dbrownmt’s picture

Interestingly, if I pin Lightning to 4.1.5, the same database update takes (the ID of the update is change_source_name). This happens regardless of the version of BLT.

alegacy99’s picture

This solution does not work consistently.

phenaproxima’s picture

Issue tags: -Needs work
farnoosh’s picture

#2 didn't work for me.

phenaproxima’s picture

What a tricky pain in the ass. We ran into this when we were forced to update Media Entity Instagram to 3.x, due to upstream changes with Instagram's API. But I did anticipate this, and I wrote code to account for it: https://git.drupalcode.org/project/lightning_media/-/blob/8.x-4.x/module...

So why, I wonder, are you still hitting it? Maybe we have a bad assumption somewhere. What we're relying on, here, is that your update process will run this function (from Media Entity Instagram): https://git.drupalcode.org/project/media_entity_instagram/-/blob/3.x/med...

After that function is run, things should be hunky-dory. But during the update process, when things are still in an inconsistent state, we need to monkey-patch the instagram plugin ID into Drupal's memory space. Once the update concludes, we don't need to do that anymore, which is why the first code I linked specifically monkey-patches it if the Drupal kernel is an instance of UpdateKernel (which is used during both the Drush update process, and update.php's process, as far as I know).

Sooo...I don't know exactly why you're running into this. Maybe Lightning Media Instagram is not installed?

robpowell’s picture

Thanks for the update @phenaproxima. I can confirm that I have lightning_media enabled

 ----------- ------------------------------------------------- --------- --------- 
  Package     Name                                                                 Status    Version  
 ----------- ------------------------------------------------- --------- --------- 
  Lightning   Media Instagram (lightning_media_instagram)      Enabled   8.x-4.4  
  Media       Media Entity Instagram (media_entity_instagram)    Enabled   3.0.4    
 ----------- ------------------------------------------------- --------- --------- 

For my site the post_update_hook completes but the site is down with the same error message. When I try to go to the site or do a config import I get the

The "instagram" plugin does not exist.

The offending config files seem to be:

  • media.type.instagram
  • core.entity_view_display.media.instagram.embedded
  • core.entity_view_display.media.instagram.default

In media.type.instagram, I see source: 'oembed:instagram' but in the other config files there isn't a source. Does anything look a miss?

...
id: media.instagram.embedded
targetEntityType: media
bundle: instagram
mode: embedded
...
robpowell’s picture

Summary from slack:

drush cget media.type.instagram

id: instagram
label: Instagram
description: 'Instagram posts.'
source: instagram
queue_thumbnail_downloads: false
new_revision: false
source_configuration:
  source_field: embed_code
  use_instagram_api: 0
field_map: {  }

Next steps are to use a debugger and step through the update hook.

robpowell’s picture

I initially experienced this issue on Acquia hosting and was able to replicate when I brought the database down locally. However, when I restored the db and did the drush updb, it worked. I confirmed it also worked in acquia. To test, I pushed up my code that had the new config changes, restored database from production and re ran the updb. Afterwords, I checked the field's config via drush, drush cget media.type.instagram:

id: instagram
label: Instagram
description: 'Instagram posts.'
source: 'oembed:instagram'
queue_thumbnail_downloads: false
new_revision: false
source_configuration:
  source_field: embed_code
  use_instagram_api: 0
field_map: {  }

I'll update if I bump into any other issues when we deploy.

bramdriesen’s picture

Thanks for the pointer @robpowell encountered a similar issue as you.

kkohlbrenner’s picture

We recently observed this bug when deploying updates to our non production Acquia environments. However, in addition to this post update, we also were trying to process a custom post update hook. Our custom hook is fairly resource intensive (updates a field value on (almost) all nodes) and once we removed the custom post update hook (we'll deploy/execute it in a future release) the error was resolved. Just something to look out for, if you are executing additional resource intensive updates along with this update.

useernamee’s picture

Hello, I'm still trying to resolve the issue. We are actually not using lightning_media but thunder_media module but the bug is the same.

Update hooks went through but now the status report page and the media type instagram pages are not working due to php error because of missing plugin instagram. When I try to edit configuration with `drush config:edit media.type.instagram` and I set `source: 'oembed:instagram' I get:

 [error]  Drupal\Core\Config\ConfigException: Errors occurred during import in Drush\Drupal\Commands\config\ConfigImportCommands->doImport() (line 247 of /app/vendor/drush/drush/src/Drupal/Commands/config/ConfigImportCommands.php). 

In ConfigImportCommands.php line 261:
  The import failed due to the following reasons:                                                               
  Unerwarteter Fehler bei Operation update für media.type.instagram: The "instagram" plugin does not  
   exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: file, oembed:video, audio_file, video_file  
  , image, oembed:instagram, pinterest, slideshow, twitter, video_embed_field       

Either the oembed: part is filtered out or it's trying to access the old value.

**update**

I was able to resolve the issue by ssh-ing into server and then running postupdate function with php console:

$ drush php
>>> require_once('/app/web/modules/contrib/media_entity_instagram/media_entity_instagram.post_update.php');
=> 1
>>> media_entity_instagram_post_update_change_source_name();
=> null
>>> exit
ultrabob’s picture

The steps in #14 worked for me. Thank you!

cobenash’s picture

#14 works for me. Thanks.

volker23’s picture

Solution in #14 worked here, too! Thanks!

happy047’s picture

Solution #14 worked for me also! Thanks !

srihari manepally’s picture

#14 worked for me. Thanks.

guardiola86’s picture

Solution #14 works for me. Thanks!

camoa’s picture

It seems to be, that when the update happens, the active config gets changed, but sometimes we forget that after drush updb touches or changes config it is necessary to export before reimport.

When doing updates is very important to run all config import into the DB on local, before running updates, then run updb and after that export config for any changes that came from the update.

andrimont’s picture

useernamee
How to set the php command ? I never used the drush php command. I get an error bash: syntax error near unexpected token but probably I am not using it well.
Is there any other way to correct this issue ?
I tried to remove lightning_media but still I get

In DiscoveryTrait.php line 53:

  The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: video_file, image, oembed:vi
  deo, audio_file, file, oembed:instagram, pinterest, slideshow, twitter

Thanks for your help.

I still try to rerun the site :
How to use this php command ?

$ drush php
>>> require_once('/app/web/modules/contrib/media_entity_instagram/media_entity_instagram.post_update.php');
=> 1
>>> media_entity_instagram_post_update_change_source_name();
=> null
>>> exit

As drush php gives :

In Configuration.php line 614:

  Unable to create PsySH runtime directory. Make sure PHP is able to write to /run/user/1000 in order to continue.
ihamadet’s picture

#2 did work for me, thank you!

yash_khandelwal’s picture

I tried solution #14 but getting PHP Fatal error: Failed opening required '/modules/contrib/media_entity_instagram/media_entity_instagram.post_update.php' in Psy Shell code on line 1

Kindly share solution of error "Drupal\Component\Plugin\Exception\PluginNotFoundException: The "instagram" plugin does not exist. Valid plugin IDs for Drupal\media\MediaSourceManager are: audio_file, file, embed:video," as I am using media_entity_instagram module.

jaykandari’s picture

As mentioned in #22.
drush ev "require_once('modules/contrib/media_entity_instagram/media_entity_instagram.post_update.php'); media_entity_instagram_post_update_change_source_name();"
Worked for me.

useernamee’s picture

You can also try doing sth like:

/**
 * Implements hook_update_N().
 *
 * Fixes missing instagram plugin.
 */
function your_custom_module_update_8007() {
  // Maybe check that media_entity_instagram module is enabled.
  require_once '/app/web/modules/contrib/media_entity_instagram/media_entity_instagram.post_update.php';
  media_entity_instagram_post_update_change_source_name();
}
jdesrig’s picture

Solution #14 works for me. Thanks!

smulvih2’s picture

Changing the source value in the media.type.instagram config worked for me:

drush cset media.type.instagram source oembed:instagram

etiennejacquot’s picture

Solution #28 worked for me, thanks @smulvih2