Drush comand to export all items from entity with defined criteria

Really useful !

I had the same need, to have default content only for taxonomy terms from a specific vocabulary that had too much terms to export them individually.

However, it seems this patch doesn't apply, rerolled patch attached.

Promising, same usecase as pquillard.
Patch confirmed as working.
Quickstart:
export terms of given taxonomy:
$ drush default-content-export-all taxonomy_term vid yourvocab --folder=modules/custom/yourmodule/content

An "import" drush command would be nice ;-)

This is a very useful patch and works pretty well. However there are a few issues and I don't think it should be RTBC:

• It exported UID 1, despite code to prevent this
• Drush alias should probably be dcea not dceall to match other aliases
• Coding style issues

Made the following changes:

- Removed the code that is supposed to prevent admin user from being exported as it only works if the user entity_type is specified. This is to provide consistent behaviour between all entity types.

- Removed the try catch since this only works in certain situations as well when incorrect property type is specified. If you specified an incorrect entity_type, it would generate a PHP error. This is now consistent with the other default_content drush commands.

- Fixed code style issues.

Marking this as needs work because a test needs to be added.

I reworked the patch and simply integrated the property name and value parameters into the existing dcer command. So instead of:

drush dcer node 1

You can now pass a property name and value such as:

drush dcer node nid 1

Let's try this again.

Here is an updated version for the latest version:

Re-rolling patch default_content-export-all-items-2786479-8.patch (from comment #9)

I'm a bit confused. Initial issue is about exporting all items from an entity, patches till #5 doing that fine by adding `default-content-export-all` command. What happened since #7? Where this feature has gone? How patch #12 suppose to export all items from the entity? Can anybody give example how one can export all items (e.g. from the given vocabulary) using the latest patch?

Ok, I got it, applied patch from #12 and the following command seems to work (doesn't generate any errors), but giving the empty list:

drush dcer taxonomy_term vid action_codes

I'm testing this right?

Applied #12 to dev, works fine.
@kenorb: add optional '--folder=' parameter to specify path, otherwise it will export to current directory.

Here is an updated patch for the DRUSH 9 integration aswell.

can this integrate with https://www.drupal.org/project/taxonomy_machine_name?

Just did the reroll stuff

The patch from #21 works well for me when exporting entire vocabularies using Drush 9.

I'd say this needs changes for drush <9 and drush 9 consolidated in a single patch? Also, isn't there a way to share code between the two (DRY)?

For #23

Needs tests too?

it should tell you if it can write to the file specified.

Re-rolling with Drush 8 support.

Here's the patch for the latest release 8.x-1.0-alpha8 of default_content.

Thanks, this works great for exporting all terms in a vocabulary, like:
drush dcer taxonomy_term vid tags --folder=modules/custom/my_custom_module/content

However, I cannot figure out how to do this to export all menu items.
I do not get any error if I do this:
drush dcer menu_link_content id main --folder=modules/custom/my_custom_module/content
But no menu link is exported.

If I replace 'main' with 'thereisnomenuwiththisid' then there is also no error, so its just not doing anything.

If I change the code to:
drush dcer menu id main --folder=modules/custom/my_custom_module/content
then I get error:
Entity "menu" with ID "main" is not a content entity

Please help on exporting all menu links, as was mentioned in the very description of this issue.

UPDATE: never mind, figured it out:
drush dcer menu_link_content menu_name main --folder=modules/custom/my_custom_module/content

Confirming that this worked as expected for exporting all terms in a hierarchical taxonomy vocabulary.

Drush 10.2.0
Drupal 8.8.5
Default Content 8.x-1.0-alpha8

Added merge requests both on 8.x-1.x and 2.0.x based on #27

I'm new to the mixed workflow between pull request and patches. For my composer.json file, I created a patch on top of the 2.0.0-alpha1 tag that seems to apply: https://git.drupalcode.org/issue/default_content-2786479/-/commit/b30f37...

@dasjo It's also possible to base a patch on an entire MR, see https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa...

I did just realize this probably means your pipeline will apply a different patch as soon as somebody changes the PR, which could be risky. It'd be nice to be able to easily get a patch based on two commits instead.

BTW, you do not need to bother with the git commit message format; it will be applied when merging an MR from the Drupal.org interface, which will do a rebase, apply the commit message template (or rather, apply the commit message that is also used for the "git command" box) and merge the result.

Updated the code accordingly as ::writeDefaultContent no longer exsists in 2.0.x branch.

This patch is based on #31.

1. +++ b/drush/default_content.drush.inc
   @@ -25,7 +25,8 @@ function default_content_drush_command() {
          'entity_type' => dt('The entity type to export.'),
   -      'entity_id' => dt('The ID of the entity to export.'),
   +      'property_name' => dt('The property name of the entity to match.'),
   +      'property_value' => dt('The property value of the entity to match.'),
        ],
   

   This changes how the command works, that seems problematic, even for an alpha release version.

   We could keep support both in case property value is empty? or use parameters instead of arguments.

2. +++ b/src/Commands/DefaultContentCommands.php
   @@ -54,24 +54,18 @@ class DefaultContentCommands extends DrushCommands {
        $folder = $options['folder'];
   -    if (is_null($entity_id)) {
   -      $entities = \Drupal::entityQuery($entity_type_id)->accessCheck(FALSE)->execute();
   -    }
   -    else {
   -      $entities = [$entity_id];
   -    }
   +    $properties = $property_name ? [$property_name => $property_value] : [];
   +    $entities = \Drupal::entityTypeManager()->getStorage($entity_type_id)->loadByProperties($properties);
        // @todo Add paging.
   -    foreach ($entities as $entity_id) {
   -      $this->defaultContentExporter->exportContentWithReferences($entity_type_id, $entity_id, $folder);
   +    foreach ($entities as $entity) {
   +      $this->defaultContentExporter->exportContentWithReferences($entity_type_id, $entity->id(), $folder);
   

   it would require fewer changes in the two drush commands if we just dynamically add a condition in the entity query, that's just what loadbyProperties() also does.

Changed the code to use options for property queries.

I moved the logic that fetches the entities to be exported, into the exporter service.

This way we can call on this functionality on both the drush 8 and 10 commands, with the least possible duplicates. Furthermore, this way we don't have the problem mentioned in #37.

I did not add the resolveExportedEntities to the ExporterInterface, because I recon it will not be a mandatory implementation for someone who would want to create his own exporter and use the it in his own command for some reason.

I corrected a minor error in the comments, where the simplesitemap module was referenced :P.

I added some logic to export all entities for all entity types, if no entity type is provided.
I thought we could also try to allow property filtering with null entity_type_id by checking, if each entity type has the property in question, but concluded that it would become a true mess and it would be an overkill anyway, so I did not do it.

I have just tested this with drush 10, since I did not have the time to create a working environment atm for drush8, but it should be working fine. It would be great though, if someone could cross check it (Or I might find the time to do it myself).

In the drush 10 command i added a user prompt for the case of exporting the whole site content.

Finally I also provide a patch for the alpha1 tag, for the case that someone wishes to apply it before the 2.0.x is merged with the alpha1.

Edit: Just noticed the files should have been named ....-38 instead of ....-32, sorry for that :P
Edit2: The patch for alpha1 failed to apply, because i tested it agains 2.0. I don't know how to retest for the tag.

Reroll

I remove the changes in the drush.inc files, since we discussed that they will be removed anyway in another issue.

I also abort the execution of the command, if no entity id is provided but properties were given.
I ended up not throwing an exception, since the check is made in the command itself, so I just stop the execution after displaying an error via the logger.

If this is not the best way to go about it, let me know and I'll change it.

Added two more comments, then I think it's ready to go.

I made both the changes requested, changing it back to needs review.

Also I am not sure If I am the one who should mark the gitlab comments as resolved or the reviewer :)

hi @flyke
i was reading this discussion to find a solution for my case and started being confuse when i read your comment.
Please what did you exactly mean by the below

UPDATE: never mind, figured it out:
drush dcer menu_link_content menu_name main --folder=modules/custom/my_custom_module/content

I have this patch applied but not really sure on how to make your command work.

Thanks

Hi @delacosta456,
In the 4 years since that comment of me, I haven't really used the default_content module anymore.
drush dcer menu_link_content menu_name main --folder=modules/custom/my_custom_module/content
You need to literally use (copy/paste) this part:
drush dcer menu_link_content menu_name
You need to replace main with the actual menu id you want to export from.
For example, If you edit the menu you like, and the url in your browser for the menu edit page is something like: https://mywebsite.com/admin/structure/menu/manage/footer-main Then the last part from that url, footer-main, is your menu id. So in the drush command you need to replace main with footer-main so it becomes:
drush dcer menu_link_content menu_name footer-main --folder=modules/custom/my_custom_module/content

The last part you need to replace is this:
--folder=modules/custom/my_custom_module/content
If you have a custom module, located at modules/custom/my_custom_module, you need to create a folder 'content' in there. That is where the exported items will be placed after running the drush command.

hi @flyke
Thanks for your clarification.
However following your suggestion returned an error of too many argument

Below is the drush command i try to use :

drush dcer menu_link_content menu_name main --folder=modules/custom/brains_mtcb_default_content/content

and this return the error message below

Too many arguments to "dcer" command, expected arguments "entity_type_id" "entity_id".

i am i missing something ?
Thanks

Hi @delacosta456,

As I said, it's been four years, so I probably got it wrong.

I did retrieve my personal notes in a readme file from a custom module in one of our projects from years ago, maybe this still works and can help you:

EXPORT block content:
drush dcer block_content 1 --folder=modules/custom/my_custom_module/content
-> change '11' to the desired block id. You can see blockid in url when editing a block.
-> a file with a unique id string as filename will be generated, like: 55489d08-4eac-4818-9cea-e3cb38b70195.json
-> change file name to something more descriptive, likke: block-about-us.json

EXPORT menu items:
drush dcer menu_link_content 45 --folder=modules/custom/my_custom_module/content
-> change '45' to the desired menulink id. You can see the id in url when editing a menulink.
-> a file with a unique id string as filename will be generated, like: 55489d08-4eac-4818-9cea-e3cb38b70195.json
-> change file name to something more descriptive, like: menulink-contact.json

EXPORT node:
drush dcer node 1 --folder=modules/custom/my_custom_module/content
-> change '1' to the desired node id. You can see the id in url when editing a node.
-> a file with a unique id string as filename will be generated, like: 55489d08-4eac-4818-9cea-e3cb38b70195.json
-> change file name to something more descriptive, like: node-homepage.json

EXPORT user:
lando drush dcer user 14 --folder=modules/custom/my_custom_module/content
-> change '1' to the desired user id. You can see the id in url when editing a user.
-> a file with a unique id string as filename will be generated, like: 55489d08-4eac-4818-9cea-e3cb38b70195.json
-> change file name to something more descriptive, like: user-contentmanager.json

Export menu items:
- create the menuitem (locally)
- edit it to see menu item id: admin/structure/menu/item/20/edit -> id = 20
- drush dcer menu_link_content 45 --folder=modules/custom/my_custom_module/content
- If that does not work, you have an older version of default_content and you should use:
- drush dce menu_link_content 45 --file=modules/custom/my_custom_module/content/menu_link_content/menu-link-overzicht-tweedehands.json
- repeat for all the menulinks you want to have (do not add menulinks that are automatically added via views pages)

And then lastly, I had code for importing contant via a update hook like this, but remember, this is from some years ago:

/**
 * Custom function to import content that was exported via default_content.
 */
function _my_custom_module_import($path_to_content_json) {
  list($entity_type_id, $filename) = explode('/', $path_to_content_json);
  $module_handler = \Drupal::service('module_handler');
  $module_path = $module_handler->getModule('my_custom_module')->getPath();

  $encoded_content = file_get_contents($module_path . '/content/' . $path_to_content_json);

  $serializer = \Drupal::service('serializer');
  $content = $serializer->decode($encoded_content, 'hal_json');

  // If entity already exists, do nothing.
  $entity_id = FALSE;
  if (isset($content["id"])) {
    $entity_id = $content["id"][0]["value"];
  }
  if (isset($content["nid"])) {
    $entity_id = $content["nid"][0]["value"];
  }
  if (isset($content["tid"])) {
    $entity_id = $content["tid"][0]["value"];
  }
  if (isset($content["uid"])) {
    $entity_id = $content["uid"][0]["value"];
  }
  if ($entity_id) {
    $entity_storage = \Drupal::entityTypeManager()->getStorage($entity_type_id);
    $existing_entity = $entity_storage->load($entity_id);
    if ($existing_entity !== NULL) {
      echo 'Skipping importing content because it already exists: ' . $existing_entity->label() . "\n";
      return;
    }
  }

  // If entity does not exist, make sure there are no revisions of it either, otherwise we still get error when saving.
  unset($content["revision_id"]);
  unset($content["revision_created"]);
  unset($content["revision_translation_affected"]);

  global $base_url;
  $url = $base_url . base_path();
  $content['_links']['type']['href'] = str_replace('http://drupal.org/', $url, $content['_links']['type']['href']);
  $contents = $serializer->encode($content, 'hal_json');
  $class = \Drupal::entityTypeManager()->getDefinition($entity_type_id)->getClass();
  $entity = $serializer->deserialize($contents, $class, 'hal_json', ['request_method' => 'POST']);

  $entity->enforceIsNew(TRUE);
  $entity->save();
}

/**
 * Add exported content via update hooks.
 *
 * Implements hook_update_N().
 */
functionmy_custom_module_update_8001() {
  _my_custom_module_import_all();
}

/**
 * Import all content that is inside this modules /content folder.
 */
function _my_custom_module_import_all() {
  $directory = \Drupal::service('extension.list.module')->getPath('my_custom_module') . '/content';
  foreach (\Drupal::service('file_system')->scanDirectory($directory, '/\.json$/') as $filename => $file) {
    // $filename = modules/custom/my_custom_module/content/block_content/footeraddress.json
    // remove module folder so $file = content/block_content/footeraddress.json
    $file = str_replace($directory . '/', '', $filename);
    _my_custom_module_import($file);
  }
}

Another review. I'm quite conflicted between this and #3001618: Add `drush dcer --bundles` option to limit export. It doesn't make sense to add both, and while this is more flexible, it doesn't support some things like multiple bundles that the other issue supports and it's more complex to use.

I think the menu use case is a good example why just bundles aren't sufficient, maybe we can expand it to also support comma separated values and add that as an IN condition to the entity query and take that from the other issue.

To be honest, not entirely sold on the new method. It needs a lot of work on docs, API changes on ExporterInterface and now that the old drush changes are removed, is really only used once. Instead of addressing all the review points, it might be easier to just inline the entity query into the drush command again and add the condition there. It doesn't even need a loop then since it really only supports a single condition.

@delacosta456: I recommend reading the built-in documentation that the drush command automatically provides, that would tell you that the property name and value are no longer arguments but options that you need to pass with --property-name and --property-value.

@berdir would it be overkill to have a settings form with the entities / bundles machine names selectable? Then we could get the entity types / bundles to export with the command via the config and simplify the exporter.

With the current diff from MR 34, I can't get menu links of the main menu to work.

Command:

drush dcer menu_link_content menu_name main --folder=../recipes/menus/content/

Error message:

Too many arguments to "dcer" command, expected arguments "entity_type_id" " entity_id".

If I try dce

drush dce menu_link_content main --folder=../recipes/menus/content/

I get the following error:

The "--folder" option does not exist

If I drop the folder part, I get:

Entity "menu_link_content" with ID "main" does not exist

For anyone that need any clarity on how to use the command:
drush dcer menu_link_content --property-name=menu_name --property-value=main --folder=location

The ability to export with bundle filtering has now landed in core: #3554978: ContentExportCommand::loadEntities() should use loadByProperties()