Entity lookup plugin not able to guess config for entity types without bundle

I'm running into the same issue. Is it possible to create a standard solution for all entity types or will we have to write exceptions for known entity types like users?

A general solution should be possible here.

Let's get #2760443: Tests for entity lookup/generate plugins committed and add test for this.

@rodrigoaguilera - as a workaround for now, you may be able to configure the various lookup fields in your process plugin configuration so that $this->destinationBundleKey never has to be used:

my_destination_field:
  plugin: entity_lookup
  source: my_source_field
  value_key: name
  entity_type: user
  bundle: 1
  bundle_key: status

Since there is no actual bundle key, but a bundle key has to be required by the current code in order to avoid getting those inexistent definitions, this uses the 'status' property of users to query on (since once a bundle key is set, it will be queried on, so needs to be for a property that does exist), querying to users that are active (i.e. status = 1). I haven't tested it, but I think that might work for you?

Obviously the best solution would be to fix the issue itself, but other work (primarily adding tests) needs doing in the module first, so I hope this workaround suffices for you in the meantime perhaps.

Not sure if this is relevant, but I had a similar issue due to not having this in my YAML-

  type:
    plugin: default_value
    default_value: <entitle bundle>

I am trying to import csv data to an entity without bundle and Migrate + Migrate plus + Migrate Source CSV should do the job.
I cannot find any documentation on Migrate to custom entities and I have tried to use the process plugin as described above, but I am not able to make it work, probably due to misconfigurations. Can someone please explain me value_key entity_type and bundle_key? A working sample yml file for import to a custom entity without bundles would take me a long way...

my_destination_field:
  plugin: entity_lookup
  source: my_source_field
  value_key: name
  entity_type: user
  bundle: 1
  bundle_key: status

@tnfno you will need to change the 'entity_type' key to your custom entity type's ID. Then the 'bundle_key' needs to match a property/field on your entity type that will exist, that will always have the same value (at least for the entities you need to import), which is what you should set 'bundle' to. So that example works for user entities, which don't have bundles, because they all have a status property, that can be filtered to the value '1' (i.e. only active users are imported). So if you don't have any properties/fields that are the same for all of your custom entities, then you may well be stuck without this bug being solved.

As for documentation, if you're able to read PHP code, I suggest that the EntityLookup class is the best/only place you can really go to at the moment? The determineLookupProperties() will take the various bits of configuration, or try to apply some defaults, and store them in properties that the query() method will then use to do the looking up.

Does that make sense? Perhaps it might help if you could share some more about your custom entity, such as what properties/fields your custom entities have?

Hi and thank you, it makes perfect sense. My custom entities are pretty simple, made with Drupal console and some fields added in the GUI.

entity_type
is OK, I already got that right.

If I understand it correctly I can the just add another field to my entity that I set to any value that are the same for all entities and use that field name as bundle_key? But in the example above, does bundle: 1 mean that the field bundle_key: status is set to 1 during import? Does the exisiting entities need to have this value set to the same value as well, or must the field just exist for lookup purposes?

Finally, the value_key: name, does name her come from a field defined in the process: part or is that the fieldname in the entity? I would presume that my_destination_field: is the fieldname in the entity and it is set to the value in value_key: ?

But in the example above, does bundle: 1 mean that the field bundle_key: status is set to 1 during import?

No, it means that during the import, existing entities are checked to see whether there are any that exist with their status field set to 1, as well as having the matching value_key field ('name' from the original example for users). If there was one, then that entity would be updated/replaced with the values from the row being imported.

But are all your entities just going to be newly created, or are you needing to update existing entities that you will want to match? If they are all going to be newly created, then it doesn't matter what the bundle value is, as you never actually want to match existing entities! And rather than using the 'entity_lookup' plugin, you will need to use the 'entity_generate' one (which still needs the other bits of configuration too, as it shares code).

Finally, the value_key: name, does name her come from a field defined in the process: part or is that the fieldname in the entity? I would presume that my_destination_field: is the fieldname in the entity and it is set to the value in value_key: ?

The 'name' is the field name in the entity. So for users, it's matching on username to find existing users with the name found in the my_source_field column. The my_destination_field is the field that the entity is to be referenced in. Are your custom entities being referenced in another entity that is being imported from your CSV, or are you just looking to import your custom entities on their own? If the latter, then you should be using the entity destination plugin, not an entity lookup/generate process plugin.

Ok, thank you that clearified a lot, will test again myself. I normally find my way around with some documentation and samples, but this one has been tricky.

I will first do a clean import, but later update the same entities against updated csv file. So entity_update will only update existing, while engity_generate will both update and add and can be used, but entity destination plugin would probably be more efficient for the initial import.

Final question: can entity_update / entity_generate match against more than one value_key field? For example if you had first and last name in different fields and wanted to match against both?

I will first do a clean import, but later update the same entities against updated csv file. So entity_update will only update existing, while engity_generate will both update and add and can be used, but entity destination plugin would probably be more efficient for the initial import.

To be honest, just going from what you have said, it does sound to me like you do need to be using the entity destination plugin. The entity_lookup (it's not called entity_update) & entity_generate process plugins are only intended for entities being referenced by other entities, which you haven't mentioned.

Final question: can entity_update / entity_generate match against more than one value_key field? For example if you had first and last name in different fields and wanted to match against both?

No, it does only match against a single field & value.

The initial plugin tests (#2760443: Tests for entity lookup/generate plugins) have been committed, and #2826636: Add non basic tests for entity lookup/generate plugins exists for some more specific ones that would cover this original issue.

Back to NW because needs tests.

Applied the patch and ran an import. I still get "The entity_lookup plugin requires a value_key, none located."

  field_content_ref_contact:
    -
      plugin: skip_on_empty
      source: contact_full_name
      method: row
    -
      plugin: entity_lookup
      source: contact_full_name

I was hoping this would work as I am trying to get the following to work

field_content_ref_contact:
    -
      plugin: skip_on_empty
      source: contact_full_name
      method: row
    -
      plugin: entity_lookup
      source: contact_full_name
      value_key: title
      entity_type: node
      bundle_key: type
      bundle: contact

and that returns absolutely nothing in the $Destination. The field I am importing into doesn't even appear in the destination list.

I would go further than this -- the bundle key configuration is superfluous, since the bundle key of an entity can always be deduced when you know the entity type: #2938112: [PP-1] bundle_key configuration for entity_lookup process plugin isn't necessary.

Minor update to the patch from #15

• I found the old patch didn't apply on current code
• I reduced the patch to exactly what is needed to solve this issue, leaving "The bundle key can be easily fetched if entity type is known." to #2938112: [PP-1] bundle_key configuration for entity_lookup process plugin isn't necessary, and removing what I found was a rather confusing 4 part if statement.

@heddn I appreciate this is NW because of the tests, I will set it back, but please can we let the tests run first.

NW for tests

There is also #2938112: [PP-1] bundle_key configuration for entity_lookup process plugin isn't necessary and #2960251: Remove entity_lookup bundle requirement which also have some bearing to this. I'm not ready to close them down as duplicate to this, but we could easily incorporate those two things into here pretty easily.

I added a test coverage and made some changes to the plugin. There is no interdiff, but I tried to explain what I did in MR comments.
It's looking for review.

Nice test coverage.

@heddn thanks for the review! Unfortunately, there were some test failures and when I started looking into it, I realized I'm not satisfied with the current state. At a minimum, it doesn't support base fields. For example, if the destination is a node I would try to look up a user by `uid` property, it should be obvious that we're searching for users because the `uid` is a reference to the user entity. At this moment, it won't guess. There are also other things to improve. I'd like to dedicate more time to it, so I'm pushing this back to NW.