We need to port the D7 feeds_tamper EFQ Finder plugin to D8 tamper.
| Comment | File | Size | Author |
|---|
Issue fork tamper-2976175
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
jamesdixon commentedComment #3
megachrizI recommend to rename this plugin to "Entity finder", as I think that name makes more sense to the end user. "EFQ" is more a developer term and it refers to the technique that is used to find the entity.
Comment #4
jamesdixon commentedSounds good to me, I had no idea what EFQ Finder even meant!
Comment #5
vijay.mayilsamy commentedComment #6
vijay.mayilsamy commentedNote to @myself - WIP Patch
Comment #7
chegor commentedLooks ok!
Comment #8
volkswagenchickTagging for DrupalCamp Asheville
Comment #9
volkswagenchickTagging for the next to North American contrib days, Asheville and Colorado
DrupalCamp Asheville contrib days are July 13-14, 2019
DrupalCamp Colorado contrib day is Aug 4, 2019
Comment #10
meecect commentedany other movement on this? I installed the patch but it doesn't seem to be complete. I don't see Entity Finder in the list of available plugins in the feeds tamper.
What still needs to be implemented to make this work?
Comment #11
meecect commentedok, I dug into this a bit...I think I see the components that are missing or wrong. I'm going to take a crack at finishing it and will share here when I am done.
Comment #12
meecect commentedHere is the new EntityFinder.php file. I was able to use it to import members into my OG groups, ie, it was able to create OG Membership entities using this tamper plugin to convert the 'group title' to an entity_id.
Comment #13
jhodgdonThis works great (patch above + revision in comment #12)! +1 for adding it to Tamper. There is one error saying that the $value variable is undefined in line 97 of this EntityFinder.php file, but it seems to work anyway.
Comment #14
davidburnsThis does work, but still a few warnings and notices that need to be cleaned up and a re-rolled patch which includes comment #12.
Comment #15
megachrizIt would be cool if the changes from #12 are incorporated in the patch and that the tests cover the issues noted in #14.
Comment #16
jhodgdonHere's a start anyway:
- The first interdiff is the differences between the patch in #6 and the plugin file from the text in comment #12.
- The second interdiff is an untested attempt to fix the errors from #14 (which also cover the errors I mentioned in #13).
- The new patch has both of these changes.
I'm not currently sitting on the machine where I can run tests, so let's see if we can get the test bot to respond...
Comment #18
megachriz@jhodgdon
Thanks for giving it a go.
A few things I see in the patch:
\Drupalin the code that needs to be replaced with dependency injection. This is required to get a useful unit test.SafeMarkup::checkPlain()is deprecated.'update' is not a configuration option for the plugin, so this can be removed.
I know this is not the easiest Tamper plugin to port. Keep up the good work!
Comment #19
jhodgdonYeah, I didn't really look at the tests, or the code... I tested the plugin with the code in #12 and it seemed to work, so we used it. And I at least made the existing working (probably? mostly?) code into a patch, so anyone needing this functionality can at least use it for now. But definitely, it needs some work!
Getting this totally viable is probably not my highest priority right now. I'm kind of in the middle of about 3 other fairly intensive efforts with my Drupal contribution time... so hopefully someone else will take it on.
Comment #20
liquidcms commentedCurious what release this patch is against. I know it should be against -dev; but -dev is older than the beta2 release (how does that happen?) and the patch doesn't apply there. So wondering what is lost if using -dev and applying patch there?
I'll look at manually applying changes to beta2 as i assume it is more recent than -dev.
Comment #21
liquidcms commentedah.. my bad.. this is on Tamper, not Feeds Tamper.
Comment #22
liquidcms commentednow that i have patched the correct module....
- i have a feed value for Email that i want to match to a user account (Authored By for the Node i cam trying to create), it shows as this in Mapping: Authored by (uid): The username of the content author. Reference by: User ID
- I see the tamper action: Entity Finder and UI seems to work as i'd expect.
- in EF tamper i set entity/bundle to User and set field to Email
when i try to import, i get this error:
Drupal\Core\Entity\Query\QueryException: '' not found in Drupal\Core\Entity\Query\Sql\Tables->ensureEntityTable() (line 367 of core\lib\Drupal\Core\Entity\Query\Sql\Tables.php).
Comment #23
liquidcms commentedi think i have found the issue.
in the tamper you add a query condition like this:
$query->condition($this->getBundleKey($entityType), $bundle, '=');
and getBundleKey is this:
\Drupal::entityTypeManager()->getDefinition($entity)->getKey('bundle');
For "node" bundles this returns "type", which is a filter on the items in the node table to limit search to this bundle. For "user", this returns "" and causes the error I reported above.
I made a small tweak to the code such that if there is no bundleKey then don't include that condition. Not sure this will work for all entity types; but it now allows it to work with users (which is probably a pretty popular one to set content ownership).
Comment #24
jamesdixon commentedThanks everyone. Queue up the test bot to see if the latest patch passes.
Comment #25
jamesdixon commentedLets see if we can get tests passing and try and test this one out manually to ensure it's working.
Comment #26
zabej commentedHello @jamesdixon,
As far as I know patch-23 addon (not independent) to patch-16. Why does patch 23 not include 16....
Sure we can merge them, but patch 23 not passed.
Just for clear understanding to complete the issue we need to
1. Apply and test patch 16
2. If it needs to be fixed do it and retest.
3. If it passed successfully then merge a code from patch 23.
4. Name should be like entity-lookup-2976175-xx.patch
And a question about interdiff. Should it be compared with 16?
Will be thanks full for any comment
Comment #27
jamesdixon commented@zabej: Thanks for your comment. It looks like the patch in #23 is not a full patch as you correctly identified. Lets try rolling patches #16 and #23 together.
1) Apply patch #16
2) Apply patch #23
3) Profit (hopefully)
Comment #28
jamesdixon commentedI don't feel I was super clear there so here's a code example of what I mean:
1) cd tamper
2) wget https://www.drupal.org/files/issues/2019-08-08/2976175-16.patch
3) git apply -v 2976175-16.patch
4) wget https://www.drupal.org/files/issues/2020-03-17/entity-lookup-2976175-23....
5) git apply -v entity-lookup-2976175-23.patch
6) Test it out
7) If it fails tests we may need to add that 4th parameter when creating the test objects
Comment #29
tbenice commentedI combined the two patches. The first one did not apply any more against 8.x-1.x, so I re-created. Not sure what to make an interdiff against since this is combined so will leave that alone.
Comment #31
zabej commented@jamesdixon,
I've cleaned the code and add several fixes here. But there still exits an error with Drupal::setContainer().
I will talk with @megaChriz
Screenshot is attached.
Comment #32
zabej commented@jamesdixon updated code
Comment #33
zabej commentedWhen I added fourth parameter ($this->getMockSourceDefinition()) creating new object of EntityFinder class
There appears error:
It is the same for
testEntityFinderWithSingleValue
as well as
testBuildConfigurationForm
methods. The last one out of the test file.
The error links to the lines in EntityFinder.php file
in EntityFinderTest.php
I've made online search around the similar issues.
https://drupal.stackexchange.com/questions/258233/how-to-create-a-test-c...
Based on the link above I guess it is something around declaration and access to particular methods from test class and from tamper class to drupal core. Need help and advice of more experienced developers.
I've added the patch derived from the comment #29. Fixed code style there following drupal linter and const SETTING_GLUE constant
Comment #34
megachrizThe entity type manager should be injected into this class. This way you could do
$this->entityTypeManager->getDefinition()instead of\Drupal::entityTypeManager()->getDefinition().To inject the entity type manager, you need to:
$entityTypeManager.EntityTypeManagerInterface.create()method - Implement\Drupal\Core\Plugin\ContainerFactoryPluginInterface.For an example, see
\Drupal\feeds\Feeds\Target\EntityReference, which is also injecting the entity type manager.Next, in the unit test, you would need to mock
\Drupal\Core\Entity\EntityTypeManagerInterfaceand make sure that it returns the correct thing forgetDefinition(). That thing may also be a mock that should return something for the call togetKey().You could take a look at what the unit test for
\Drupal\feeds\Feeds\Target\EntityReferencedoes in\Drupal\Tests\feeds\Unit\Feeds\Target\EntityReferenceTest, though it could look a bit overwhelming when seeing that code for the first time. (I remember it being a bit overwhelming to me when I saw it for the first time.)Comment #35
jamesdixon commentedAdding code examples to support @megachriz's suggestions:
Comment #36
jamesdixon commented@zabej and I took a crack at this and @zabej will be posting results at some point.
Comment #37
zabej commentedHello @megachriz,
We've generate such a code (attached). Is it a right direction on you opinion?
Test gives the 8 errors. We continue to work on it
Comment #38
jamesdixon commented@zabej: Looks like we need to call parent:setUp() at end of setUp() function instead of beginning. That may fix issues.
If that doesn't work maybe we need to call $this->setUp() from top of instantiatePlugin().
Comment #39
megachriz@zabej
create()method does need to pass a SourceDefinition object as well to the constructor. Looking at TamperManager that object appears to be in$configuration['source_definition']:use Drupal\tamper\SourceDefinitionInterface;I think you won't need all of this code (but you do need some of it), but I suppose you're were still working on this bit?
Comment #40
zabej commented@jamesdixon thanks for the advice I tried both suggestions but they do not work
The error move to
for tests are absent in the EntityFinderTest directly:
For example,
@megachriz
I'm a bit confused with you remark.
1.1. In the create method we pass the $configuration array. Do we need put this in the constructor directly as a separate parameter?
1.2 Or we need to add createInstance method where provides Factory method and pass all the parameter as they are in it?
1.3 Is it not enough to SourceDefinitionInterface in the constructor having the parameter already?
3. Honestly, I would be delete almost all, because a lot of code is not familiar what it does
Remain only like this
Comment #41
megachriz$source_definitionneeds to be passed to the parent.This needs an extra line between
$plugin_definitionand$container->get('entity_type.manager').Most likely this should be
$configuration['source_definition'], based on the code fromTamperManager::createInstance().Not sure if I understood this question, but we don't need a new factory or
createInstance()method.SourceDefinitionInterfacedoes not exist in the current namespace and thus it needs to be referenced at the top of the file.SourceDefinitionInterfacelives in the namespaceDrupal\tamperand the plugin's namespace isDrupal\tamper\Plugin\TamperAdd the following line:
use Drupal\tamper\SourceDefinitionInterface;That's the hard part indeed. Try if you can break it down in parts so it becomes somewhat understandable.
Comment #42
jamesdixon commentedI pushed this along a little further.
Some classes needed including through use at the top and followed direction from @megachriz. Thanks for the help.
We're closer but I think we need to mock up more test functionality probably surrounding the entity type configuration for the EntityFinder.
Now tests are giving me:
The EntityReferenceTest example got us this far, and now I believe we need to determine what mock functionality we need specifically for the EntityFinder Tamper plugin as it's requirements are different.
Comment #43
danharper commentedI've applied this patch but I'm now getting this error when editing any of my feeds that previously used the entity finder.
ArgumentCountError: Too few arguments to function Drupal\tamper\Plugin\Tamper\EntityFinder::__construct(), 4 passed in /app/web/modules/contrib/tamper/src/TamperManager.php on line 50 and exactly 5 expected in Drupal\tamper\Plugin\Tamper\EntityFinder->__construct() (line 46 of modules/contrib/tamper/src/Plugin/Tamper/EntityFinder.php).Comment #44
liquidcms commentedYes, i had this patch installed and working for a while; but haven't touched the importer in a while. Since then i migrated all contrib over to composer install and tried to make sure i included all the patches. Missed this one. Now that i have added the latest patch i get the same reported in #43.
.. and by "editing any feed", i think @danharper likely means when going to Tamper tab of any Feed Type.
Comment #45
liquidcms commented#29 still works.
Comment #46
jamesdixon commentedMaking notes from @megachriz:
1)
I think the following line is wrong in the Entity Finder class:
$query = \Drupal::entityTypeManager()->getStorage($entityType)->getQuery();
The code should not call \Drupal::entityTypeManager() here, but use the injected service. Use $this->entityTypeManager instead.
2) I see there are more \Drupal::service() calls, they should all be replaced with injected services.
Comment #47
jamesdixon commentedMade some progress based on @megachriz's feedback. The latest patch is a work in progress, but likely isn't functional yet.
Comment #48
jamesdixon commentedMade some more progress on the test. Since Entities have so many moving parts, there's lots of classes to prophesize!
Now I'm working on getting the prophesized QueryInterface $entity_query returning the correct values. It's returning null for it's functions instead of behaving properly. I've included the QueryInterface class up top, trying to figure out what's up there.
This is likely not a functional patch so I'd avoid using it in production.
There were 2 errors:
1) Drupal\Tests\tamper\Unit\Plugin\Tamper\EntityFinderTest::testEntityFinderWithMultipleValues
Error: Call to a member function range() on null
/app/modules/tamper/src/Plugin/Tamper/EntityFinder.php:259
/app/modules/tamper/tests/src/Unit/Plugin/Tamper/EntityFinderTest.php:103
2) Drupal\Tests\tamper\Unit\Plugin\Tamper\EntityFinderTest::testBuildConfigurationForm
Invalid argument supplied for foreach()
/app/core/tests/Drupal/Tests/Listeners/DeprecationListenerTrait.php:164
/app/modules/tamper/src/Plugin/Tamper/EntityFinder.php:170
/app/modules/tamper/src/Plugin/Tamper/EntityFinder.php:93
/app/modules/tamper/tests/src/Unit/Plugin/Tamper/TamperPluginTestBase.php:86
Comment #49
jamesdixon commentedActually correction: $entity_query is not being returned at all by the EntityStorage object, it's coming back as NULL. That's the issue. Not sure why the getStorage() function is failing to return $entity_query properly.
Comment #50
jamesdixon commentedMade more progress.
Still running into the same issue but at least crafted the $entity_query->condition() statement closer to what it should, and made $entity_query->execute() return what it is supposed to I think.
For some reason EntityFinder is still returning NULL on it's $entity_query->condition() call so I need to see what kind of query parameters it's actually passing and make sure the test matches up as a next step.
Gotta love debugging. :)
Comment #51
liquidcms commentedtried #50 but still getting error reported in #43
tried #29 in D9 (last patch i know still works for D8) but now getting this error:
Error: Class 'Drupal\Component\Utility\SafeMarkup' not found in Drupal\tamper\Plugin\Tamper\EntityFinder->getEntityTypes() (line 138 of modules/contrib/tamper/src/Plugin/Tamper/EntityFinder.php).
due to use of deprecated SafeMarkup.
would provide patch to fix this but unclear the direction since #29 as i don't think any of those work.
Comment #52
kazah commentedAny update for drupal 9.3
Comment #53
liquidcms commentedWent back to patch from #29 and fixed the deprecated method issues and it worked with D9.5
Comment #54
ericgsmith commentedHello all.
Firstly I wanted to say thank you to everybody who has contributed to this issue. It has been a very long slog and so many people have contributed.
I have taken a quick look and made some tidy ups.
Please consider this a work in progress, and I have removed a bunch of stuff to get the tests going. I am not trying to discourage those ideas, I just wanted to share progress before I put them down.
Namely
- removed entity repository. Querying by uuid still works with entity query, I couldn't see why we needed to introduce another service for this field
- used a Kernel test. We can maybe revisit this, but this is a complicate plugin - the current state of the unit test was not complete, and being unfamiliar with the work so far it was easier for me to look at this with a kernel test.
Now a warning - I only looked at this via tests - I have not even looked at the UI or how that renders.
Adding patch now but it is still needs a few improvements. I will try tidy up the issue summary and remaining steps soon.
Comment #55
ericgsmith commentedI need to revisit the form side - this should either be there or removed, advice welcomed on what this is for and what other plugins are doing.
Should be $this->t
Case needs to match where this is called
Comment #56
ericgsmith commentedOops - wrong namespace in test - ignore 54
Comment #57
ericgsmith commentedOk cool, green test! A few coding standard violations in the plugin and test noted in ci
Comment #58
kopeboyThank you for the work you're doing to finally take tamper out of alpha stability! 🙏🏻 Unfortunately I'm not skilled enough to help directly with code but I'm with you emotionally and can test when ready 💪🏻
Comment #59
jrochate commentedDoes not work with current DEV branch.
Comment #61
megachrizI'm hiding all the patches. Work should continue in the MR.
And I wonder if this plugin should support config entity types? I limited it to content entity types for now, because I accidentally selected a config entity type and it resulted into the error that the entity type did not have base field definitions.
Comment #62
hepabolu commentedFYI I've created a similar tamper module: https://github.com/hepabolu/feeds_tamper_lookup_entity
This one works for me, but since this is my first Drupal module feel free to correct and improve or assimilate into your version.
Comment #63
megachrizTests are passing! This looks ready for a new review!
Highlights of the changes:
@hepabolu
Thanks for your contribution 🙂. I see that your plugin has one feature that looks useful that Entity Finder currently doesn't have: a return field. Entity Finder now always returns the entity ID, but being able to select a different field to return sounds very useful.
I'm not sure yet if we should add it to Entity Finder right now (since this issue has been open for a very long time already) or if it would be better to add it in a follow-up. Do you want to help adding that feature here?
Comment #65
megachrizI merged the code! Thanks all who contributed to this issue.
Comment #66
megachrizI've opened a follow-up for adding an additional configuration option (called "Return field") for the Entity Finder plugin: #3509334: Entity Finder: add return field option
The idea comes from @hepabolu who added this option to a plugin similar to Entity Finder in https://github.com/hepabolu/feeds_tamper_lookup_entity, so it would be great if we could port that option to the Entity Finder plugin.