Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
Currently the feeds integration only works with GUID, need to make it work with entity id as well.
See #1341012-45: Feeds integration for Entity reference fields, #1341012-54: Feeds integration for Entity reference fields, #1341012-75: Feeds integration for Entity reference fields.
Comment | File | Size | Author |
---|---|---|---|
#91 | interdiff.txt | 0 bytes | slashrsm |
#91 | entityreference-feeds_integration-1616680-91.patch | 18.7 KB | slashrsm |
#66 | interdiff_64_66.txt | 1.1 KB | slashrsm |
#66 | entityreference-feeds_integration-1616680-66.patch | 18.7 KB | slashrsm |
#64 | entityreference-feeds_integration-1616680-64.patch | 18.62 KB | slashrsm |
Comments
Comment #1
jrao CreditAttribution: jrao commentedHere's a patch based on references module's feeds integration, only did some limited testing on node guid/nid import, more testing are welcome.
Comment #2
Damien Tournoud CreditAttribution: Damien Tournoud commentedThat would likely break existing configurations? Let's make sure the default key stays the same.
Comment #3
jrao CreditAttribution: jrao commentedGood point, didn't think much about existing configuration. So do we keep both $targets[$name] and $targets[$name . ':guid'], or change $targets[$name . ':guid'] to $targets[$name]?
Comment #4
Damien Tournoud CreditAttribution: Damien Tournoud commentedI think we just change
$targets[$name . ':guid']
to$targets[$name]
.Also, if you re-roll this patch, please remove your debugging code.
Comment #5
jrao CreditAttribution: jrao commentedOk, should be fixed now.
Comment #6
dealancer CreditAttribution: dealancer commentedTested for a Entity ID, works fine for me!
Waiting for more reviews and looking forward to see a commit of this patch!
Comment #7
dealancer CreditAttribution: dealancer commentedI see here a different indentation but it is a minor problem.
Comment #8
laurikar CreditAttribution: laurikar commented#5 worked fine for me except I couldn't apply it with Git and had to apply manually.
Comment #9
GaëlGI posted there first, but then found this issue which is a best place to submit my patch.
I used a more generic way of mapping against title/username: use entity label.
Comment #10
GaëlGBetter like this.
Comment #11
MTecknology CreditAttribution: MTecknology commentedI can't seem to make this work at all. I have one column in a CSV file for the mapping. Everything imports just fine but the entityreference field is left blank after the import finishes. No errors are produced.
I attempted to do this just to make sure this section of code is at least being entered for that field and indeed it is. The output of $instance looks like this..
Hopefully that's everything that's useful and not much more.
I don't know if I'm running into this bug or fighting something else. Any help will be very much appreciated.
Side note- this last patch did apply cleanly for me and caused no issues. It, however, didn't seem to fix what I was hoping it would.
Comment #12
jrao CreditAttribution: jrao commentedSome more ideas on debugging the issue:
1. Use watchdog to log messages, this way you can keep it running
2. It may be more useful to debug the method entityreference_feeds_set_target, since that's where the value is imported, it should be fairly easy to find out what's happening.
Comment #13
GaëlGOr you may try the Devel module and its great function dpm().
Comment #14
MTecknology CreditAttribution: MTecknology commentedGOT IT!!!
I just realized how this patch fits in and this patch 100% fixed my issue! I LOVE YOU ALL!!!!!!!
Yes, I'm that excited. My brain bled out of my ears twice (twice baked brain) trying to figure this out. It's so bloody simple now! Thanks!
Comment #15
hoangdk80 CreditAttribution: hoangdk80 commentedYou rule !
We were stuck at this very point yesterday.
Then we found the patch.
We installed it.
And EVERYTHING WORKS PERFECTLY NOW !
Nice work !
Comment #16
derekw CreditAttribution: derekw commentedUsed #5 on -dev. Works great here, tested with node title.
Comment #17
arski CreditAttribution: arski commentedgreat stuff! works with feeds_tamper for multiple values as well. nice!
Comment #18
scottrigby#10 is working great - mapping to `label` for Title - on 30,000+ rows
Comment #19
czigor CreditAttribution: czigor commentedI have a commerce product entity with an entity reference to a Person node.
I have a csv file with the products. The Person column has the title of the Person node.
I applied patch #10 and the four mapper target options appeared all right in the select list. My mapper looks like this:
SOURCE TARGET
Person Person (Entitiy reference by Entity label)
However, neither the Entity label nor the Feeds GUID mapper target does not create the reference between the product and the Person. Importing the Person nodes separately beforehand does not help. Encapsulating field texts in the csv file in " does not help.
The ui seems trivial, and still I can't get it working. Am I missing something here?
Comment #20
resullivan CreditAttribution: resullivan commentedNot sure if I am doing something wrong, but I have tried both patches using the patch manager module. With both it is saying that all 3 hunks failed. I have never patched anything before.
Comment #21
scottrigby@resullivan you'll want to read http://drupal.org/patch/apply to help you get started
It applied cleanly for me
Comment #22
scottrigby@czigor - If your CSV column is called "Person", then your Entity reference Feeds mapping - SOURCE: 'Person', TARGET: 'Person (Entitiy reference by Entity label)' - sounds correct. Yes you will want to import your Person nodes first, so they can be referenced. That is exactly what I tested, and everything worked perfectly.
A few things to check:
Can anyone else try this? I haven't checked with Commerce product entities, but I don't see why they wouldn't work while other entities seem to.
Comment #23
czigor CreditAttribution: czigor commented@scottrigby: Thanks for answering!
So I should not use any other mappers with targets like 'GUID' or 'Entity reference by Feeds GUID'?
Comment #24
czigor CreditAttribution: czigor commentedIt's working now, but I don't know what I did. I was messing around with my fields but I can't see how that should be related to the import.
So #10 is working for me.
Comment #25
Andrew_Mallis CreditAttribution: Andrew_Mallis commentedStellar. #10 works, but was not as intuitive as it could be.
I selected the (Entity reference by Feeds GUID) option. This connects to the previous feed import's GUID, to which is associated an entity_id.
I didn't understand by the label that's what would happen, but reading the code explained it.
Comment #26
resullivan CreditAttribution: resullivan commentedApparently I had a module called entity and entityreference. I fixed the module name and it worked great. Also tested this out and it worked like a charm. Was able to upload records based on entity ID.
Comment #27
anavarreWhat's the status on this? Are solutions provided by #5 or #10 acceptable as is or is there more work to do? Also, let's say one of this patches make it through, couldn't this be a good start already and we can file other issues for what still needs to be done?
Comment #28
KarenS CreditAttribution: KarenS commentedConfirming that #10 works great to import nodes that have references and find the correct reference on the destination, so long as the referenced node exists.
Comment #29
KarenS CreditAttribution: KarenS commentedActually, I added one line to the patch in #10 to add a log message if this process is unable to find the right reference. That way you have a clue when viewing the log that you lost some values.
Comment #30
KarenS CreditAttribution: KarenS commentedOops, one more small change. I am getting error messages on the original patch because we don't have anything in $mapping or $input_format. It looks like we're passing too many arguments to entityreference_feeds_set_target(). I removed the last two, which we weren't even using. This makes this function match the pattern used by feeds for field_feeds_set_target_numeric(), etc.
Setting it back to needs review because I made changes to it, but it works for me.
Comment #31
codevoice CreditAttribution: codevoice commented#9: entityreference-feeds_integration-1616680-9.patch queued for re-testing.
Comment #32
scottrigbyRe #30
We should also strip the documented params removed by this patch =)
Comment #33
franzComment #34
star-szrFor the record, I tested #33 and it worked great, no errors.
Comment #35
7wonders CreditAttribution: 7wonders commentedPatch in #33 worked perfect for me using entity ID and entity label.
Comment #36
scottrigbyBack to RTBC =)
Comment #37
7wonders CreditAttribution: 7wonders commentedI have to take back my comment in #35 :(
After updating entity module to latest dev earlier its now only the entity ID's that are working and not entity labels. At least I think its that!
Getting this message in feeds log:
'No existing entity found for entity 0 entityreference to source entity MY_ENTITY_LABEL'
Comment #38
7wonders CreditAttribution: 7wonders commentedI have to take back again my comment. It does work its just there is a strange behaviour in feeds where if you do the import on submission it runs as admin (if your logged in as admin) but then background processes follow the imported nodes permissions. As I was using some field permissions modules the entityrefs were importing on submission but not on background (with the above error). A bit of a feeds gotcha.
Comment #39
tinker CreditAttribution: tinker commented#33 Works great using entity ID and with feeds tamper for multiple values! I think this is an excellent addition of new features to to the module. Thanks for all the hard work.
Comment #40
7wonders CreditAttribution: 7wonders commentedOk, sorry for this, I feel like im spamming this issue but I think im running into a problem that IS related to this. I am using the feeds_fetcher_directory and feeds_jsonpath_parser modules. Quite simply, when using the entity label functionality it works fine when I add the directory and it does the initial import. It also works fine if I go in and click cron to run. But if I leave it to go on background process everything except the entity label based field import works fine and thats when I get the message:
'No existing entity found for entity 0 entityreference to source entity MY_ENTITY_LABEL'
I have tried different permissions, different authors and so on and cannot pinpoint the problem. It would be good if someone who is using a different parser/fetcher can see if the problem is the same for them also.
Comment #41
argo27 CreditAttribution: argo27 commentedI ahve attempted to apply the patch in #33 but when I apply it I got this error?
error: patch failed: entityreference.feeds.inc:16
error: entityrefreence.feeds.inc: patch does not apply
applying it to entity reference 7.x-1.0-rc5
Need your help.. Iv ebeen doing this for 2 dyas now :(
Comment #42
argo27 CreditAttribution: argo27 commentedWhy do i get an error on this line of code of the patch.
Warning: Missing argument 4 for entityreference_feeds_processor_targets_alter() in entityreference_feeds_processor_targets_alter() (line 13 of C:\Users\Andlen\SSA WEB\drupal\ssaweb3\ssatempsite\sites\all\modules\entityreference\entityreference.feeds.inc).
function entityreference_feeds_processor_targets_alter(&$targets, $entity_type,
$info = field_info_field($name);
if ($info['type'] == 'entityreference') {
// We don't use ":guid" in key, not to break existing configurations.
$targets[$name] = array(
'name' => check_plain($instance['label'] . t(' (Entity reference by Feeds GUID)')),
'callback' => 'entityreference_feeds_set_target',
'description' => t('The field instance @label of @id matched by Feeds GUID.', array(
'@label' => $instance['label'],
'@id' => $name,
)),
);
Thanks hope someone can help me
Comment #43
tinker CreditAttribution: tinker commented"$name => $instance" is not valid in the function arguments. Maybe you have badly applied a patch? The functions should be
Comment #44
argo27 CreditAttribution: argo27 commentedThats right the code has an error at that part but how come this is part of the patch, pls. take a look at the patch codeComment #45
klase CreditAttribution: klase commentedI have successfully applied #33 and imported both single and (via tamper explode) multiple values using Node ID. Used current release (7.x-1.0-rc5). Works great and a total of over 10,000 records were imported and referenced to existing content. Thanks for a job well done and I look forward to seeing this in the next release. Please include it soon as I think current (GUID) solution merely is a special case of what most users need!
Comment #46
retiredpro CreditAttribution: retiredpro commentedThanks. Patch #33 along with tamper explode module worked for me as well. I was able to import multiple User IDs into an entity reference (user) field.
Comment #47
senortim CreditAttribution: senortim commentedI second @klase's sentiment in #45 to release this soon! As someone new enough to Drupal that I don't know how to apply a patch, this could save lots of time back-pasting GUIDs into my CSV files. (Fortunately for me, I'm good with spreadsheet lookup formulas.)
Comment #48
bradjones1This patch needs an update due to a a commit from Oct. 28 that created an index on bundle type in the array returned by
getReferencableEntities()
. See (#1821060: Return values from getReferencableEntities() keyed by bundle)Here's my change to the patch in #33, with an updated complete patch attached.
By best practice I am marking this back to "needs review," but given this quick fix and the fact many others have asked for RTBC, I'd advocate for this getting into -dev sooner rather than later.
It is worth noting that I believe this patch will match on an entity label of any entity of the target type, regardless of bundle. Not sure if that's enough of a reason to postpone a commit, since I think this is super-helpful functionality.
Comment #49
Jibus CreditAttribution: Jibus commentedTested patch #48 againt the 7.x-1.x-dev version (2012-Oct-29).
Created 140 nodes: worked like a charm
Thanks !
Comment #50
drett CreditAttribution: drett commented#10: entityreference-feeds_integration-1616680-10.patch queued for re-testing.
Comment #51
kclarkson CreditAttribution: kclarkson commentedI too am confirming that the patch in #48 was applied cleanly and allowed me to import 500 event nodes that have entity references for Organic Groups.
Hint: If you are using entityreference names in your spreadsheet be sure to use the entityreference "label" option in the Feeds Mapping section.
Thanks @bradjones1 for the Patch !!!
Comment #52
kclarkson CreditAttribution: kclarkson commentedComment #53
cdmo CreditAttribution: cdmo commentedI've applied #48, and it isn't working. I'm using XPath XML parser as my parser and I'm mapping "xpathparser:13" to "Space (Entity reference by Entity label)" and I'm getting a slew of notices like "No existing entity found for entity 2431 entityreference to source entity Auditorium." The name is spelled exactly the same in the feed import as it is in the Space node Title field. I know I'm not making an error with XPath because I'm importing the same data into a text field just fine. Thoughts?
Quick update, after fiddling a bit now I'm getting this notice over and over:
"No existing entity found for entity 0 entityreference to source entity Auditorium"
Comment #54
cdmo CreditAttribution: cdmo commentedUPDATE: It is now working just fine, updated 260 nodes based on entity reference by label method. I had to change the feed parser to the actual xpath expression instead of passing in a variable from a parser higher up in the chain (if that makes sense). Thank you to the developers who made this patch!
Comment #55
slashrsm CreditAttribution: slashrsm commentedI can confirm that patch works as expected. I am also using it with XPath XML parser.
Looking forward to have this patch commited!
Comment #56
Désiré CreditAttribution: Désiré commentedI'we tested the patch with entity label (node title), and it works, and the code seems correct.
Comment #57
Désiré CreditAttribution: Désiré commentedreset() (http://php.net/manual/en/function.reset.php) requires a reference as argument, so only variable should be passed to it. This solution causes PHP notices.
I've fixed this in the attached patch.
Comment #58
Désiré CreditAttribution: Désiré commentedBTW this patch needs tests.
Comment #59
lolcode CreditAttribution: lolcode commentedI applied #57 cleanly and imported 853 nodes using the entity_id mapping. No issues and scan of the code looked fine.
Comment #60
slashrsm CreditAttribution: slashrsm commentedPatch needs tests before beeing RTBC.
Comment #61
slashrsm CreditAttribution: slashrsm commentedStarted to work on tests today...
Comment #62
slashrsm CreditAttribution: slashrsm commentedTests added.
Comment #64
slashrsm CreditAttribution: slashrsm commentedLet's try this way....
Comment #66
slashrsm CreditAttribution: slashrsm commentedThe problem here is that test bot does not know that feeds needs to be included in test env. Test bot calculates dependencies from time to time (once a day I believe), but it only searches project's git. That means that feeds will not be included until this patch is committed.
Attached patch just checks if feeds exist and skips tests if not.
Comment #67
tripper54 CreditAttribution: tripper54 commented#66 works a treat for me - importing referenced custom entities based on label.
Thanks so much for everyone's work on this - you all rock.
Comment #68
kclarkson CreditAttribution: kclarkson commentedApplied patch and Everything imported perfectly from a CSV file. Thanks again to the community !!!
PS -when importing content don't forget to create some sort of id column and map it to GUID. I forgot to do this so when i selected the delete items on the import feeds page it deleted other content that was imported using feeds :)
Comment #69
lquessenberry CreditAttribution: lquessenberry commentedI may be completely lost here, but where do the options show up when you are mapping this? I am looking to feed labels into an Entity Reference field, but I am not sure it is working at all, and I am not real sure on how to configure this. I applied the patch in post 48, but I have no idea how to set up my feed. I have a feed that is parsing from a PgSQL dump. I can see my columns from SQL dump just fine. I also see the field that I wish to map to just fine, but I have no way of knowing what is going to be assigned into the field. I thought I would be able to assign it to map a label to it. Can someone help me understand this a little better?
Comment #70
lquessenberry CreditAttribution: lquessenberry commentedThis is a problem I am having. My mapper options are not showing up at all in the select list.
Comment #71
kclarkson CreditAttribution: kclarkson commented@lquessenberry
I have never used feeds to import from anything besides a CSV but Here are some things to try:
- delete the feeds module completely and grab the dev version from GIT
- apply the most recent patch which would be #66
- go to your feeds mapper and make sure that the right Node type is selected. (This is what got me. It appears that after apply the patch that the node mapper content type is reset. Be sure to save the feed after each step)
- now go to the mapper fields and you should now be able to see in your dropdown what fields to map your data to.
Ps - don't forget to map a unique number to the GUID
Comment #72
juliangb CreditAttribution: juliangb commentedTo confirm that I'm using the patch from #66, which fixes the problem without any issues.
What other testing is required for this to be committed?
Comment #73
AaronBauman#66 works perfect for me.
RTBC++
Comment #74
appaulmac CreditAttribution: appaulmac commented#66 worked a treat for me, using Entity Reference label, just what I needed! Thanks! :)
Comment #75
Jeffrey C. CreditAttribution: Jeffrey C. commentedPlease commit.
Comment #76
kclarkson CreditAttribution: kclarkson commentedAgreed ! Please commit. Nothing more frustrating the starting another project and having to remember to patch it :)
Comment #77
Jeffrey C. CreditAttribution: Jeffrey C. commentedChanging the priority to see if we can get more attention.
Comment #78
amitaibuSeems #66 adds a new ER_feeds module. Why not create a new contrib module, and move the integration there?
Comment #79
everpat CreditAttribution: everpat commented#66 worked for me. Thanks.
Comment #80
slashrsm CreditAttribution: slashrsm commented@Amitaibu #66 does not add any new module. It just improves current feeds integration and makes it actually usable. I don't see any point in moving feeds integration to a new module, if ER already provides it.
Comment #81
pedrop CreditAttribution: pedrop commented#66 worked for me. Thanks!
Comment #82
__cj CreditAttribution: __cj commented#66 worked for me. Thank you!
Comment #83
slashrsm CreditAttribution: slashrsm commentedWould be nice to have this committed. It has been RTBC for more than a month and people keep posting positive feedback...
Comment #84
checker CreditAttribution: checker commentedPerhaps the issue tag "Needs tests" should removed because the patch includes tests?
Comment #85
Jeffrey C. CreditAttribution: Jeffrey C. commentedRemoved.
Comment #86
gaele CreditAttribution: gaele commentedWell then
Comment #87
Rob230 CreditAttribution: Rob230 commented#66 is wonderful. Works perfectly. I imported from CSV without even having to change my data. It just used the Node title (entity label).
Comment #88
kclarkson CreditAttribution: kclarkson commented20 + rtbc so is there a reason this isn't being committed ?
Comment #89
slashrsm CreditAttribution: slashrsm commentedAs already mentioned before, test are included in that patch.
Comment #90
amitaibuMinor code style issues, before commit:
false => FALSE
Instead of doing this, just add
'dependencies' => array('feeds')
to the getInfo() function.Comment #91
slashrsm CreditAttribution: slashrsm commentedI can so that, but I think that tests will fail until this gets committed and testbot rebuilds dependency information (which is done every 24h).
Comment #92
slashrsm CreditAttribution: slashrsm commentedComment #93
squarecandy CreditAttribution: squarecandy commented#91 works for me.
the .info part of the patch failed against 7.x-1.0, but I'm probably supposed to be using dev...
Anyways - it works!
I'm using it successfully along with feeds tamper to get multiple NIDs from a single CSV field into a multivalue entity reference field.
Comment #94
presleyd CreditAttribution: presleyd commentedThis gives entity reference the same functionality node reference (References module) had. This is a big help in adoption of entity reference for me!
Tested today using GUID, label and entity ID and all worked fine in my use case.
Comment #95
amitaibuCommitted, Thanks!
Comment #96
bradallenfisher CreditAttribution: bradallenfisher commentedUsing 7.x-1.0+4-dev i was able to import three reference fields from a csv file using Entity label. :) Thanks this rocks.
Comment #97
kingswoodute CreditAttribution: kingswoodute commentedSubscribe,
If someone could post a simple how-to for importing nodes with entity reference fields that reference other nodes, it would be hugely appreciated. If you could include which version of modules you need to use that would be even better.
Thanks for all the work on this feature.
Comment #99
oryx CreditAttribution: oryx commentedI'd be one more guy extremely interested by such a tutorial ! ;)
Comment #100
Rob230 CreditAttribution: Rob230 commentedThere is not really much to explain. It's mostly self-explanatory just by going through the options.
There are some instructions for making feeds importers here: http://drupal.org/node/638804
The only part that's different is you will have more destinations to choose from in the mapping stage so that you can choose if you want to map onto the ID or the title of an entity reference field.
As for versions just use the latest releases, but for Entity reference you will need the dev version that has this patch in it.
It might be easier to start from the example node importer that comes with the Feeds Import module of Feeds.
Comment #101
slefevre1 CreditAttribution: slefevre1 commentedI've checked out commit 06089a92ee8b269f153064ad3090ce6b464f38aa from the dev repo which has this patch. When I try to run the FeedsMapperFieldTestCase using `drush test-run`, I get a big explosion of errors:
Feeds integration (field mapper) 65 passes, 88 fails, 33 exceptions, and 23 debug messages [error]
Test FeedsMapperFieldTestCase->setUp() failed: User n39zpEE9 successfully logged in. [error]
Test FeedsMapperFieldTestCase->test() failed: User n39zpEE9 successfully logged in. [error]
Test FeedsMapperFieldTestCase->test() failed: Found Entity reference field field_er_id. [error]
Test FeedsMapperFieldTestCase->test() failed: Found Entity reference field field_er_label. [error]
...
continuing on for 123 lines. The three other test cases (EntityReferenceAdminTestCase, EntityReferenceHandlersTestCase, and EntityReferenceTaxonomyTestCase), run without failures or warnings.
A lot of the test results seem to be successes but are marked as errors, such as these:
"Test FeedsMapperFieldTestCase->test() failed: Found Entity reference field field_er_id. [error]"
"Test FeedsMapperFieldTestCase->test() failed: Child node: Import by entity ID OK. [error]"
"Test FeedsMapperFieldTestCase->addMappings() failed: Found the Add button [error]"
While others seem to be actual failures:
"Test FeedsMapperFieldTestCase->getCurrentMappings() failed: array_key_exists() expects parameter 2 to be array, null given [error]
Test FeedsMapperFieldTestCase->addMappings() failed: Failed to set field source to parent_nid [error]
Test FeedsMapperFieldTestCase->addMappings() failed: Failed to set field target to field_er_id:etid [error]"
Am I doing something wrong? Is anyone else getting this behavior?
Comment #102
slefevre1 CreditAttribution: slefevre1 commentedWell, it looks like this test fails because of being run in drush-- I just tested this in the UI and it went off without a hitch.
Comment #103
klase CreditAttribution: klase commentedJust a quick thank you to everyone who was working on this!
Comment #104
TBarina CreditAttribution: TBarina commentedThanks a lot.
Patch in #91 works great...
Comment #105
Markoz CreditAttribution: Markoz commentedI think this patch doesn't handle not unique GUID in feeds_item table when mapper "Entity reference by Feeds GUID" is selected.
Guid is not unique in feeds_item table, it comes from the external source.
Waiting for this: https://drupal.org/node/661606
Comment #106
held69 CreditAttribution: held69 commentedSorry for opening this up again. But reading above issues for hours i am left with some questions whether my usecase should be working.
I am trying to import a rss feed into groupcontent-nodes.
I have set up a feedimporter adding a groupreference field and groupvisibility field.
I have added these same fields to my feed-item as well.
My problem is that i dont seem to be able to map these fields.
They are listed under targets however not under ' sources'.
How can i map these fields?
Thanks
Feeds 7.x-2.x-dev
Entityreference 7.x-1.x-dev
Comment #107
k_zoltan CreditAttribution: k_zoltan commented@held69 This issue is intended to solve another problem that has been commited see comment-#95 For questions like this open another ticket, and mark as support request.
Marking as closed.
Let it rest in peace. :)