Hi,

I m trying to import data to nodereference cck fields.
I guess I have to write a mapper for this, even I don't have enough experience on drupal or php.
Is this even possible or I m wasting my time?

Thank you.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alex_b’s picture

Title: nodereference mapper » Mapper for nodereference field
Version: 6.x-1.0-alpha12 » 6.x-1.x-dev

Possible.

Look at feeds.api.php in the CVS head version of this module, look for the mapping API hooks. Look at the existing implementations in feeds' mappers/ directory for an example on how it's done. the content.inc mapper may be very close to what you are looking for.

Note that for getting a nodereference mapper into the Feeds project, tests will be required. Even if you don't want to go the whole nine miles, a solid patch here will be much appreciated. Alternatively, you can always implement a mapper in a separate module.

katsikas’s picture

FileSize
1.79 KB

Thank you alex_b, for the guidelines.

I think I made some progress here.

Maybe you 'd like see the attachment.
It seems to work but I don't know if I am missing something.
You opinion will be valuable for me.

alex_b’s picture

Status: Active » Needs work
FileSize
2.44 KB

This is looking very good.

Here is what's missing to get this mapper committed:

- Roll a patch (I just rolled one, you can roll yourself by using cvsdo for adding a new file)
- Make sure the Coding standards are followed. Most importantly, use 2 spaces for indentation but not tabs.
- Clean up oddities like the $i incremental variable in nodereference_feeds_set_target().
- Comment all functions
- Write tests.

Anonymous’s picture

Does this patch work for one-to-many node reference relationships?

For example I have a node that has a CCK node reference field with 'Number of values: Unlimited'.

My Custom type is a city, and the node reference is used to list users who have visited that city. So each city has a multiple node references to users.

ybabel’s picture

Tested a little. It worked for me, in a 1-n relation !

Anonymous’s picture

I'd like to use this for importing a CSV file.

What's the syntax for writing 1-many relationships values in a CSV file?

For example if I have a node reference with unlimited values, and one of my nodes should reference node #1 and node #2, how do I write that out to CSV?

How should the (1,2) below be written?

guid,title,noderefence
1,test node,(1,2)
tauno’s picture

To accommodate field specific settings, I tweaked this to use _nodereference_potential_references() to find the nid of the referenced node instead of the custom function. Also got rid of the tabs and the $i incremental variable.

If this is a desirable direction to take the patch, tests are still needed.

brst t’s picture

Oh, I somehow overlooked the patch in #7.

The one in #3 didn't work for me. Should note that I'm not importing a feed, I'm using it to import nodes from a csv. Update is set to never, no url, no feed beyond the big stack of csv's.

The node reference field showed up, but the importer stopped working.

I'll give #7 a go.

webwriter’s picture

Works for me. :-) Importing CSV file.

alex_b’s picture

At a quick glance, #7 looks good. Tests needed to get it committed.

lunk rat’s picture

Um, silly question, but do I apply this to feeds.module?

And if I put the node title in the CSV does it map that to the node reference field?

Thanks for the patch! Great timing in my case.

linko

Funkymoses’s picture

tauno’s picture

Yes, it matches on the node title.

Funkymoses’s picture

This is working pretty well for me but my node references are optional and when I pass in a blank field (specifically, it's an empty string as part of a csv file) it's grabbing the first available node of the type it's allowed to match. Problem on my end or something the patch doesn't account for?

Funkymoses’s picture

Nevermind: problem on my end.

webwriter’s picture

*Edited. Nevermind, my bad! Still works fine.

alex_b’s picture

Issue tags: +Needs tests

+ Needs tests

arski’s picture

charming! can't wait to have it in a release ;)

Exploratus’s picture

subscribe..

infojunkie’s picture

I encountered the following related issue while using this mapper:
* Create referenced nodes
* Run import
* Delete referenced nodes
* Re-create referenced nodes
* Run import again
=> The importer reports that no items need updating, which leaves the node references dangling.

Any idea on how to detect the change during import?

alex_b’s picture

I'm pretty sure that's happening because none of the items is actually changing between imports - see #836090: Include mappings configuration in hash. If I'm not mistaken here, UPDATE feeds_node_item SET hash = ''; should make your items re-import.

infojunkie’s picture

@alex_b: thanks for the hint on clearing the hashes. But since this scenario causes data inconsistency, I guess it should be part of the tests for this mapper.

iaminawe’s picture

When I try and access the Mapper page after applying the path from #7 I get the following error

Fatal error: Call to undefined function feeds_get_form() in /Users/***/Sites/***/sites/all/modules/feeds/feeds_ui/feeds_ui.admin.inc on line 340

Has anyone else encountered this, any suggestions to help test if further.

Thanks
Gregg

parasox’s picture

I updated to Beta3 of feeds, and the patch applied fine.

I then added a node reference field to my "feed item" content type, but I don't see it show up in my Target list?

I flushed caches, ran update, all that..

Am I missing something about how this works?

arski’s picture

tested - works like a charm for me

arski’s picture

On second thought - is there any chance of matching nodes by some field other than the title? Assume for example I have a content type with some custom id field and my feed also has that id field somewhere - would it be possible to match those without having it in the title of the nodes?

Thanks!

Hanno’s picture

Makes sense if you can use a view to accomplish that. As in cck nodereference field settings.

iaminawe’s picture

I updated feeds to Beta 3, applied the patch and get the same issue as Parasox in #24
Caches are cleared and update has been run.

Cannot see my nodereference field in the Target dropdown for in the mapping tab.

I have been using the xml parser for feeds but try with CSV too and cannot see any noderef field.

Am I also missing something about how this is supposed to work?

Thanks
Gregg

parasox’s picture

@arski: I agree, as a workaround to not having the node reference mapper, I've been trying to figure out how to display a view of my feeds on a node type that isn't a Feed, but no luck yet.

Like I have a Site node, Feed node, and its Feed Items.. Now I want to display those Feed Items on a Site node. I haven't been able to figure out the view argument to do so, using a saved field value that could fill in the bits I need.

Following the tutorial I was able to get a view displaying Feed Items on the Feeds page, taking the bits it needs from the URL. But without that bit in the URL on a Site node, it doesn't work.

alex_b was kind enough to tell me to write an argument handler, but that's over my head. I've tried to find a PHP snippet to use as an argument, but no luck.

So if anyone knows how to use a field as a views argument, please enlighten. :)

arski’s picture

I'm a little confused.. but actually I think that's just how node reference field works.. You know that you can use a view to show available nodes for selection that you want to reference, and you can list specific fields out of those nodes. But even then, the list items will look like "Field-1-value, Field-2-value; Title: title_here".. So I assume node reference still uses the title as the value for each node and all the mapping works through the title.. argh.. If that is the case then really all we can do is extend the node reference mapper beyond the nodereference field by adding more options, like a field selection to use for referencing.

I hope this makes any sense... it's late on a Sunday here :x

Thanks for keeping this up :)

Hanno’s picture

@arski: you can have a look at the function _nodereference_potential_references_views in cck, nodereference.module.
It might be useful. It asks the advanced view of the nodereference field for all referencable nodes. This view might include one or more fields of a node to display. It uses the values of the fields with '-' as seperator as the 'title' for matching. If we can implement that in this mapper, you will have a universal method to use any data for matching.

edited: you were right about this limitation of cck node reference, it's only matched on title, see my later comment

Renee S’s picture

Tested, this works for me. THANKS :)

locomo’s picture

subscribe

alex_b’s picture

Today I saw on another issue that there is sometimes confusion around what "Needs tests" means. It means that a patch needs Simple tests in order to get committed to Feeds.

yellek’s picture

I deleted my current feeds module, unzipped beta 4 and then applied the patch in #7. When I go to map my nodereference fields I don't see them in the list. Any suggestions as to what I am doing wrong?

yellek’s picture

Found the issue, when I patched I did it from the module directory which meant that nodereference.inc ended up in sites/all/modules/feeds/contributions/modules/feeds/mappers. When I moved it to sites/all/modules/feeds/mappers all was well

Anonymous’s picture

I'm working with Feeds 6.x-1.0-beta4 and still experiencing the problem: When I go to map my nodereference fields I don't see them in the list. Should we expect this fix to be bundled in upcoming beta5 ? Please say it is so, Joe! ;-)

Hanno’s picture

@arski: Did some testing and you are totally right that mapping is done by title only. Drupal cck node reference matches on titles only, its hard coded, and there seems no plans to change this behavior in D6. See #375690: Nodereference autocomplete doesn't match all of view's fields.

One possible workaround is to use the super_nodereference.module that tries to overcome the limitation and matches on fields defined in a view. A mapper could use the function _super_nodereference_potential_references

arski’s picture

@Hanno: Thanks for that - I'll take a look at that module.. really ridiculous though, why don't they make it more dynamic...

PS. Just looked into the CVS application of that module and it's silly - they say you are only allowed to offer core extensions by providing patches and then on the other hand they refuse to accept these...

Anyway I don't really want to use and/or support an externally hosted module, so I guess there's nothing more we can do "officially" about this :/

Hanno’s picture

@arski I was surprised too, node_reference is so broadly used, but patches are not commited to change this. If we don't want to rely on an external hosted module, we probably can re-use the relevant code in function _super_nodereference_potential_references_views from super_nodereference. If we use the first result and return this nid, we are done.
Would be possibly outside the scope of the standard node_recerence.inc as proposed in this thread.

boogsbobo’s picture

subscribe

pdrake’s picture

The is a little bit of a bug in this mapper in that blank values seem to inexplicably map to the first node of the appropriate type. Adding a check that the value is not empty on line 58 as shown below resolves this:

    if (!is_array($v) && !is_object($v) && !empty($v)) {
webwriter’s picture

Oh, thanks!! This has been driving me crazy. I should have posted it but I thought I had done something wrong- I've been having to delete tons of node references to a single node where they should have been empty.

dbritton’s picture

I was able to implement a mapper for super_nodereference fields by copying the file feeds\mappers\nodereference.inc to a new file feeds\mappers\super_nodereference and replacing all occurrences of "nodereference" by "super_nodereference". Now when a super_nodereference field is defined for a content type, it shows up in the list of possible targets for Node processor Mapping (admin/build/feeds/edit/{feed importer}/mapping).

This means than an import file can make reference to nodes of a given content type by using any arbitrary piece of data that identifies a particular node of that type (provided that piece of data is contained in one of that content type's fields).

andrewlevine’s picture

Nodereference can match on specific IDs with the $ids parameter of _nodereference_potential_references. Is there any consensus on whether we want to support a source that supplies NIDs, titles, or both?

webwriter’s picture

Both would be ideal.

stephenj’s picture

subscribe

Renee S’s picture

#42 worked for me as well. Possible to get it committed? :)

Anonymous’s picture

Version: 6.x-1.x-dev » 6.x-1.0-beta4

I am currently using Feeds 6.x-1.0-beta4 to import CSV to create nodes. In the "Mapping for Node processor" pane, the targets in the "Legend" area suggest that ALL of my CCK fields are node REFERENCES but in fact they are NOT. Some of them are TEXT, others are INTEGER, and yet they are all viewed as references!!! And when I attempt to import, the text/integer values are not imported. Will upgrading to 6.x-1.0-beta5 (2010-Sep-10) of Feeds solve my problem ? Patching, as y'all describe above, is beyond my skill-level for the time being. Help!

andrewlevine’s picture

Version: 6.x-1.0-beta4 » 6.x-1.x-dev

moving issue back to dev

Anonymous’s picture

Version: 6.x-1.x-dev » 6.x-1.0-beta5

I still have the following problem, even after upgrading to Feeds 6.x-1.0-beta5 to import CSV to create nodes. In the "Mapping for Node processor" pane, the targets in the "Legend" area suggest that ALL of my CCK fields are node REFERENCES but in fact they are NOT. Some of them are TEXT, others are INTEGER, and yet they are all viewed as references!!! And when I attempt to import, the text/integer values are not imported. Patching, as y'all describe above, is beyond my skill-level for the time being. Help!

alex_b’s picture

Version: 6.x-1.0-beta5 » 6.x-1.x-dev

This patch needs to be developed against DRUPAL-6--1 branch of Feeds. Learning how to patch is beyond the scope of the Feeds issue queue. Please refer to http://drupal.org/patch, forums, Google to learn how to use patches. http://drupal.org/patch

Aron Novak’s picture

I think such a nodereference mapper should implement feeds_after_import, cause i see the following problem:
If the imported nodes contain nodereferences to each other, it will fail randomly (depending of the order of the items in the feed), as the referenced node is not created yet.
I assume the more robust approach would be to store the relationships using the unique feed item ID temporarily at mapping stage and create the actual nodereferences in hook_feeds_after_import().
What do you think?

joshuajabbour’s picture

Attached patch is a reroll of #7 incorporating #42. Working on tests...

joshuajabbour’s picture

And AFAICT, it doesn't support multiple nodereferences (it works with fields that support multiple nodereferences, but you can't import multiple -- at least with the QueryPath parser). I'll see what I can do...

Also, @aron (#53) I agree that might be a good way to handle any race conditions...

twistor’s picture

@Aron Novak, I think you're talking about two different use cases. The nodereference field would be used to reference existing fields. There could be another kind of reference, a guid reference, that allows for imported nodes to reference each other. This second one could be converted to nodereferences after import.

alex_b’s picture

#53 / #56: yes, the use case of referencing nodes that are being imported, too is a little more complex. We'd have to establish a UUID for each mapping set (that could be just the feeds_node_item.guid field or it could be configurable) and the imported items would have to reference each other by that very UUID. Once an import is done, a routine invoked on hook_feeds_after_import() can resolve these UUID references into node references by nid.

I think such functionality is out of scope for a simple node reference mapper.

joshuajabbour’s picture

OK, attached patch correctly handles multiple value fields *if* the associated parser does (i.e. passes an array of values as opposed to a single value).

joshuajabbour’s picture

Changes:

* Supports multiple value nodereference fields (if the parser does)
* Now supports matching by nid
* Multiple mappers per import won't overwrite each other
* Validates values and ensures nodereferences are only added once per node

primozsusa’s picture

Hi, this works for me... I have one question... Would it be possible to set feeds importer on the parent content type, so that when parent node would be created it would import child nodes from some source... ok, its possible... But, would it be possible to, automatically match parent nodeid to nodereference field in newly created nodes so it would not have to match it from outside source. For example Gallery to Photo content type nodes... How would it be logical to set, define this matching setting?
Nice day
Primoz

twistor’s picture

Based on #59. Simplified a bit. Get to the tests soon.

@joshuajabbour, Is there a need for feeds_nodereference_map that I'm missing?

joshuajabbour’s picture

@twistor: yes, otherwise a node can get added twice to a field. I guess this could be desired behavior, but I'm not sure how or why...

also, this should probably be is_numeric in case it's a string:

if (is_int($v) && $v > 0) {

twistor’s picture

For some reason I thought array_unique would handle complex arrays. Wouldn't checking if the reference exists be enough though?

The reason I avoid is_numeric is that it will pass for floats and exponents.

Ooxo’s picture

Hi , I was wondering if the patch #63 handles the multiple nodereference (as I have fields using unlimited) and if yes, what separator should I use in my csv, I have tried (from patch #53) "," ":" ";" but it always failed, leaving a blank title where the csv row contains multiple nodereference ?

twistor’s picture

@Ooxo, multi-valued fields for csv files currently isn't supported in Feeds. See #759966: CSV: Support one to many relationships. This can be done with Feeds Tamper using the explode plugin on the field.

twistor’s picture

Now with tests!

Ooxo’s picture

@twistor, Thanks so much for the information it worked like a charm, brilliant

joshuajabbour’s picture

Component: Miscellaneous » Code
Status: Needs work » Needs review
FileSize
9.64 KB

OK, couple of comments and a rerolled patch...

* fixed using t() and replacements in name key of nodereference_feeds_node_processor_targets_alter
* calling content_fields() moved outside foreach loop to cache field definition
* regarding ctype_digit... I've got no real problem w/ it, though I've had a patch elsewhere (not in feeds) rejected because the drupal standard is to use is_numeric (with all its flaws). I did add a check for an empty string, as that's a pre-5.1 bug...

Bigger change: on further thought, I'm not sure we should eliminate duplicates by default. The default nodereference field allows duplicates, so we probably should too. Where this really becomes a problem is with updating already imported nodes, and this was the reason for my somewhat convoluted checks before. So with this patch, I've added duplicate and non-duplicate mappers. Still not sure if this is the best way, but I could see cases where each would be desired. I have not yet added new tests for the duplicate mappers, or updated the current tests to test for duplicates.

Setting this to "needs review", as probably someone other than twistor or I should review this...

twistor’s picture

* t() fixes were definitely needed. I think we should come up with more succinct names for each target. These are getting pretty long and we can give explanations in the description. I didn't change this last time as I'm terrible at naming things.
* I moved content_fields() so that it wouldn't get called when using nids directly. It caches the results internally in _content_type_info(). I really don't want to get too pedantic about micro optimizations though.

As far as allowing repeated references... makes me think of #860748: Config for mappers?. I don't think that supporting multiple references to the same node is necessary (although I can see use cases) since CCK's ui doesn't support it.

joshuajabbour’s picture

well, CCK's UI *does* support it, that's why I added it back in. but yes, it would be great if there was config for mappers...

lazysoundsystem’s picture

Component: Code » Miscellaneous
Status: Needs review » Needs work

Trying patch from #68 and all seems good...

joshuajabbour’s picture

@lazysoundsystem is it good? if so, was setting it back to needs work a mistake?

lazysoundsystem’s picture

Status: Needs work » Needs review

Sorry, yes, completely didn't mean to change the status. It's working great!

lazysoundsystem’s picture

Component: Miscellaneous » Code

And nor the Component. Don't know how that happened.

twistor’s picture

Status: Needs review » Needs work

Hate to do this, but the fact that the tests need to be adjusted means it needs work. However, I'll get on it.

Nephele’s picture

I'm trying to set up a Drupal 7 site, so I've started a separate issue at #988856: Feeds mapper for node_reference and user_reference fields for the Drupal 7 equivalent to this issue.

While looking through the D6 code for my D7 version, I noticed that validation was missing for references provided as nids. In particular, I was thinking of cases where the node reference field is limited to one content type -- a check is needed that the nid corresponds to the specified content type. I've provided a new patch which adds this extra validation.

Re #68: At this point, I've basically done a code review of the 'references.inc' file, providing one more set of (newbie) eyes for that part of the code. But I haven't deciphered the testing process, yet, so I can't speak to the other files.

joshuajabbour’s picture

The check added in #76 is the right call; good catch @nephele.

I tried to update/add more tests, but right now a bunch aren't passing. So I've got to figure that out, but won't have time til this weekend at the earliest...

joshuajabbour’s picture

After a bit more thought, I'm actually inclined to remove the duplicate handling code from this patch. CCK allows duplicates in nodereference fields, as do all other multi-value CCK fields (both by default, and when imported w/ feeds). So I'm not sure why nodereferences should be the exception.

My new thinking is that if you are getting duplicates, you probably need to fix your data source, or use something like Feeds Tamper. This can always be added back in (correctly) once we have mapper config settings, but for now it will simplify the code and tests (and usage) to not have the extra mappers.

tayzlor’s picture

tayzlor’s picture

From a UX perspective, the length of the drop down is quite large on admin/build/feeds/edit/node/mapping (because of the length of the description text for the node reference field). This causes the middle column to expand horizontally and pushes the right column off the side of the screen forcing a scrollbar along the bottom.
Viewing on 1024x768 resolution in Garland theme with a left sidebar.

Nephele’s picture

@78, @80: I agree that the duplicate-handling option seems unnecessary and makes email handling different from other fields. It's exacerbated by the extra length added to the description text for node reference fields -- which stand out in length even without the duplicate-handling text.

jdlind38’s picture

subscribing. Is this feature committed to the next release?

joshuajabbour’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
8.17 KB

OK, finally got around to reworking this patch:

* Removed mappers that disallow duplicates. By default, duplicate nodereferences in your import are not merged and will result in duplicate field values. If this is a problem, you might need to fix your data source, or use something like Feeds Tamper.
* Feeds that are configured to "Update existing nodes (slower than replacing them)" will overwrite all field values on each update. This is how the base CCK importer included with Feeds works, and I believe it is the way it must work. (Otherwise you'd never be able to delete field values, and each update would add duplicates.) See #996808: Update existing doesn't reset targets that have real_target set. for more info.
* Other minor code cleanup.

joshuajabbour’s picture

One more bonus feature: Now you can match a nodereference by Feeds URL or GUID fields.

twistor’s picture

Status: Needs review » Needs work

#84, Awesomesauce.

#83,
1. Agree entirely. I'll write a tamper plugin for this asap.
2. I believe you're referring to $field = isset($node->$target) ? $node->target : array();. This is necessary to support mapping to a target multiple times. See #840626: Support using same mapping target multiple times. If the field isn't getting reset before mapping then that's a bug in feeds. Will track it down, I think I ran into this yesterday actually but just modified my own code in a similar way.

We need tests for the url/guid mapping, I think. Other than that I think this is looking splendid.

joshuajabbour’s picture

Added back in the line to allow mapping target multiple times, per #85.

Depends on the bugfix in #996808: Update existing doesn't reset targets that have real_target set. in order to work correctly.

uwe_a’s picture

applied this patch, and the bugfix mentioned in the post, still i do not get node reference cck fields in the target select menu in the importer UI ! my bad ...

jeffschuler’s picture

The patch in #86 is working fairly well for me, except for one thing: whitespace in my XML feed is interfering with the nodereference lookup.

My XML import doc includes newline whitespace like the following, where parcel_num is the title of the node to be referenced.

<parcel_num>
58117061
</parcel_num>

When importing, I get an error:

' 58117061 ' does not match a valid node title for the 'field_property_ref' field.

I tried using feeds_tamper's trim feature, but ran into some other issues there. And the strange thing to me is that the whitespace is not saved when the same parcel_num source is mapped to a CCK text field, even though I don't see any trimming being done in Feeds' basic content.inc mapper, (although taxonomy and content_taxonomy mappers do use trim() before matching...)

I've re-rolled the patch from #86 with a trim() that fixes my issue. (That's the only change.)

clintthayer’s picture

Greetings everyone,

Thanks to all for providing this option. I'm in need of importing CSV data, and having the node_ref available is a huge bonus.

If I could have some knowledge sharing on how to proceed getting this up and running, that would be great. I've downloaded the .inc file from comment #2 and I'm unsure if I should be applying all, or just some of the patches? Using just the .inc file from #2 results in no errors, but also no data being saved into my CCK node-Reference field. I'm also assuming I can just provide the nid number within my CSV file. Perhaps that is not correct.

Any and all suggestions and/or tips would be welcomed,
clint

twistor’s picture

@clintthayer, You should apply the patch in #88 only. Yes, you can provide just the nid, just select the (by nid) target in the mapper.

johnv’s picture

Title: Mapper for nodereference field » Mapper for nodereference field in Drupal 6

Subscribe. Changing title to reflect difference with http://drupal.org/node/988856

Dubs’s picture

Hi there,

I ran into a small problem with the GUID lookup when I was trying to use it on many node references across a very complex dataset I am working with.

AFAIK, GUIDs should only be unique per feed import, so the same GUID may be used for different imports (i.e. different mappings). To work around this I changed the code slightly: -

    switch ($match_key) {
      case 'url':
      case 'guid':
        // Lookup node ID by Feeds unique value.
        $result = db_query("SELECT nid FROM {feeds_node_item} WHERE %s = '%s'", $match_key, $v);
        while ($row = db_fetch_array($result)) {
          $nids[] = $row['nid'];
        }
        // Ensure nid is a valid node id for this field (and use first result).
        $nodes = _nodereference_potential_references($field_info, '', NULL, $nids);
        $nid = key($nodes);
        break;

As you can see, this passes an array of potential NIDs to lookup a matching node reference.

Apologies for the lack of a proper patch file, but I don't know how to create these right now!

Thanks,

Dubs

jeffschuler’s picture

@Dubs: In that case, shouldn't we ensure that the GUID corresponds to an item imported via the same feed, rather than sourcing from all possible feeds?

Dubs’s picture

Hi Jeff,

In my case the node reference GUIDs would be from other feeds, so it's really useful to me the way it is.

I am importing car data, and the manufacturer feed import has a unique guid for each manufacturer's name.

When I import models via a different feed, there is a field on this import which in manufacturer_id (as well as a whole load of other relationships to other tables and imports). This would link via node reference to the manufacturer's GUID and other feeds GUIDS which all came from other feeds.

The created model node then has a correct node reference to the relevent manufacturer node as well as all the other nodes.

So for me, it definitely needs to search across all feeds, and the _nodereference_potential_references($field_info, '', NULL, $nids); line ensures we get a match from the correctly associated node reference.

Thanks,

Dubs

jeffschuler’s picture

Status: Needs work » Needs review
FileSize
10.06 KB

So, Dubs, _nodereference_potential_references() must be doing the work, then, of ensuring that only the appropriate nid -- of the multiple returned -- is chosen, (probably via Content Type.) Right?

Here's a patch that incorporates this, with an additional comment, and removal of $nodes:

    switch ($match_key) {
      case 'url':
      case 'guid':
        // Lookup node ID by Feeds unique value.
        $result = db_query("SELECT nid FROM {feeds_node_item} WHERE %s = '%s'", $match_key, $v);
        // Since GUID and URL are only guaranteed to be unique per feed,
        // multiple nids from different feeds may result.
        $nids = array();
        while ($row = db_fetch_array($result)) {
          $nids[] = $row['nid'];
        }
        // Ensure nid is a valid node id for this field (and use first result).
        $nid = key(_nodereference_potential_references($field_info, '', NULL, $nids));
        break;

I'm setting this to Needs Review... twistor: should it still be "Needs work" because of #85? -- or did joshuajabbour's fix in #86 (though with blocking #996808: Update existing doesn't reset targets that have real_target set.) satisfy that?

joshuajabbour’s picture

Status: Needs review » Needs work

I don't think this is there. What if your nodereference field allows multiple node types? _nodereference_potential_references() may then return more than one result, and we're only arbitrarily taking the first.

I need to think on this more, but the whole idea of GUIDs not actually being unique seems wrong to me. The are supposed to be globally unique after all...

Dubs’s picture

The first point is a good one and thankfully it doesn't apply in the case I'm working with at the moment.

In my case, we are using a data set supplied to us, and the IDs are sequential for each table in the source data, e.g. 1,2,3,4,. In each table ID is unique and would be the primary key.

I've used Feeds on many projects like this (but never used the node references before), and usually the value we map to GUID will only be unique per source table, so assuming that GUIDs will be unique for entire data set across multiple tables also seems wrong to me. Sequential IDs are very common, and I think there would be a lot of cases where GUID would not be globally unique unfortunately.

joshuajabbour’s picture

Yes, sequential IDs may be common, but they are not GUIDs. They are missing the "G" (globally). Just like Drupal IDs are not GUIDs. I believe your source data should be constructing real GUIDs.

I guess this is a call for the module maintainers, but the field is named GUID, and in my opinion, we should honor that. Unfortunately, just because something works with the way you have things setup, doesn't mean that's what this module should support.

jeffschuler’s picture

We shouldn't require Feeds users to have complete control over their data sources. Often what's used as GUID is not a true [32-char hex string] GUID, but, rather, is just guaranteed unique for that datasource.

Check out comment #1 on #761076: Importing content without guid where alex_b suggests generating a GUID by concatenating a couple fields...

GUIDs are globally, universally unique only in an ideal world. Feeds should work in a less perfect place.

johnv’s picture

To elaborate on #99: the first time I used Feeds, I couldn't find the GUID select-box, until I mapped a field to GUID...
I would expect a select-box behind every field, so I could select the fields which together form the GUID. A weight indicator would then complete this 'widget'.
But honoustly, this discussion better proceed in Attach multiple importers to one content type.

thedavidmeister’s picture

#99 makes a very good point. I can count the times that I've had control over a feed sent to me on one hand.

most of the time, a "guid" is only "global" internally in the system that i'm importing from, often that isn't even true as many systems have no need for a guid across all tables.

if i want to import content from two different sources, and set up relationships between them with a nodereference (the main objective of this thread), there's unlikely to be any guarantee that each of these sources have a field that is 100% unique.

thedavidmeister’s picture

btw, just set this up to use the Feeds GUI, #88 didn't work, but I used #95 and everything seems to be importing smoothly.

i love your work guys!

joshuajabbour’s picture

While I understand that the data may not originally have a true GUID, and you may not be able to alter the data at the source, I do believe you shouldn't pretend sequential IDs are GUIDs.

What I am saying is that you should craft a GUID for use in Feeds (as @alex_b mentions in comment 1 of #761076: Importing content without guid). Feeds Tamper is perfect for this. Personally, I'm not interested in supporting a patch to handle fake GUIDs, seeing as how no one has yet to answer the first part of my comment in #96. AFAICT, that problem is not solvable, so what you're asking is for us to commit code that will break in a fairly common scenario, without any way to make it not break.

Dubs’s picture

So with the Feeds Tamper approach and I right in saying that?

1. The name of the source table would be appended (or preprended) to the sequential ID when importing the data and that would become the GUID.
2. When importing the feed with the node reference GUID, we would also append or prepend the matching source table name (from step 1) to this column.

If so, this could work fine and would probably be the better approach.

joshuajabbour’s picture

@Dubs, yes that would be the process. It doesn't need to be the name of the source table, though that should work. Just any string that will be unique per importer. Effectively you'd be creating your own GUIDs.

thedavidmeister’s picture

yeah, actually, looking at #104 joshuajabbour's suggestion does make sense.

jeffschuler’s picture

I'm fine leaving this special case to Feeds Tamper, too, and rolling back those latest changes.

thedavidmeister: you said in #102 that the patch in #88 didn't work, (yet the one in #95 did.) How so? Was it because your situation similar to Dubs'?

thedavidmeister’s picture

i didn't look into it too deeply, sorry. I just applied the other patch and went on with my life happily.

my situation is similar to Dubs'. I have two tables, both have sequential IDs, and one references the other. Both need to be pulled into my drupal install as nodes, and a nodereference set up to mirror what is in the original table.

i could use Feeds Tamper if i had to, provided that there's some kind of guarantee that the "Tampering" will always happen before the "Mapping".

joshuajabbour’s picture

@thedavidmeister That's how the module works, and it's whole raison d'etre. So yes.

thedavidmeister’s picture

cool.

milesw’s picture

Ran into a strange issue with #95. While using GUID/URL, matching nids were being found and saved when no matches actually existed. Turns out you should not call _nodereference_potential_references() unless you provide it something to match against, otherwise you get unexpected results. I guess that's the risk you run using a module's private functions...

Very small change: line 91 in nodereference.inc
$nid = key(_nodereference_potential_references($field_info, '', NULL, $nids));
changed to
$nid = !empty($nids) ? key(_nodereference_potential_references($field_info, '', NULL, $nids)) : NULL;

Updated patch attached.

milesw’s picture

What is the reasoning for using the only first value returned by _nodereference_potential_references()? As mentioned in #96, there could certainly be more than one match. Am I missing something?

aaroninaustin’s picture

subs

jeffschuler’s picture

Status: Needs work » Needs review
FileSize
9.51 KB

@milesw: the decision (in comments #96-#110) was to drop the changes from comments #92-#95.

I've rolled your change into the patch from #88 (instead of #95.)

milesw’s picture

@jeffschuler
Thanks for moving this along, but I don't see how the two patches are ultimately different. Both versions arbitrarily choose a single value even when multiple values may be found. As joshuajabbour mentioned in #96, it's quite possible that a nodereference field references multiple content types, so there can be multiple valid matches.

The conclusion I gather from #96-#110 is that it's up to Feeds users to manage the uniqueness of identifiers used for GUID across multiple feeds. IMO the proper behavior for this mapper is to add all valid matches as opposed to arbitrarily using the first one found from the feeds_node_item table.

This patch (based on #88) allows multiple matches per value to be added as references. The test is passing, though it should probably be updated to test with multiple matches.

iamc0b’s picture

Excuse my ignorance but do each of the patch files need to be applied in turn or does the latest patch file include all patches before?

jeffschuler’s picture

@iamc0b: individual patches in an issue queue will almost always be a complete [attempted] solution to the issue, and you should only apply one at a time.

If you want to try a different patch, reverse the last one you used, first.

In this case, the patch in #115 is the current proposed solution to this issue, and the one to test out.

iamc0b’s picture

Thanks Jeff.

senzaesclusiva’s picture

Hi folk

i'm having a problem trying to patch nodereference.inc.
Done it , mappers page is totally white...seems to be a php bug .

Someone can explain me why and help me ?

tnx a lot

milesw’s picture

@senzaesclusiva: Got any more information? Did the patch apply cleanly? You say you're patching nodereference.inc, but that file doesn't normally exist, it's included in the patches here. Try applying the patch against a clean copy of feeds.

senzaesclusiva’s picture

@ milesw: maybe I'm wrong to apply the patch (I'm totally new about patch);
I am following this procedure ... it is perhaps wrong? http://drupal.org/node/60818.

I'm looking for the right patch to achieve this:
Through Extractor module i pick out from each title and body, terms of the corresponding terms in the "feeds" vocabulary.
Now, in my feeds mapper (Depending on feed-item-content-type), I've two fields. The first (text-feed) is just for my control view (setup: textfield unlimited-numbers autocomplete) and the second (noderef-feeds) (setup: nodereference unlimited-numbers autocomplete) for join it to the nids.

What happens is that when I download the feeds and the feed's title is, for example:
"Sony has bought AMD, " I have rightly in the "text-feed", the two values ​​"Sony" & "AMD" and then they would be
match to each of the two content nid.

Instead, in the "noderef-feeds" i've a multiple values , but uncorrectly , it has the same values, "Sony" & "Sony".

Do you know if this issue can be solved ?
Sorry for my bad english....

milesw’s picture

@senzaesclusiva: I'm not familiar with Extractor, but it sounds like you're getting duplicate values and not all the values you expect. This nodereference mapper does not do anything to handle duplicates, it just maps everything it finds. And when using the title field for matching, it is set to look for exact matches.

In nodereference.inc the title lookup happens here:

case 'title':
  // Validate title.
  if ((is_string($v) && $v !== '') || is_numeric($v)) {
    // Lookup potential exact matches for the value.
    $nids = array_keys(_nodereference_potential_references($field_info, $v, 'equals', array()));
  }
  break;

Note the 'equals' part. Most uses for this Feeds mapper will probably need exact matching for titles. But if you look at _nodereference_potential_references_standard() in nodereference.module, it should also be possible to use 'contains', 'equals' or 'starts with' if you simply change that piece code I pasted above.

senzaesclusiva’s picture

@milesw: Many tanks for a your suggestion! Did the patch cleanly (how to previously read was not much accurate) , rebuilded node-reference-field, flushed cash, changed 'equals' in 'contains'...and now it works fine.
So , when i've a item title like "iPhone 5 with 8 megapixel produced by Sony", i've exactly the two words 'iPhone' and 'Sony' extracted in nodereference_field and now i can see it in both node-content-type 'producers/sony and 'producers/apple'.Very grateful to you.

Extractor is a very simple module and can be very useful in many needs:
Create for example a vocabulary "tags" with term to extract and assign it to feed-item content-type
Create in "feed-item content-type" a new field "myfeedstags"
Go to your feed-importer, change "common syndacation parser" with "common syndacation parser with term extraction".
Select source 'Extracted term names' to your target field 'myfeedstags'
Voilà! It supports multiple terms.
There's a little bug in the last version and here the easy patch:

@@ -94,7 +94,7 @@ function _extractor_simple_lookup($word)
}
}
if (!$loaded) {
- if (empty($vid)) {
+ if (empty($vids)) {
$result = db_query_range("SELECT td.tid, td.name FROM {term_data} td JOIN {extractor_lookup} e ON td.tid = e.tid", 0, 2000);
}
else {

Extractor ad nodereference.inc can work togheter, the first for tags and the second for the node-referencing.

milesw’s picture

@senzaesclusiva: I'm glad to hear you got it working. If you fixed a bug with Extractor you should try and post a patch in the Extractor issue queue. People are not as likely to find it here :)

bambilicious’s picture

Subscribing.

R.J. Steinert’s picture

R.J. Steinert’s picture

I'm having issues applying patch #115. When I apply the patch from the feeds directory "git apply feeds-mapper-nodereference-724536-115.patch" I get:
feeds-mapper-nodereference-724536-115.patch:119: trailing whitespace.
$field[] = array('nid' => $nid);
warning: 1 line adds whitespace errors.

And the nodereference.inc file get placed in the root feeds directory when it should go in the mappers directory. When I run "patch -p1 < feeds-mapper-nodereference-724536-115.patch" I get:
patching file nodereference.inc
patching file feeds/nodereference.csv
patching file feeds_mapper_nodereference.test

And the same results with everything placed incorrectly. What's the deal?

milesw’s picture

@rjstatic: Probably my fault, I created it with "--no-prefix" so it could applied with the familiar "patch -p0 < patch.name". Seems like Git doesn't appreciate that and we should always create Git style patches and expect to use "patch -p1 < patch.name".

I've gotten the whitespace errors with a few patches recently. Not sure why Git yells about that.

milesw’s picture

Cleaned up whitespace and recreated patch from #115 to be more Git-friendly. Should apply with either "git apply" or "patch -p1".

hosais’s picture

subscribe

jdwfly’s picture

I just tried out #129 and was able to successfully import using node title. I'd like to see this go into the module.

senzaesclusiva’s picture

Hi guys
A new little problem during matching with nodereference.inc.
It works fine, but I'm finding that the words of two letters (eg:hp,lg) are not associated to the own nid.
Then, title like "/producers/hp" are not referenced.
I don't understand if that is a nodereference.inc limit or if there is something else that affects this result.

Someone had this problem al solved it ?
tnx a lot

Encarte’s picture

subscribing

Anonymous’s picture

#129 worked for me with NID.

cfennell’s picture

#129 works well for me, thank you.

My only comment is that I'd prefer we use watchdog rather than drupal_set_message for the node mismatch warning.

Also, +1 on getting this into feeds as a core feature.

Júlio Boaro’s picture

subscribe

vasike’s picture

subscribe. i think all cck fields should be available for mapping.

IT100’s picture

What about the drupal 7 version? Is there an update available?

mry4n’s picture

This is great. What should we be doing to help move this along? Just more testing?

mry4n’s picture

(a reply to #11)

I believe the patches here did the following:

1. Create a file called nodereference.inc, which resides in feeds/mappers.

2. The rest of the patches are changes to that file.

mry4n’s picture

I'd like to try to apply these patches, but I can't find Version 6.x-1.x-dev.

I don't see it anywhere here: http://drupal.org/node/583948/release

Is the latest beta version the same thing as 6.x-1.x-dev?

milesw’s picture

@mike_tt: I believe the dev version can only be downloaded using git. On the Feeds project page, look for "Git instructions" in green up at the top.

Unfortunately the Feeds module is not nearly as active since alex_b handed it off, and it's not clear whether someone else will step up as the new caretaker. Testing patches will need to happen either way. :)

vasike’s picture

#129 worked for me, thanks.

mry4n’s picture

@vasike Are you importing multiple nodereferences via CSV?

vasike’s picture

no. actually i use mailhandler to import emails

mry4n’s picture

Thanks, miles.

I got my git in order, cd'd to my sites/all/modules folder and cloned the 6.x branch per Feed git instructions:

git clone --branch 6.x-1.x http://git.drupal.org/project/feeds.git

After cloning, I run git status and get this:

$ git status
# On branch 6.x-1.x
nothing to commit (working directory clean)

Does that mean I'm using the 6.x-1.x-dev branch, even though it doesn't explicitly include "-dev" in the tag?

At this point, which patch do I apply? If I apply yours in #129, do I only need to apply that one, or every patch that came before it as well in the thread?

And then, lastly, I get the impression that testing means more than just trying it out, not having your site crash, and then post that it worked? Do we need to implement Simpletest as stated by Alex in #34? Has anyone done that with these patches?

Once I get my head around this stuff, I plan to test away and even take a hack at the code. I really would like this to be able to import multiple nodereferences via CSV.

-M

milesw’s picture

@mike_tt: Yes, it sounds like you have the dev branch checked out. You should be able to simply use git apply file.patch. There's good general and Drupal-specific Git documentation here: http://drupal.org/documentation/git

By 'testing' I meant trying it out and trying to break things, but Simpletest tests are certainly nice to have too. You'll notice that someone already wrote a basic test and it's included with the patch.

Also, I should also point out that I didn't come up with this code and don't want to take credit for it. All I did was a little cleanup.

senzaesclusiva’s picture

#132 , #133

Hi All
Anyone can help us ?

mry4n’s picture

Thanks again, miles.

@senzaesclusiva I'm not really sure what the problem is by your explanation. What do you mean when you say: "title like "/producers/hp" are not referenced."?

My first thought is to ask if you've set up your urls so that /producers/hp actually points to something.

mry4n’s picture

OK, I finally got #129 applied. Pretty sweet. Properly adds nodereference by title and throws a nice error message when the nodereference title does not exist, as in when I tried to import multiple nodereferences by title, separating them with commas and ||.

So, my main question is: if I want to be able to import multiple nodereferences, does that belong in this issue/patch or should a new issue be created?

Also, I plan to check out the code in http://drupal.org/project/node_import which does allow multiple nodereferences to be imported. It even lets you choose what you want for delimiters. Wondering if some that code can just be repurposed here?

R.J. Steinert’s picture

@mry4n I think multiple node references is a must but for the sake of getting this committed it could be spun out into another ticket. But since it's working.... The main problem you are having is the node being referenced not existing prior to the import. I think that it would be good to have both options to not create new nodes as it does now and an option for creating new nodes. But, I think the latter should be a different field mapper. For now you can just make two Feeds importers that use the same source. The first one that runs should be the Importer to create the nodes that will be referenced (perhaps with the Title being the designated GUID as to avoid duplication), and then the second Importer imports the nodes that will be referencing that which were just created.

Status: Needs review » Needs work

The last submitted patch, 724536-128-feeds-nodereference-mapper.patch, failed testing.

johnv’s picture

About support for multiple nodereferences: As I understand maintainters comments in other issues, Feeds does not support parsing multiple values. At first I thought is was strange, but handling different delimiters (commas and ||) would be complicated.
IMO the right approach is:
- the formatter must support inbound arrays and 'cardinality';
- the import file can have multiple values like "artist name";"album1, album2";
- parsing the multi-value string should be done by Feeds Tamper. Here hou can specify the delimiter and much more.
Without the feeds parser, the "album1, album2" should/will/might be parsed as 1 value.

mry4n’s picture

Thanks, rjstatic & johnv. By what you're saying, it looks like all we care about here is that this mapper works for a single noderefernce. In my experience, it does.

However, now I see a System Message saying: "Failed to run tests: failed during invocation of run-tests" (#129 has the details)

@milesw (or anyone else): Do you know off-hand how we can fix this? I'll have to do some reading to understand what that message means.

In the meantime I'll check out Feeds Tamper.

-Mike

R.J. Steinert’s picture

@johnv Not supporting multiple values does seem strange considering that CCK fields are architected to have multiple values. It would be nice if the NodeProcessor class had support for this, but it can also just be handled on a each mapper class. Thank you for reminding me of Feeds Tamper. Feeds Tamper module seems a bit overkill to just support a standard CCK feature. Perhaps Feeds Tamper is on it's way to becoming part of the Feeds module itself, or at least this one critical part.

senzaesclusiva’s picture

#149 (#132-#133)

Hi mry4en and thanks for your interest.
Maybe I gave a bad explanation of this problem, then i repeat it in another way. It isn't a url problem..

Then...i'm importing an external feeds from IT sites and referencing them by node-title with nodereference.inc as feeds mapper and cck autocomplete text widget.
My content site is divided in categories; the node-referenced is "producers" with url mysite.com/producers/* , e.g:
mysite.com/producers/apple
mysite.com/producers/ibm
mysite.com/producers/asus ...an so on .

All works fine....but....
...if node-title has three or more then three chars , feed is correctly referenced with its node-title (nid), but if node-title has less than three chars, feed was not referenced, leaving autocomplete textfield blank.
e.g: mysite.com/producers/hp ...is not referenced with feed containing "hp" in feed title.

it seems that one of them (nodereference.inc or autocomplete-text-field) does not "recognize" titles less than three chars.

I wish to understand if this is a property that can be set in nodereference.inc code or depends to Drupal core /CCK settings. What else may affect this?

thanks a lot

mry4n’s picture

@senzaesclusiva

I can't say for sure what the cause might be, but have you tried making your node reference field widget something other than autocomplete? I know other modules will only work if the field is using the select list widget.

Also have you tried it without changing the URL of your nodes, so that the node url would just be example.com/node/54, for example?

milesw’s picture

@senzaesclusiva: I'm not completely clear on what you're doing because you're talking about a few different things. You mention node titles, nids, and path aliases. If there's a problem with the actual matching it's probably in the function _nodereference_potential_references_standard() in nodereference.module. Check the $match_clauses portion. I'm not sure how LIKE clauses work in MySQL.

@johnv @mry4n @johnv: When you're talking about multiple nodereferences, what do you mean exactly? Mapping multiple feed values to a single target?

Regarding the automated test failure in #129, I don't know how to fix it. On my system Simpletest doesn't have a problem loading any files, but this is the error coming from the automated system:

Output: [PHP Fatal error: require_once(): Failed opening required 'sites/default/modules/feeds/tests/feeds_mapper_test.inc' (include_path='.:/usr/share/php:/usr/share/pear') in /var/lib/drupaltestbot/sites/default/files/checkout/sites/default/modules/feeds/tests/feeds_mapper_nodereference.test on line 9].

mry4n’s picture

@milesw what I (and I assume the others) mean by multiple nodereferences is a node with a nodereference field that can take more than 1 nodereference.

Example:

NodeA has a nodereference field called subnodes for node types of NodeB.

There is a node of NodeA type called n1, and three nodes of NodeB type called nX, nY, and nZ.

nX, nY, and nZ already exist; n1 does not.

The CSV file for importing n1 looks like:

"title","subnodes"
"n1","nx||nY||nZ"

where the node references are separated by pipes (||).

title is mapped to Title

subnodes is mapped to subnodes

milesw’s picture

@mry4n: Splitting those values is something that would need to happen at the parsing stage. I've never used the CSV parser, so I can't offer much advice, but this nodereference mapper can indeed map an array of values to a single nodereference field. What I don't believe will work is mapping more than one source (say you had the "subnodes" column and then a "more_subnodes" column also) to a single nodereference field.

mry4n’s picture

@milesw

I assume that this nodereference mapper has the "potential" to import an array of values from a CSV file to a single nodereference field, but currently does not?

So, what do you think about using Feeds Tamper for doing this parsing? Do you think that the task of handling an array of values should be handled by something outside of this mapper?

milesw’s picture

@mry4n:

Currently this mapper can take an array of values and map it to a multi-value nodereference field. It doesn't care how or where that array comes from. The CSV parsing is a separate part of the process.

I've never used Feeds Tamper, but the project description sounds promising. The other possibility is you modify/extend FeedsCSVParser to parse the CSV data however you like. This might be what you're looking for: #759966: CSV: Support one to many relationships

Renee S’s picture

Patch #129 worked great for me. Thanks.

senzaesclusiva’s picture

Title: Mapper for nodereference field in Drupal 6 » #132 #133 Solved (nodereference.inc wasn't matching words less than three chars)

Hi All

Today I solved this problem .. that did not relate directly to the file nodereference.inc.

In my configuration i use the Extractor module to extract terms from feeds content and then let nodereference.inc do its job.

I did not want to believe that somewhere there not was a "length limit " to extracting or to matching. So I went to look at all the files/modules potentially involved.
Nodereference (nodereference.module as @milesw suggestion)
Extractor
Feeds

Finally I found this code in Extractor / extractor_simple.inc:

/ / Do not bother if word starts with a numeric or if word is in stop word list.
if ((strlen ($ word) <3) | | is_numeric ($ word [0]) | | in_array (strtolower ($ word), _extractor_simple_stopwords ())) {
return array ();
}

and I changed it to:
if ((strlen ($ word) <2)

Wowww! I tested it and now the titles of nodes (HP, EA, LG, etc) are properly referenced!

Thanks a lot to all!

milesw’s picture

Title: #132 #133 Solved (nodereference.inc wasn't matching words less than three chars) » Mapper for nodereference field in Drupal 6

@senzaesclusiva: Glad you were able to solve your problem. Please note that when you add comments in Drupal.org issue queues, the settings box applies to the whole thread, not just your comment. Changing the title back...

senzaesclusiva’s picture

Sorry all !!!

How can i change back title ? I don't find how in edit mode..

hosais’s picture

Hi,

Could anyone help me to clarify the current status?

I have feeds version 6.x-1.0-beta10 installed in my site. Is that right: to support node reference CCK field, I just change to 6.x-1.x-dev and apply the patch #129 ? Does this patch support multiple value?

By the way, anyone knows any issue 6.x-1.x-dev for production site? Well, except node reference, 6.x-1.0-beta10 works perfect for me. Any opinion are welcome.

Thanks.

hosais

mry4n’s picture

@hosias - Yes, just use the dev version and apply #129.

Anyone have any luck determining what's up with Simpletest? See #158

I'm going to look into it to see if I can figure out what's wrong.

AtlasX’s picture

Patch worked for me...almost.

Here's my problem:

It doesn't work with Node Reference Create. Instead of inputting the information in the field and then creating a node if one doesn't exist, I get this error:

'Blah-blah-blah' does not match a valid node title for the 'blah-blah' field.

Otherwise, it works perfectly for node reference fields which actually contain a node to be referenced.

Curious if I can find a solution to this.

twistor’s picture

@AtlasX, I doubt that it will work with noderefcreate explicitly. It would be possible to add the same functionality to the mapper that noderefcreate provides, but I think that's outside the scope of this issue.

joaomachado’s picture

Patch worked well, node reference on upload from CSV no issues. But when updating a node reference field to "empty", will not remove the reference. Let me explain. Let's say I have a node reference field to another node and the first time I upload the CSV file it has a value of B00, which is linked to the node B00. On the second update with a new CSV file, the field is no longer populated, it is empty. With fields that are node reference, it will not remove the reference. On fields that are not node referenced, it is not a problem, works just fine.

Any Ideas?
D6.22, Feeds 6.x-1.x-dev, PHP 5.2.17, MySQL 5.1.58

twistor’s picture

@JoaoMachado, looks like you've been bitten by #996808: Update existing doesn't reset targets that have real_target set.. Sure could use a review on that patch.

joaomachado’s picture

@twistor...Thanks, I just checked that link and will try it out and let you know!

joaomachado’s picture

@twistor I just posted on issue 996808 the test results...

itai_biren’s picture

Version: 6.x-1.x-dev » 7.x-1.x-dev
Component: Code » Feeds Import
Assigned: Unassigned » itai_biren
Priority: Normal » Major

hi, I'm using the drupal 7 and the patch seems to do nothing on that version.
is there a patch for me? can you guys pleas help me

twistor’s picture

Version: 7.x-1.x-dev » 6.x-1.x-dev
Component: Feeds Import » Code
Assigned: itai_biren » Unassigned
twistor’s picture

Priority: Major » Normal
verres’s picture

Category: feature » support

I'm trying to import a node-reference field, but need the referencing field to be one of the CCK fields in the target node.... reading this thread, I'm not sure if this patch will provide that. I currently can import node reference fields that map to 'nid', 'title, 'feeds URL' and 'feeds GUID'. I installed the patch in #115 - thinking that, once patched, I might see other options in the 'target' window, but still do not.

any help (even a quick: this cannot be done)- would be appreciated. If someone has imported node-reference fields mapped to cck fields - I'd love to hear what they did.

limehouse’s picture

Hi,

I'm trying to import (via CSV) a parent content type with a node reference field set to unlimited.

When I import only one child is displayed with the parent, but no others.

I don't know if I have configured the mapping incorrectly, or if there is an issue with the mapper.

I see from above that there are issues around multiple values, but I take it this is to do with them being in an array, rather than in separate rows in the CSV as I have done (the idea being to run 2 separate parent and child processors (child first) on the same CSV file). But I'm not a developer, so I can't pretend to understand everything discussed above!

Any help much appreciated.

Thanks

UPDATE: Just in case anyone else has the same issue, I resolved it by changing the node processor setting to 'update existing nodes' for the parent processor.

prodigeek’s picture

This may be slightly silly, but I've taken the nodereference.inc file, put it in the feeds mapper folder and I'm not sure if I'm supposed to do something else. The nodereference still does not show up in the mapping list. How do I activate it?

Thanks.

dman’s picture

I'm finding #129 working great for me today.
What is left holding it up? Can we get this working version in and look at the issues with multiple fields later or in a different thread (this one is too long and scary now)
Right now, this seems to work, and is incredibly useful - thanks especially for that GUID reference - it fit perfectly with what I needed to do.
It's flagged needs work - but testbot doesn't seem to provide more details

victormoya’s picture

Hi, can anyone upload an already patched version of the file "nodereference.inc" to use with D6 ?

I don't know in what order should I apply the patches... I tried downloading the first posted version of the file in #2 and then adding the patch from #129, but it seems to break the code. I found about 17 different patches posted in this threat, do they all need to be applied?

Thanks for your help

jeffschuler’s picture

@victormoya: Individual patches in an issue queue are generally directly against the version to which the issue is assigned. Though patches often build on each other, each should be complete in itself; you don't need to chain them.

In this case, the patch in #129 is the currently proposed solution to this issue, and should be applied to 6.x-1.x-dev.

See also the answer to this same question in #117.

victormoya’s picture

FileSize
11.3 KB

@jeffschuler: Thanks for your explanation, it seems to be the more sensitive, but I ran in doubts when I saw that applying #129] to #2 was breaking the file, with several <?php opening tags in the middle of the code and duplicated functions...

You say:

In this case, the patch in #129 is the currently proposed solution to this issue, and should be applied to 6.x-1.x-dev.

I assume the patches are for nodereference.inc file, posted on #2, not to the feeds module itself on its 6.x-1.x-dev version.

I'm posting here the results of my patch applied so you can understand what I mean, it's quite odd. Can anyone who has successfully applied the patch, post a patched and fully functional version of the mapper?

Thanks in advanced for your help

victormoya’s picture

FileSize
5.09 KB

Hi everyone,

I finally found a valid nodereference.inc file.
I downloaded it from http://drupalcode.org/sandbox/emorency/1128690.git/blob/9000665:/mappers/nodereference.inc. Wel... on fact I just copied it ('cause I didn't know how to download it) and removed the line numbers one by one... :S tedious work but it worth the pain, working like a charm, so finally I can import node references!

I will attach it so anyone can download it and just start using it without having to patch anything at all.

Thank you all!

dman’s picture

FYI, in the git repo, just hit the "raw" link at the top of the code pages.
You'd get http://drupalcode.org/sandbox/emorency/1128690.git/blob_plain/9000665:/m...

:-B

victormoya’s picture

@dman: yes, I just noticed it was a Firefox error, it leads to to a page saying "corrupted content"... With chrome you can download it correctly
However I managed to copy it manually and it's working now :)

jeffschuler’s picture

@victormoya: per #184 and for future reference, again: the patch in #129 is not to be applied to nodereference.inc from #2. It should indeed be applied directly to the feeds module itself, version 6.x-1.x-dev.

You should be able to use this general rule with patches any issue. They apply directly to the project.

Summit’s picture

Hi, Also interested in this, but for D7. Got all the time:

[node-naam] does not match an existing node
colan’s picture

Project: Feeds » Content Construction Kit (CCK)
Component: Code » General
Category: support » feature

Wrong queue. Although some contrib modules got their code into the Feeds module, they shouldn't have.

twistor’s picture

Project: Content Construction Kit (CCK) » Feeds
Component: General » Documentation

That's a nice thought, but most of the fields provided by CCK are already in Feeds so this should be in as well. We could move all of the mappers to CCK, but that's a separate issue.

twistor’s picture

Component: Documentation » Code
mikaoelitiana@gmail.com’s picture

Can I use this patch with the Feeds API to create a mapper that matches a node reference title from the CSV and create a reference to this node for the newly created node? I have 2 different CSVs, one containing references used in the other one, no ID provided inside.

Thanks

mikaoelitiana@gmail.com’s picture

Hi,

Have you found the right way to import 1-n nodereference?

Thanks

pdrake’s picture

mikaoelitiana, yes you can use this to create node references by title. In order to do 1:n references, you must be using an import file format that supports 1:n mapping, but it is also possible.

mikaoelitiana@gmail.com’s picture

Thanks @pdrake , I will try that

Encarte’s picture

Status: Needs work » Reviewed & tested by the community

This issue is getting chaotic. Lots of people have tested and are effectively and successfully using #129, to the point that someone has actually posted an already patched version of the code.

Isn't it more than time to get this committed and to start opening new issues for multi values and other problems?

As it is, this issue has became a problem in itself. Its a nightmare reading this.

jamesdixon’s picture

I've tested patch #129 and successfully imported node references using the node title. This seems like a good addition to the module.

shawngo’s picture

#129 also worked for me using node titles.

dakke’s picture

I think I might be up for a different use case, so let me explain first.

The situation:
- I have a feed that imports a blog
- The author of the blog does not have a Drupal account (cf. user)
- The 'blog' content type has a field 'node reference' linking to a different content type 'author' (for clarity I label it here 'node_ref_blog')
- I create a feed with an additional field 'node reference' ('node_ref_feed')

I want to import the blog and
- for each blog node created by the feed, the field of the feed content type (cf. 'node_ref_feed') should populate the node ref in the blog content type (cf. node_ref_blog)

Example:
- create a feed and add 'author x' in the node_ref_feed
- Pull in all data of the blog
- blog items created and in the 'node_ref_blog' the 'node_ref_feed' is copied

The patch in #129 does not provide a 'target' such as "Feed node CCK node ref", but it does provide a source with "Feed node CCK node ref".

Am I missing something?

asb’s picture

Re. #197: I fully agree, the patch is working well, and it is "RTBC".

ashedryden’s picture

Version: 6.x-1.x-dev » 6.x-1.0-beta12

Patch from 129 applied cleanly; can now target node reference fields.

jeffschuler’s picture

Version: 6.x-1.0-beta12 » 6.x-1.x-dev
chadmkidner’s picture

Has the working patch for this ever been committed? I'm using 6.x-1.0-beta12, should I apply the patch or install the dev version? Thanks!

asb’s picture

Priority: Normal » Major

@chadmkidner: No, the patch is not available in 6.x-1.0-beta12, for whatever bizarre reason.

I'm using the include file referenced in #186 which does it's job (reference by Feeds GUID, by Feeds URL, by NID, or by title).

ashedryden’s picture

Update: I've been using this patch for a few weeks and am noticing that when editing existing nodes, they eventually lose their node references and become locked in a way that makes them non-editable. Still trying to hunt down the source, but something to keep in mind for people testing this patch.

asb’s picture

Sounds very ugly, and hopefully something like this won't happen to me as well. However, I have not noticed a behaviour like this yet. But definitely yes, nodes created through 'Feeds' are not "clean" (you'll notice, for example, when changing the node type and filter a view without the new type). However, those oddities are not introduced through this patch.

Have you already opened an issue somewhere else for your problem?

chadmkidner’s picture

Thanks for the reply, I eventually just downloaded one the above patches (@asb very well could be the one you referenced) and I can confirm that I have had no issues using "Node Reference by Title".

Thanks for everyone work on this, was exactly what I needed!

twistor’s picture

Status: Reviewed & tested by the community » Fixed

Oh boy, I did it now.

Thanks to everyone who worked on this.

Please, please, please, do not reopen this issue.

http://drupalcode.org/project/feeds.git/commit/b63d9c7

joshuajabbour’s picture

Oh yay! So glad this monster finally got resolved. I haven't used feeds in a while, but am just now starting a project that might, so this is great... good job everyone!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.