It would be fantastic if it was possible to map incomming data to Node Reference fields. It is my only hurdle for importing a lot of data which I have customized into feeds from an export script I have.

The nodes are related, all the imported books are related to authors. First I import the author feeds, then I import the book feeds. The books should then use Node Reference to reference authors.

Using taxonmy for this might be possible by letting the Author nodes be sticky, but I would get a listing of all the books on the same page as the author, and not in the way I would like them; since I have views for the book listings already.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

wmclark’s picture

+1 for this - I have a similar need. CCK Reference Fields are commonly used for most sites.

wmclark’s picture

I went ahead and gave this a shot. It currently only handles the $feed_element as a string so nid's won't work in the patch I wrote. The string must match exactly to the referenced node title.

It seems to be working on my test site.

Please test and report.

wmclark’s picture

Status: Active » Needs review

Forgot to mark as "needs review".

Also, I wanted to mention that even if the field being mapped to is a required field, if no valid nodes are found, then the node is created anyway.

Is this a bug in FeedAPI not using the validate step of nodeapi or is this just something that isn't supported yet?

pkej’s picture

Also, I wanted to mention that even if the field being mapped to is a required field, if no valid nodes are found, then the node is created anyway.

I think that is the way it usually works. I'll give your patch a run around asap.

pkej’s picture

I tried this yesterday, and I had problems getting it to work. I will try on a clean install tonight (I hope) and detail what I do on the website after registering the first account on the site.

I have high hopes for this functionality.

alex_b’s picture

Waiting for pkejs findings here before I test myself.

pkej’s picture

These steps were performed on a Drupal 5.7 fresh installation.

  1. Go to ?q=admin/build/modules
  2. Activated CCK modules: "Content", "Node Reference", "Text" and "Number" all version 5.x.1.6-1
  3. Activated FeedAPI Default modules: "FeedAPI" and "FeedAPI Node" modules, both version 5.x.1.2
  4. Activated FeedAPI Add On modules: "Common syndication", "FeedAPI Aggregator" and "FeedAPI Inherit" all version 5.x.1.2
  5. Activated FeedAPI Add On module: "FeedAPI Mapper", version 5.x.1.0-beta6

That started up the initial modules; had to bump up the memory_limit of PHP to 16 MB at this point before continuing.

  1. Went to ?q=admin/content/types
  2. Selected "Add content type" tab
  3. Gave the new type a name "Author", left the rest of the page unchanged
  4. Saved
  5. Repeated the above steps and created a content type "Feed Story"
  6. Then I selected "Edit" in "Operations" on the "Feed Story" content type.
  7. Selected "Add Field" tab
  8. Created a field named "Author" of type "Node Reference"->"Autocomplete Text Field"
  9. Checked the "Required" check box in the "Data settings" field set.
  10. Selected "Author" as "Content types that can be referenced:"
  11. Saved the field settings

This should give me two content types where one should be able to reference the other. Now I needed to add an "Author" and a "Feed Story" just to see that I can reference an "Author" from the "Feed Story".

  1. Went to ?q=node/add/author
  2. Submitted an "Author" with the Title "Paul K Egell-Johnsen" and the Body "A thorough tester"
  3. Went to ?q=node/add/feed-story
  4. Filled in Title: "A New Story", started writing "Pa" into the Author: field and selected the suggestion from the autocomplete field "Paul K Egell-Johnsen [nid:1] and in the Body I wrote: "Lorem ipsum sic amet."
  5. Then I submitted the "News Story".
  6. Went to the front page to see my two newly created nodes, both functioning as expected. The "Feed Story" node shows a link to "Paul K Egell-Johnsen" as expected.

Now it is time to create the Feed

  1. Went to ?q=node/add/feedapi-node
  2. Title: "Cool Feed"
  3. Body: "A cool feed testing the node reference addition to the Feed API Mapper."
  4. In the Feed field set
  5. Feed URL: "http://drupal.org/node/feed"
  6. Checked: "Refresh feed on creation"
  7. Checked: "Update existing feed items"
  8. Opened the "Processors" field set
  9. Changed the "Node type of feed items:" to "Feed Story"
  10. Changed the "Promoted items:" to "3"
  11. Left the rest as is.
  12. Submitted

Next step is to create the mapping.

  1. Selected the "Map" tab
  2. Mapped "options->original_author:" to "Map to field_author (nodereference module)"
  3. Updated
  4. Selected the "Refresh" tab
  5. Went to "Home"
  6. At the bottom there are three stories from Drupal, when I tested those where "Businessweek picks Dries Buytaert as a 2008 top innovator", "Drupal 6.2 released, fixing security issues" and "Announcing Drupalcon Szeged 2008"
  7. None of them had the "Author:" rendered, as opposed to the "A New Story", which shows it on the same page.
  8. I went into one of the stories from Drupal by clicking on its title.
  9. Selected the "Edit" tab
  10. Verified that there is in fact an required "Author" field in the edit form, and that it is empty.

So, there it is. I am able to select the mapping for the feed, but the mapping never happens in reality.

To verify that it is the autonodereference at fault, I created a required text field called "Original Author" in the "Feed Story" node type. That field showed up after I deleted the old entries and refreshed the feed.

I also tried turning on Clean URLs, for some modules in the past it has been an issue wether it has been on or off.

Best regards,
Paul

pkej’s picture

OK

I have misunderstood how it works.

I added an Author named "drupal.org - Community plumbing", and now it automatically appeared. I was under the impression from Comment #3 in this thread that the referred node would be created if it didn't exist.

Upon rereading with the knowledge of how it works I now understand that wmclark meant that the feed node was created regardless.

So, no problems, it works here.

Best regards,
Paul

wmclark’s picture

Sorry for the confusion!

These are limitations because of other modules:

1. nodereference.module will not create new nodes if the title does not exist
2. FeedAPI Node does not validate fields before creating nodes. (So nodes might be created with invalid data)
3. FeedAPI Node does not allow you to moderate which feed items should be created as nodes. (Nodes with blank required fields can be created and published. Unwanted/Unrelated feed items can be created.)

I've created feature requests for some of these limitations.

Summit’s picture

Subscribing, somewhat the same feature request asked here:http://drupal.org/node/241226#comment-792378, right?
If not sorry to interrupt.

Greetings,
Martijn

pkej’s picture

Works here.

kehan’s picture

subscribing

vanderlip’s picture

Thanks for this node reference mapper!

I am using the Node Reference Mapper in #2 with Feed Element Mapper 5.x.1.x-dev. The first time I run a feed refresh after creating (or after deleting all feed items), the mapper works flawlessly, but on each subsequent refresh (either manual or cron), new node references are not mapped to the feed items. The new feed items (nodes) are created correctly and other Feed Mapper Mapped items, CCK Date, and links, are mapped correctly, but the node reference is not mapped.

The node references that are not being mapped in subsequent refreshes are the same as those that were successfully mapped in the first refresh, so I don't suspect there there is a problem with invalid node reference data.

Any assistance would be greatly appreciated.

vanderlip’s picture

Update to #13

Upon closer examination, node reference mapper appears to work sporadically after the first refresh (which works 100% correctly). For instance, in a resent refresh 1 of the 4 new feed item's node reference values were mapped, the other three feed items were not mapped, even though the value to be mapped was mapped correctly in a previous refresh. I have checked the feed and all the data to map correctly is there.

Thanks again for any ideas.

alex_b’s picture

Status: Needs review » Needs work

Changing status as a response to #14

scottrigby’s picture

Version: 5.x-1.x-dev » 6.x-1.x-dev
Status: Needs work » Needs review
FileSize
2.37 KB

Updated patch for 6.x-1.x-dev. I haven't been able to reproduce the problem from #14 in D6 with this patch.

  • The main change I made was to _nodereference_potential_references()
    D5: _nodereference_potential_references($field, $return_full_nodes = FALSE, $string = '', $exact_string = false)
    D6: _nodereference_potential_references($field, $string = '', $exact_string = FALSE)
    See http://api.freestylesystems.co.uk/api/function/_nodereference_potential_....
  • The only other changes are comments, etc, to match feed_mapper_content.inc.

Any hesitations for including this in 6.x?

jesss’s picture

I just tested this patch and it didn't work for me. This may be because I'm using 6.x.-1.0-beta4, not 6.x-1.x-dev, but just in case it's helpful, I thought I would post my results.

However, it did work when I replaced the else if ($op == 'map') { section with the matching section from feedapi_mapper_content.inc, replacing $field[0]['value'] with $field[0]['nid'].

The original code from the patch

  else if ($op == 'map') {
      // Here is where the actual mapping happens.
      // When we are called at this point, $field_name contains the name of the field the user has 
      // decided to map to and $field_element is the feed item element the user has decided to map.
      // We just need to put the two things together. The data structure here depends a lot on
      // CCK. We stick the value in $feed_element into $node->$field_name[0]['value'].
      if (!is_array($feed_element)) {
        $cckfield = content_fields($field_name);
        // We are checking for nodes that exactly match the string mapped to $feed_element. We 
        // could have checked to see if it was numeric first and do something else
        $nids = _nodereference_potential_references($cckfield, $feed_element, TRUE);
        $field[0]['nid'] = (!empty($nids)) ? array_shift(array_keys($nids)) : 0;
        $node->$field_name = $field;
        return $node;
      }

The code that worked for me

else if ($op == 'map') {
      // Here is where the actual mapping happens.
      // When we are called at this point, $field_name contains the name of the field the user has
      // decided to map to and $field_element is the feed item element the user has decided to map.
      // We just need to put the two things together. The data structure here depends a lot on
      // CCK. We stick the value in $feed_element into $node->$field_name[0]['nid'].
      if (!is_array($feed_element)) {
        $field = isset($node->$field_name) ? $node->$field_name : array();
        $field[0]['nid'] = $feed_element;
        $node->$field_name = $field;
      }
      return $node;
    }
alex_b’s picture

Status: Needs review » Needs work

Needs a patch.

jesss’s picture

Here you go. This is my very first patch -- please be gentle. :-)

patchak’s picture

This does not work for me, I tried with node titles as the reference and not the nid's maybe that's the reason.

Using beta 7 patched version from no. 19

Patchak

jesss’s picture

The patch matches on nids, not titles. Since the title isn't necessarily unique, matching on title could open the door to a lot of potential mis-matches.

alex_b’s picture

@jesss - I only see this now, but you've patched the patch in #19.

Can you roll a new patch that includes the entire mapper? Hint: You will need cvsdo for adding a new file to the module.

patchak’s picture

Well In my use case I have only use for titles, so I think that would be a choice for a user...

I reverted back to the original mapper for d6 in no.16 and that one works fine. I think we should offer a choice to use either titles or nid's for the mapping, since most clients would have titles and not nid's in their legacy data exports...

Patchak

alex_b’s picture

Title: Node Reference fields... » Mapper for Node Reference fields
looplog’s picture

+1

ndm’s picture

Version: 6.x-1.x-dev » 6.x-1.0
Status: Needs work » Needs review
FileSize
2.15 KB

I can't understand interest to map to node id. How can a feed know your nodeid???

So i have take code from #2 and adapt with standards of 6.x-1.0. Now link is made with node title, if doesn't exist even if you have a autocreate, nothing happens (more sure, i think for miss orthographic problem).

So for me that's work, and for you?

SocialNicheGuru’s picture

subscribing

MikMik’s picture

Category: feature » bug

Hi there,

I tried the patch in #26, but it didn't work at all! I don't see a difference in the mapping-screen. Actually, this counts for all the patches in this thread.
What I noticed in the #26-patch is that the function it declares is "content_feedapi_mapper", but this function has already been declared in another file, so php throws an error (cannot redeclare function etc).

Are there any updates for this highly anticipated feature?

Jax’s picture

@26:
I can't understand interest to map to node id. How can a feed know your nodeid???
I'm writing my own eparser which can retrieve the correct related node id's throught the guids.

Say I have a first feed with authors:

<authors>
  <author id="1">name #1</author>
  <author id="2">name #2</author>
</authors>

I've written an eparser to turn those ids into guids for feedapi. Then I have a second feed with books:

<books>
  <book id="1">
    <name>book name</name>
    <authors>
      <author id="1"/>
   </authors>
  </book>
</book>

I've written a second eparser that can take the id from the author and fetch the corresponding nid from the feedapi_node_item table. In this case I need to be able to map the nid.

ncameron’s picture

Had problems with some of the above inc files / patches. I hashed the attached patch together from using other examples in the folder. Works really well for me.

ncameron’s picture

attached to this is original inc file which might be easier to use than the patch

nemchenk’s picture

Version: 6.x-1.0 » 6.x-1.3
FileSize
1.75 KB

I couldn't get #30 or #31 one to work with Feed Element Mapper 6.x-1.3, so I used them to write a mapper that does work, which I attach to this post.

Summit’s picture

Hi, would be great to move this also to feeds (www.drupal.org/project/feeds), while feedapi will move completely to feeds.
greetings, Martijn

jcbou’s picture

#30 31 and 32 still don't work for me. Could you help?

nemchenk’s picture

I could try with #32, jcbou -- what's the problem?

nemchenk’s picture

Updated mapper for D6 syntax -- the previous mapper would match the first partial title match, not an exact match.

501cGeek’s picture

I have essentially the same need, importing news stories from RSS and creating a node reference in each story to the node of the journalist that wrote the story. But I'm trying to use Feeds instead of FeedAPI since it's the new hotness, also we have a need to import CSV data which Feeds also provides.

I created a Feeds importer that is attached to the journalist node type, which puts a field in each journalist node for the URL of an RSS feed of stories for that journalist. Perfect. I just want to add a node reference in each imported feed story that links to the journalist node that feed is attached to.

Any advice?

501cGeek’s picture

sorry if that question was offtopic, to somewhat answer it... it looks like there are patches to this effect, that are moving toward someday being included in the Feeds module, but for the short term maybe FeedAPI offers more options relevant to this specific task. Need to investigate both options further, patch Feeds or switch to (and possibly patch) FeedAPI...

http://drupal.org/node/724536
http://drupal.org/node/632920

nemchenk’s picture

"for the short term maybe FeedAPI offers more options relevant to this specific task."

Precisely why, while Feeds may be the "new hotness", I continue to use the "old and busted" FeedAPI for now :)

hanmant.sapkal’s picture

How to use this file?

alibama’s picture

patch 36 worked fine for me - with ~40,000 nodes and two references per node the import took ~2.5 days...