Rules Integration / Working with the feeds field

I am a Rules rookie. How does a field need to be declared so that Rules is aware of it?

We are not talking about an CCK field, right? Or are we?

In case of an CCK the "add a node" and the "populate a field" action should do it. Else, it would be best to add an action for setting the feeds field value for a node.

ok, then I suppose this is now a feeds feature request to provide an action for setting the feeds field value for a node, maybe also a condition to check it?

I've been working on this. So far I'm thinking, create/update/delete feed nodes, an action to update a feed (importer or node), and a condition to check if it's a feed node. Should have something here soon.

Here's what I have so far. I'm definitely open to ideas on the topic, but this should provide enough to address the initial post and then some.

nm

Cleaned up a bit and fixed a couple minor bugs.

@twistor -

This is awesome. I had written a bit off one off code to do a subset of this, but your patch is much more well organized. One additional request - could we expand the patch include a few feeds triggers (e.g., when a feed importer is run)? A user could then, for example, configure emails to be sent to the admin every time that the feed items were updated.

My beta code for such a trigger is:

/**
* Implementation of hook_rules_event_info().
*/
function feeds_rules_event_info() {
  return array(
    'feeds_importer_run' => array(
      'label' => t('A feed importer has been run.'),
      'module' => 'Feeds',
      'arguments' => array(
        'source' => array('type' => 'source', 'label' => t('The feeds source.')),
      ),
    ),
  );
}

The trigger event itself would then go (I think) inside of the feeds batch callback (in feeds.module):

function feeds_batch($method, $importer_id, $feed_nid = 0, &$context) {
  $context['finished'] = 1;
  try {
    $context['finished'] = feeds_source($importer_id, $feed_nid)->$method();
  }
  catch (Exception $e) {
    drupal_set_message($e->getMessage(), 'error');
  }
  rules_invoke_event('feeds_importer_run', $source);
  watchdog('feeds',t('A feed !source has finished importing', array('!source' => $source->importer_id)));
}

Thoughts?

subscribe

@twistor - two thoughts as I've been playing around with the actions in this patch:

1. For the action "Set the url of a feed source", there seems to be a 128 character limit in the URL field, while the feed source URL has a significantly larger character limit. Especially for URLs that utilize replacement patterns, this limit seems way too low.

2. Eventually, I imagine that this patch will have to play nice with nodes that have more than one feed importer attached (#634462: Attach multiple importers to one content type (in D6)). I don't know what problems that pending patch could cause to these actions, but I thought I'd call it out here.

@brycesenz,

#1. Good catch.
#2. We will have to handle that approach, I'm not looking forward to it.

We should definitely have triggers/events. What about using feeds hooks as the events?

@twistor,

Any idea how to fix the character limit issue?

Regarding triggers/events with feed hooks - I originally tried to add the module invocation in the feeds.rules.inc file in the form of:

function feeds_rules_feeds_after_import(FeedsImporter $importer, FeedsSource $source) {
  rules_invoke_event('feeds_importer_run', $source);
}

However, the event never showed up in the Rules administration area. Reading into the Rules documentation (http://drupal.org/node/298549), I found this:

Note that all event related code except the invocation can live in the .rules.inc file as described here. So best the invocation is added directly to the .module file and wrapped into a module_exists('rules') check.

My takeaway is that we'd need to add the rules_invoke_event code to the feeds.module file directly, and so I don't see what we gain by using feed hooks.

Update regarding the character limits - this is a change that can be easily fixed in Rules itself #448922: Make recipient input expandable.

Two more bug reports: In the function _feeds_rules_action_set_path($source, $path, $type), the variable $url should be $path. It also needs to save the $source back to the database, or those changes are lost (unless I'm misunderstanding the design of the function).

Additionally, in the case of URLs at least, it's possible that token replacements would leave some undesirable characters in the rewritten URL (e.g. spaces). There should be some sort of validation/filtering preprocessing.

The updated code is below:

function _feeds_rules_action_set_path($source, $path, $type) {
  //Remove potentially bad characters from the Path.
  if ($type == 'FeedsHTTPFetcher') {
    $path = str_replace(' ', '-', $path);
  }
  $config = $source->getConfig();
  if (isset($config[$type])) {
    $config[$type]['source'] = $path;
    $source->setConfig($config);
    $source->save();
  }
  return array('feed_source' => $source);
}

Ok, this is a big one. I'll probably have to break it up, but I promise I have valid reasons for all of it.

• The $url, $path issue is fixed.
• The saving issue should be handled automatically. Rules is supposed to intelligently save data types. If not, there is a save source action, although it's most likely a bug.
• The reason for putting the events in hooks is three fold:
  • It's common practice.
  • Having rules specific code mixed up in regular code is uncouth.
  • We also get decent reasons for adding more hooks. See #1005128: Rules integration & enable modules to customize imports.

Beware of bugs in the code below; I have only proved it correct, not tried it

@twistor -

1. Thanks for the update. Hopefully I can start testing out some of these new features in the next few days.
2. Thanks for the clarification on events.
3. A new bug report:

The action 'Import from URL' gets stuck in an infinite loop in the function _feeds_rules_action_helper(). The variable $state is never declared (to the best of my knowledge).

subscribe

very interesting initiative, i'd love to be able to add feeds this way.
sadly the patch returns a lovely

Fatal error: Cannot redeclare feeds_feeds_after_import() (previously declared in /var/www/smiro2000/dev/sites/all/modules/feeds/feeds.module:847) in /var/www/smiro2000/dev/sites/all/modules/feeds/feeds.module on line 947

feeds-beta10
rules-1.4
looks like i should be using the beta version though... will update if status changes.
thank you!

@twistor,

I appreciate this Rules integration very much. One critical line of code is missing, though...

I was trying to add a rule to set the URL for a feed.
Upon reaching the step where I should fill in the arguments needed, there was only the text field for feeds_url argument, and not the select-box for feeds_source argument, as should be:

'arguments' => array(
   'feeds_source' => array('type' => 'feeds_source', 'label' => t('Feeds source')),
   'feeds_url' => array('type' => 'string', 'label' => t('Url')),
),

This is because the feeds_source data type was not defined. In feeds_rules_data_type_info() (Line 386):
return $info;
is missing, and the hook doesn't return the new data type to Rules module.

Other than that - great improvement... thanks.
Amit

Subscribing.
And one thing: What about adding new action to create a feed source node?
I didn't figure out how to create a feed source.

twistor, Thanks so much for your hard work on this so far. Unfortunately it still doesn't do what I am hoping it would. Or at least, I can't figure out how to do what I wish. I wish to manipulate the items that were created by the import. In my case, they are nodes, and I want to change the workflow status upon import/update. Can this be done with the patch that you've created already, or is this yet to be created? Thanks!

I have tried the patch at #20. As I see, the event "An import has been performed" works... But "A feed has been parsed" doesn't seems to work with all the other settings are exactly same...

Actually I think, the problem is with the condition "importer is"... If I set this condition, it doesnt work.

An update on my comment. Using the "An import has been performed" rule, I was able to pair a Views Bulk Operation (VBO) view that grabbed the newly created nodes and performed my workflow state change. Thanks so much for the patch and I look forward to having it in the production copy.

Hmmm - can someone reroll #20 and #25 together into one patch, please? I'm not sure I understand where to insert #25.

Nevermind I figured it out. New patch attached.

Whoops, wrong tag

This stuff needs to go into 7.x first. The relationship between this and feeds_rules needs to be sorted out as well.

Hi,

How is this related to http://drupal.org/project/feeds_rules ?
Greetings, Martijn

Hi,

Might be a bit late. But the patch in #32 works great for me. There is just the small problem of whitespace error when applying the patch, so I have fixed that and rerolled. Note: this is for D6.

Thanks, paulgemini!

Updates?!