FeedsImporter/FeedsScheduler should log better stats when refreshing. This will be very helpful when fixing performance problems later.

Members fund testing for the Drupal project. Drupal Association Learn more


m3avrck’s picture

Version: » 6.x-1.x-dev

Also would be great to have an admin page showing each feed, last time updated, etc... nice quick browse page for admins.

alex_b’s picture

alex_b’s picture

From #870540-6: Option to suppress watchdog message: relate log entries to feed nodes so that they can be browsed by (together with the) feed node.

rbrownell’s picture

+1 Subscribe

alex_b’s picture

Version: 6.x-1.x-dev » 7.x-2.x-dev
Status: Active » Needs work
7.85 KB

This is a first untested shot at moving all watchdog logged activity to a feeds log. The vision here is to log related to importer / source and higher detail and eventually provide a log tab on sources.


alex_b’s picture

This will also allow for much more verbose logging as we won't clog the watchdog log anymore.

- Add log messages when items are created/updated #948936: Add watchdog entry on node create/update

alex_b’s picture

Title: Log stats » More detailed log
32.81 KB

- Added update hook.
- Added views integration.
- Added default view.

See default view on a feed node:



- log tab on standalone form is not working (you'll see I've reworked hook_menu() but log tab is still not showing).
- add a global log view on admin/reports/feeds.

alex_b’s picture

Status: Needs work » Needs review
36.36 KB

This patch adds:

- log tab on standalone form is not working (you'll see I've reworked hook_menu() but log tab is still not showing).
- Severity filter on both feed nodes and standalone form.


An all subscriptions log on admin/report/feeds

alex_b’s picture

39.17 KB

This is complete now. Will commit shortly.

alex_b’s picture

46.86 KB

More tests revealed... that I forgot to export the improved view.

- Adds all subscriptions log on admin/reports/feeds, including necessary views integration.

alex_b’s picture

Version: 7.x-2.x-dev » 6.x-1.x-dev
Status: Needs review » Patch (to be ported)

This is committed now.


It would be great if someone wanted to backport this patch to 6.x 1.x, shouldn't be too hard and I think a lot of people could benefit from this improvement.

twistor’s picture

Status: Patch (to be ported) » Needs review
45 KB

This is a quick port. It's still using views 3. There should probably be logging added to processors besides the node processor or we could move it into FeedsProcessor. Tests pass.

EvanDonovan’s picture

Looks like a good improvement; maybe I'll test tomorrow, but I would have to upgrade to Views 3.

Would it be out of scope to add logging of feed import times to this patch? I'm trying to figure out how long it takes for my feeds to import, to assess the scalability of the system. See http://drupalbin.com/18789 for how I'm trying to do it.

It looks like if I could get access to the currently active $importer, I could at least log the importer title using $importer->config['name'];. But could I also get the number of nodes imported?

EvanDonovan’s picture

If I'm not mistaken, it looks like the issue marked duplicate at #948936: Add watchdog entry on node create/update to show how many created/updated is not actually in the committed patch, or the one for 6.x. Am I correct?

Here's some code that adds more detailed logging for nodes created/updated. The only thing I haven't figured out yet is how to get the name of the feed from which the nodes are being imported:

  public function process(FeedsImportBatch $batch, FeedsSource $source) {
    // Keep track of processed items in this pass, set total number of items.
    $processed = 0;
    if (!$batch->getTotal(FEEDS_PROCESSING)) {
      $batch->setTotal(FEEDS_PROCESSING, count($batch->items));

    while ($item = $batch->shiftItem()) {
    // snip the loop code...
    $timer_value = timer_stop('feeds_import');
    $import_time = $timer_value['time'] / 1000;

    // Set messages. (Evan: added logging to watchdog.)
    // TODO: The @feedname information is not here - find out how to get access to the $importer->config.
    if ($batch->created) {
      drupal_set_message(format_plural($batch->created, 'Created @number @type node.', 'Created @number @type nodes.', array('@number' => $batch->created, '@type' => node_get_types('name', $this->config['content_type']))));
      watchdog('feeds', 'Feeds module created @number @type nodes from @feedname feed in @time seconds', array('@time' => $import_time, '@number' => $batch->created, '@type' => $this->config['content_type'], '@feedname' => $this->config['name']), WATCHDOG_DEBUG);
    elseif ($batch->updated) {
      drupal_set_message(format_plural($batch->updated, 'Updated @number @type node.', 'Updated @number @type nodes.', array('@number' => $batch->updated, '@type' => node_get_types('name', $this->config['content_type']))));
      watchdog('feeds', 'Feeds module updated @number @type nodes from @feedname feed in @time seconds', array('@time' => $import_time, '@number' => $batch->created, '@type' => $this->config['content_type'], '@feedname' => $this->config['name']), WATCHDOG_DEBUG);
    else {
      drupal_set_message(t('There is no new content.'));
      watchdog('feeds', 'There was no new content in @feedname', array('@feedname' => $this->config['name']));
njbooher’s picture

35.27 KB

This is the patch from #12 updated to latest head with Views 2 support.

emackn’s picture

Status: Needs review » Postponed (maintainer needs more info)

looks like some of this got in already, so I'm marking this postponed until it's sorted out.

twistor’s picture

Version: 6.x-1.x-dev » 7.x-2.x-dev
Issue summary: View changes
Status: Postponed (maintainer needs more info) » Closed (fixed)

Not interesting in backporting this anymore.