Context:
I have a feeds importer that tries to import a text value into a text field that has a limit of 20 characters. When the imported value is bigger than 20 characters, then an PDO Exception is thrown and error gets logged into feeds log along with a var_dump of the entity and imported item. Since the entity and item have a lot of fields/columns, the message is super big (~170k characters).

Problem:
On the feeds log page, that long message is displayed using t() function. But calling t() will trigger some other code (not sure if Drupal or contrib) that tries to add the whole message into locales_source table but the message is too long even for a blob field (65k characters) and then another PDO Exception is thrown inside Views render and the whole report page is replace by exception page.

Proposed solution:
Quick fix: do not use t() function for messages longer than 65k characters. Patched attached.

I've also checked to see how dblog module handles this, and they don't call t() at all if there are no arguments for the message. But this doesn't work in feeds, because we have a log of messages that don't have arguments but should be translatable. Or, we can move var_dump() output into arguments and don't add them directly into message (make it translatable).

Looking forward for your feedback.

CommentFileSizeAuthor
feeds.patch848 bytesLuxian

Comments

megachriz’s picture

I think Feeds shouldn't translate anything that is derived from an exception message. In some case that exception message is already translated, for example in includes/FeedsSource.inc, ± line 791:

protected function acquireLock() {
  if (!lock_acquire("feeds_source_{$this->id}_{$this->feed_nid}", 60.0)) {
    throw new FeedsLockException(t('Cannot acquire lock for source @id / @feed_nid.', array('@id' => $this->id, '@feed_nid' => $this->feed_nid)));
  }
}

I tracked down two places where Feeds logs a message derived from an exception message:

  1. From feeds.module, ± line 155:
    catch (Exception $e) {
      $source->log($method, $e->getMessage(), array(), WATCHDOG_ERROR);
    }
    
  2. From plugins/FeedsProcessor.inc, ± line 269:
    // Something bad happened, log it.
    catch (Exception $e) {
      $state->failed++;
      drupal_set_message($e->getMessage(), 'warning');
      $message = $this->createLogMessage($e, $entity, $item);
      $source->log('import', $message, array(), WATCHDOG_ERROR);
    }
    

At the following places there are also already translated messaged logged:

  • From plugins/FeedsProcessor.inc, ± line 345:
    foreach ($messages as $message) {
      drupal_set_message($message['message']);
      $source->log('import', $message['message'], array(), isset($message['level']) ? $message['level'] : WATCHDOG_INFO);
    }
    
  • From plugins/FeedsProcessor.inc, ± line 471:
    $message = format_plural(
      $state->deleted,
      'Deleted @number @entity',
      'Deleted @number @entities',
      array(
        '@number' => $state->deleted,
        '@entity' => strtolower($info['label']),
        '@entities' => strtolower($info['label plural']),
      )
    );
    $source->log('clear', $message, array(), WATCHDOG_INFO);
    drupal_set_message($message);
    

Perhaps the function feeds_log() and the method FeedsSource::log() should get an extra optional parameter for whether or not the message is translatable?

Luxian’s picture

Issue summary: View changes
megachriz’s picture

Priority: Normal » Critical
Status: Needs review » Needs work

Inspired by the patch from #2502419: Log messages XSS attack vector, I think we should log exception messages like this:

$source->log($method, '@exception', array('@exception' => $e->getMessage()), WATCHDOG_ERROR);

At places where for the message format_plural() is used, there should be
two $source->log() lines, one for each message. Something like this:

if ($state->deleted == 1) {
  $source->log('clear', 'Deleted @number @entity', $vars, WATCHDOG_INFO);
}
else {
  $source->log('clear', 'Deleted @number @entities', $vars, WATCHDOG_INFO);
}

Bumping to critical because these logged messages can bloat up the locales_source table.

  • twistor committed 9b6059c on 7.x-2.x authored by Luxian
    Issue #2364103 by Luxian, MegaChriz: Feeds error log crashes when log...
twistor’s picture

Assigned: Luxian » Unassigned
Priority: Critical » Major
Status: Needs work » Fixed
Issue tags: -PDO exception

I'm going to create a different issue for sorting out logging. We need a lot of work in that area.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

twistor’s picture