(I had earlier reported this as part of Deleted feed aggregator reappeared but decided to report this as a separate bug).

this feed worked fine in 6.26: http://feeds.feedburner.com/uua/Lxnn

however, in 7.14 it causes cron to fail with an error in the log:

PDOException: SQLSTATE[22003]: Numeric value out of range: 1264
Out of range value for column 'timestamp' at row 1: ... [:db_insert_placeholder_5] => -2209143600

I tracked this down to an incorrect pubDate in the 5 oldest entries (actually VERY OLD entries ;-) ): December 29, 1899 9:00 PM
which generates the negative timestamp value in the SQL error.

now, I have contacted the source of the feed and they acknowledge the problem and say that they will fix or remove the items -- but they haven't yet...

in the meanwhile, if this error happens with, say the 1st item in the feed, no items are updated in 7.14 (though as I mentioned earlier, 6.26 didn't have this problem). so I'm wondering whether 7.x might just skip problem items and continue processing remainder of the feed ?

Steve

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jday’s picture

Version: 7.14 » 8.x-dev
Status: Active » Postponed

The offending feed items must have been deleted/fixed in the url listed above so I can not reproduce the error. Seems like the next step would be for someone with SimpeTest chops to write a test to check for negative timestamp values.

Aron Novak’s picture

Title: incorrect pubDate in feed causes SQL error, and no updated items » Avoid breaking whole save process at default processor when there are faulty item(s)
Status: Postponed » Needs review
FileSize
7.73 KB

I think even if we care negative timestamps by hand, there will be such situations when saving one item is not successful. This should not prevent the rest of the feed to be saved.

Aron Novak’s picture

Updated comment and a bit better (hopefully) wording.

Status: Needs review » Needs work

The last submitted patch, aggregator-processor_faulty_item-1695852-3.patch, failed testing.

Aron Novak’s picture

Status: Needs work » Needs review
Aron Novak’s picture

Status: Needs review » Needs work
FileSize
4.03 KB

Let the work continue on this.
I re-rolled the things for the restructured aggregator module, tests to be included.

ParisLiakos’s picture

Issue summary: View changes
Issue tags: +Needs tests

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev
Spokje’s picture

Project: Drupal core » Aggregator
Version: 9.3.x-dev » 1.x-dev
Component: aggregator.module » Code

The aggregator module has been removed from Core in 10.0.x-dev and now lives on as a contrib module.
Issues in the Core queue about the aggregator module, like this one, have been moved to the contrib module queue.

larowlan’s picture

Issue tags: +Needs reroll

Needs re-roll for contrib

dcam’s picture

Version: 1.x-dev » 2.x-dev

The originally-reported database exception is still reproduceable in Drupal 10.1.

Drupal\Core\Entity\EntityStorageException: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'timestamp' at row 1: INSERT INTO "aggregator_item" ("langcode", "fid", "title", "link", "author", "description", "timestamp", "guid") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7); Array ( [:db_insert_placeholder_0] => en [:db_insert_placeholder_1] => 14 [:db_insert_placeholder_2] => Bad timestamp feed item title. [:db_insert_placeholder_3] => http://example.com/bad/timestamp [:db_insert_placeholder_4] => [:db_insert_placeholder_5] => This item has a very old timestamp, which is known to have caused a PDOException that breaks the entire import process. See https://www.drupal.org/project/aggregator/issues/1695852. [:db_insert_placeholder_6] => -2209172400 [:db_insert_placeholder_7] => http://example.com/bad/timestamp ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (line 815 of core\lib\Drupal\Core\Entity\Sql\SqlContentEntityStorage.php).

dcam’s picture

Issue tags: -Needs tests
FileSize
2.71 KB

I'm not happy with the current state of testing the importing pipeline. Because there are dependency problems in the pipeline the processor is tested in bits in other places. But although we have a test that ensures that the processor plugin system works correctly, we don't have a test just for the DefaultProcessor plugin's functionality. So I made one. It ought to be expanded to test for everything that the DefaultProcessor does, but for now this patch contains a test for the problem that kicked off this issue. Locally, this patch is failing with the exception shown in the comment above.

dcam’s picture

Status: Needs work » Needs review
dcam’s picture

FileSize
2.73 KB

I forgot to add the @group to the new test class.

Status: Needs review » Needs work

The last submitted patch, 23: 1695852-23-test-only.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

dcam’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
6.09 KB

I made some modifications from #6. Instead of wrapping the entire contents of the foreach loop in the try block I only wrapped the item's save() call. And instead of checking broadly for any Exception, I changed it to specifically catch EntityStorageExceptions. Also, I had to implement the logger channel because #6 was written in the D7 era.

dcam’s picture

FileSize
6.03 KB

I updated the new test class' docblock and removed an unneeded use statement.

  • dcam committed 93a1e06d on 2.x
    Issue #1695852 by dcam, Aron Novak: Avoid breaking whole save process at...

  • dcam committed e60f1864 on 1.x
    Issue #1695852 by dcam, Aron Novak: Avoid breaking whole save process at...
dcam’s picture

Status: Needs review » Fixed

I've been avoiding committing some of these recent patches because I want them to be checked first. But this one is a straightforward bug fix that corrects an uncaught exception. So I didn't mind getting this one in.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.