In function aggregator_parse_feed() the publication date / timestamp is derived out one of several fields. Sometimes a feed does contain none of these fields. The consequence is the item is created several times, each with the current date. The real problem is, these items are always shown on top.
This can be easily solved by replacing the timestamp with the timestamp from an earlier record of the item. I don't know how to create a diff, so I paste my current code:

    // Save this item. Try to avoid duplicate entries as much as possible. If
    // we find a duplicate entry, we resolve it and pass along its ID is such
    // that we can update it if needed.
    if (!empty($guid)) {
      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND guid = '%s'", $feed['fid'], $guid));
    }
    else if ($link && $link != $feed['link'] && $link != $feed['url']) {
      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));
    }
    else {
      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));
    }

    // if timestamp not known from input, but from a known record, use the records-timestamp
    if ($date == 'now' && $entry->timestamp) {
      $timestamp = $entry->timestamp;
    }

My changes:
1. select from the aggregator_item record also the timestamp.
2. if the timestamp still is the default 'now' and retrieved from a record from the database, use the latter as the timestamp.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

NROTC_Webmaster’s picture

Version: 6.22 » 6.x-dev
Status: Active » Needs review
FileSize
1.81 KB

While this seems fairly straightforward to me I'm not sure that this will be implemented.

This is also very similar to the way they implemented it in D8 although it was moved to aggregator.processor.inc

        if (!empty($item['guid'])) {
          $entry = db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = :fid AND guid = :guid", array(':fid' => $feed->fid, ':guid' => $item['guid']))->fetchObject();
        }
        elseif ($item['link'] && $item['link'] != $feed->link && $item['link'] != $feed->url) {
          $entry = db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = :fid AND link = :link", array(':fid' => $feed->fid, ':link' => $item['link']))->fetchObject();
        }
        else {
          $entry = db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = :fid AND title = :title", array(':fid' => $feed->fid, ':title' => $item['title']))->fetchObject();
        }
        if (!$item['timestamp']) {
          $item['timestamp'] = isset($entry->timestamp) ? $entry->timestamp : REQUEST_TIME;
        }
PROMES’s picture

Thanks for your patch.
I presume that either D8 or wrong or your patch. See the equal where clauses in the second and thir query, both: where ... link = "%s" ... . In the original lines it states once link = "%s" and once title = "%s".
But the title and link in the D8 implementation are reversed between the second and third query. So I don't know when to use title or link.

     else if ($link && $link != $feed['link'] && $link != $feed['url']) {
-      $entry = db_fetch_object(db_query("SELECT iid FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));
+      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));
     }
     else {
-      $entry = db_fetch_object(db_query("SELECT iid FROM {aggregator_item} WHERE fid = %d AND title = '%s'", $feed['fid'], $title));
+      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND link = '%s'", $feed['fid'], $link));

A further extension in my code is after the first query a test whether the entry allready exists (coming from real life data):

    if (!empty($guid)) {
      $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND guid = '%s'", $feed['fid'], $guid));
      // test whether this entry allready exists with anothere guid; is it an updated item?
      if (!isset($entry->iid)) {
        $entry = db_fetch_object(db_query("SELECT iid, timestamp FROM {aggregator_item} WHERE fid = %d AND link = '%s' AND title = '%s'", $feed['fid'], $link, $title));
      }
    }

I hope to improve this fine module.

NROTC_Webmaster’s picture

Status: Needs review » Needs work

That was in your original code that you posted. Yes, it should be title and I didn't catch that when creating the patch. If you could copy your entire section of code from line 799 to the end of your changes I can create a patch for this.

You will still need to get some other people to review this and then approval from one of the core maintainers to get this in.

PROMES’s picture

You are right. The difference is in my original code.
I now can create diffs. Attached is my current diff-file, based on version 6.26.
I don't know any other user of this patch. If someone likes this, please drop a line.

NROTC_Webmaster’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, aggregator-timestamp-fix-1304758-2.patch, failed testing.

The last submitted patch, aggregator-timestamp-fix-1304758-2.patch, failed testing.

NROTC_Webmaster’s picture

I'm not sure why this failed testing but you should make patches against the dev branch.

PROMES’s picture

New patch against dev branch.
I presume the test fails because I used as module name: aggregator-new.module.

NROTC_Webmaster’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, aggregator-timestamp-fix-1304758-3.patch, failed testing.

PROMES’s picture

The problems are:
1. failed testing: Detect invalid patch format. Ensure the patch only contains unix-style line endings.
2. I am working on a Windows pc, so the requirement seems not so easy to meet.
Do you know a Windows program to change the line endings into Unix style? I found a lot of same as #1 warnings on this site, but no solution.

NROTC_Webmaster’s picture

I like notepad++ as it has lots of options to set it up based on the coding standards. Eclipse can integrate the drupal coding standard with DrupalCS so that may be useful but in my opinion it is rather difficult to get started with.

NROTC_Webmaster’s picture

Additionally, you can go to Development toolsand pick from the list on the page.

PROMES’s picture

Thanks NROTC_Webmaster. After some testing with the settings of Notepad++, which already was one of my editors, I presume the file is now in Unix line endings.

ParisLiakos’s picture

Version: 6.x-dev » 8.x-dev

Should go in 8.x first

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev
Spokje’s picture

Project: Drupal core » Aggregator
Version: 9.3.x-dev » 1.x-dev
Component: aggregator.module » Code

The aggregator module has been removed from Core in 10.0.x-dev and now lives on as a contrib module.
Issues in the Core queue about the aggregator module, like this one, have been moved to the contrib module queue.

larowlan’s picture

Version: 1.x-dev » 2.x-dev
Category: Bug report » Feature request
Issue tags: +Needs reroll