I am setting up Aggregator2 and have problems with lost and/or duplicate items on feed refresh. I am running on Linux / PHP 5 with taxonomy, pathauto, trackback and menu_otf modules enabled and showing up on aggregator node edit form.

I configured 2 test feeds (see attached) to update existing items and have problems with both with the latest from cvs (rev 1.16) and with an earlier version before recent GUID/ID fixes (rev 1.14). However, the behavior is very different:

===== [Case A] Rev1.16: =====

A1) The feed node always has "Feed currently contains0 items" messages no matter how many refreshes I do. The nodes/items actually are created.

A2) The feed items are NOT shown on the feed node page and the "source site" link points to my base URL, not to the specific feed/source node.

A3) Every refresh creates duplicates of every node item.

===== [Case B] Rev1.14: =====

B1) On first refresh the zero-items message is the same as in A1, even though items are created. On further refresh the correct number of items appears.

B2) Same as A2

B3) Every refresh creates only ONE extra duplicate of the top feed item (link=http://www.gruntdoc.com/). which always has (fid=0)

======================

I hope I am wrong about that, but seems like there are some bugs here...

CommentFileSizeAuthor
agg2_sql.tar10 KBdkruglyak

Comments

ahwayakchih’s picture

ahwayakchih commented

I tested aggregated both feeds on my test site without problem :(

I think it may have something to do with reported earlier incompatibilites with PHP5. Currently i'm testing everything on PHP4 (which still seems to be most popular). I'll test on PHP5 when i'll finish planned changes.
But if You or any other developer could work on PHP5 fixes in the meantime it would be great :)

dkruglyak’s picture

dkruglyak commented

This indeed appears to be PHP 5 issue, apparently due to different handling of objects. It is solved by replacing the line 1387 in aggregator2.module from this:

$edit = $feed;

to this

$edit = clone $feed;

How shall we merge this into the main tree? What other PHP 5 gotchas might exist ? Surely PHP 5 is needed for a real 4.6 release.

dkruglyak’s picture

dkruglyak commented

Another issue discovered in the course of testing my PHP 5 fix.

When I create settings to promote all items to the front page, this causes a near infinite loop to 1000000000. The loop is ultimately broken with a PHP timeout and a SQL error (creation of a duplicate taxonomy term, while updating).

ahwayakchih’s picture

ahwayakchih commented
Status: Active » Closed (fixed)

Thanks!

I've just commited changes to CVS. Fixes cloning and loop issue.

dkruglyak’s picture

dkruglyak commented
Status: Closed (fixed) » Active

Looks like we are not done with cloning...

The problem is that many other contrib modules extend nodes as they please. Cloning feed node into feed item could therefore lead to unpredictable behavior. I ran into this while testing Node Relativity module that ended up copying feed node relations to feed items, cluttering my feed index. I am sure there are many other places where cloning would wreak havoc.

What is the minimum subset of fields that really needs to be copied from feed to feed item? I suggest we make this modification.

ahwayakchih’s picture

ahwayakchih commented
Assigned: Unassigned » ahwayakchih

We can't copy just few fields. That's the idea to copy as much as possible so it will work with additional modules also.

In future i plan using "template" nodes. So instead of using feed as template for item, there will e special item template. But it still may be not enough. So maybe we have to add list of fields to copy? But that will be too complicated for "common" end-user :(

For now please let me know which modules connect specific data to specific node. We can "filter out" those fields (like it is done with path alias field now).

dkruglyak’s picture

dkruglyak commented

I cannot speak for all aggregator2 use cases, but templates may be a good idea at some point.

My immediate roadblock is with Node Relativity module. It is an awesome way to organize nodes, so I created a directory of feed nodes (as a flexinode) and use relativity module to form child relationships with feed nodes. On feed node cloning feed items become relations of the index which foils the objective of getting the feeds organized.

So we need to remove these relations on feed item cloning. The name of the project / module is relativity.module

ahwayakchih’s picture

ahwayakchih commented
Status: Active » Closed (fixed)

Ok, now "parent_node" field is unset when creating item node. That should prevent messing up relativity module's data.