Problem/Motivation

The module currently creates queue items for every published node on every cron. It's likely, and entirely possible, that duplicate queue items are being created.

An example with cron running every hour:
Cron runs. During the queue time not all the items are processed.
Con runs again. Since every published node gets a queue item, some of the items are duplicates.

Proposed resolution

A database solution like Overriding Queues in Drupal 7. Or How Not to Create Duplicate Queue Items is not performant. It takes creating items from seconds to too many minutes to even count.

I'm looking into a solution like the one described in How to check if an item exists in a Queue?

Adding this feature to the module will be a performance boost for the module since duplicate queue items won't exist.

Remaining tasks

1. Add a GoogleAnalyticsCounterProcessor and GoogleAnalyticsCounterProcessorInterface to the module which only creates an item when the same item hasn't already been created.

User interface changes

None

API changes

GoogleAnalyticsCounterProcessor service will be added to the container.

Data model changes

None

CommentFileSizeAuthor
#7 3022540-7.patch928 bytesid.conky
#5 3022540-5.patch927 bytesstrykaizer

Comments

esod created an issue. See original summary.

  • esod committed e84959e on 3022540-prevent-duplicate-queue-items
    Issue #3022540 by esod: Rename service class.
    
lucashodge’s picture

I am working on a site with ~100000 nodes. The number of queue just continues to grow so it quickly gets past 3million items in the queue table. I am using the latest 3.x-dev branch and it was also doing the same thing on the alpha19 version. I am sometimes getting a timeout on the cron run so maybe this is the problem and why duplicates are created. Any help on this matter would be greatly appreciated.

lucashodge’s picture

@esod I can't see these files in the latest dev version of the module:
/src/GoogleAnalyticsCounterProcessor.php
/src/GoogleAnalyticsCounterProcessorInterface.php

Did these changes get committed?

strykaizer’s picture

StatusFileSize
new927 bytes

Patch attached

strykaizer’s picture

Status: Active » Needs review
id.conky’s picture

StatusFileSize
new928 bytes

Rerolled patch for latest 8.x-3.x-dev.

  • id.conky authored e2976a7d on 4.0.x
    Issue #3022540 by StryKaizer, id.conky, lucashodge, esod: Prevent...
kaszarobert’s picture

Version: 8.x-3.x-dev » 4.0.x-dev
Status: Needs review » Fixed

I committed the patch to branch 4.0.x as of July 1st, 2023 Universal Analytics will stop working and that branch will support GA4 instead of UA.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.