Problem/Motivation

Finalized assets uploaded to Acquia DAM are not consistently syncing to Drupal across sites using the Acquia DAM module.
In certain environments, finalized assets are neither automatically detected nor processed by the Drupal DAM queue.
As a result, assets can take multiple days to appear in Drupal unless manual intervention is performed.

To mitigate this, a manual workaround is required to explicitly populate the DAM queue using a Drush command.
However, even with this workaround in place, queue processing behavior is inconsistent: on some sites the queue drains normally, while on others the queue backlog grows continuously and items remain unprocessed.

This inconsistency creates operational risk and prevents teams from relying on Acquia DAM as a near‑real‑time content source for Drupal.

Steps to reproduce

  1. Upload a new PDF asset to Acquia DAM.
  2. Mark the asset as finalized in DAM.
  3. Wait for the automated DAM → Drupal sync process.
  4. Observe that the finalized asset is not automatically added to or processed by the Drupal DAM queue.
  5. Manually run the following Drush command to populate the queue:
    drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version
  6. Verify that DAM items are successfully added to the Drupal queue.
  7. Allow Drupal cron to run and process the queue.
  8. Observe that on some sites the queue drains correctly, while on other sites queue items accumulate and are not fully processed.

Proposed resolution

Provide a reliable and consistent mechanism within the Acquia DAM module to:

  • Automatically detect newly finalized DAM assets without requiring manual intervention.
  • Ensure DAM queue workers process items at a consistent and predictable rate across sites.
  • Avoid queue backlogs caused by throttling, stalled workers, or silent failures.

Additional logging, rate‑limit awareness, or configurable batch/throughput controls for DAM queue workers may help identify and resolve the underlying issue.

Remaining tasks

  • Identify why DAM queue workers process items successfully on some sites but stall on others.
  • Determine whether API rate limiting or worker concurrency contributes to the backlog.
  • Clarify the expected behavior and SLA for finalized asset synchronization.
  • Document any recommended cron or queue configuration for stable production usage.

User interface changes

None.

API changes

None known.

Data model changes

None.

Comments

pradeepjha created an issue. See original summary.

rajeshreeputra’s picture

rajeshreeputra
he/him
Location Pune
commented

Hello @ pradeepjha,
If you execute drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version command to queue assets.
You must execute drush acquia-dam:process-update-queue to process the queued assets, otherwise gets accumulated.

pradeepjha’s picture

pradeepjha commented

We currently run two scheduled Drush commands per site:

  • drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version
    (every 10 minutes) to enqueue finalized DAM assets.
  • drush cron
    (every 5 minutes), which we expect to execute all registered queue workers, including Acquia DAM queue processing.

With this setup, the DAM queue is processed successfully on two sites and the queue drains as expected. On one site, however, queue items continue to accumulate despite cron running normally.

This suggests that queue processing is being triggered via cron, but is not behaving consistently across sites. Please clarify whether
acquia-dam:process-update-queue must be scheduled explicitly, and why cron-based processing works on some sites but not others with the same configuration.

rajeshreeputra’s picture

rajeshreeputra
he/him
Location Pune
commented

To illustrate the concern, consider the following scenarios:

Site 1 (manageable load):

  • Total assets: 100
  • Assets queued every 10 minutes: 20
  • Cron runs every 5 minutes
  • Each cron execution processes ~7 items

In this case, the queue remains under control:

  • First cron run processes ~7 items
  • Second cron run processes the remaining items
  • The queue is consistently drained, and no backlog accumulates

Site 2 (high load):

  • Total assets: 1000
  • Assets queued every 10 minutes: 200
  • Cron runs every 5 minutes
  • Each cron execution processes ~7–10 items

Here’s what happens:

  • First cron run processes ~10 items → ~190 remain
  • Second cron run processes another ~10 → ~180 remain
  • Meanwhile, the command
    drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version
    executes again and adds another 200 items

Now the queue grows to ~380 items, and this pattern continues.

This leads to a compounding backlog because the rate of queue consumption is significantly lower than the rate of queue production.

pradeepjha’s picture

pradeepjha commented

We have two sites where the number of PDF assets is around 400–500, and in both cases the DAM queue is processing without any issues.

The problem is occurring only on one site where the number of PDFs is around 1,600. This does not seem like a very large volume, and it should process correctly. Also, we are not updating all assets at once; updates are happening incrementally.

All three sites are using the same Acquia DAM connection, which suggests this is not a DAM configuration or credential issue.

Based on these observations, it looks like there may be an issue with how the module handles multisite environments when multiple sites are requesting DAM asset syncs, especially when one site has a higher number of DAM-linked media entities.

It seems possible that something is not being handled correctly at the queue or locking level when multisites are involved.

Could you please advise if:

  • There are known limitations with DAM asset syncing in a multisite setup
  • Queue locking or cron overlap could cause this behavior across sites
  • Additional configuration is recommended when a site has a larger number of DAM assets
rajeshreeputra’s picture

rajeshreeputra
he/him
Location Pune
commented

We would suggest updating to Acquia DAM 1.1.14 version and removing the asset update queue job(drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version).

pradeepjha’s picture

pradeepjha commented
Version: 1.1.13 » 1.1.14

Hi Rajeshreeputra,
I upgraded the Acquia DAM module to version 1.1.14 and removed the asset update queue job (drush acquia-dam:queue-update-assets --media-type=acquia_dam_pdf_asset --finalized-version) the day before yesterday.

During testing yesterday, uploading 7–8 documents for a single site worked as expected. I also tested uploading a single updated document across all three sites, and the versioning behaved correctly—I was able to see the latest version.

However, today, when we tested uploading 7–8 documents simultaneously across all three sites, none of the updates were reflected, even after waiting for over an hour.

Additionally, when I manually click on “Check for Updates” and run cron, everything works as expected. However, our objective is to ensure that the automated cron process functions reliably across all three sites.

Could you please confirm whether Acquia DAM asset syncing has been tested in a multisite architecture with shared configurations across sites?

pradeepjha’s picture

pradeepjha commented
Status: Active » Closed (won't fix)

Thanks, Rajeshreeputra, for the explanation. Since this appears to be a known DAM issue and there is an alternative approach when marking an old version of a document as finalized, I’m closing this issue for now.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.