Dependency Calculation should recurse deeper than 4 levels

This actually looks like it cannot find commerce_product -

The fatal in the log makes it look like commerce_product is the module that isnt enabled:

Fatal error: Call to undefined function commerce_product_save() in /var/lib/drupaltestbot/sites/default/files/checkout/sites/default/modules/authcache/modules/authcache_commerce/authcache_commerce.test on line 91
FATAL Drupal\authcache_enum\Tests\AuthcacheEnumKeysTestCase: test runner returned a non-zero error code (255).

But the test shows:
Enabled modules: entity, entity_token, rules, addressfield, ctools, views, field, field_ui, field_sql_storage, commerce, commerce_product, commerce_price, commerce_customer, commerce_line_item, commerce_order, commerce_product_reference, commerce_payment, commerce_tax, commerce_product_pricing, commerce_ui, commerce_checkout, commerce_cart, commerce_line_item_ui, commerce_order_ui, commerce_product_ui, commerce_customer_ui, commerce_payment_ui, commerce_product_pricing_ui, commerce_tax_ui, commerce_payment_example, authcache_commerce, authcache_form, authcache_p13n, authcache_p13n_test, cacheobject

Does this test pass locally?

Ah, I see. This isnt testbot related, but dependency calculation related (the testbots get passed in the dependency info). The same issue happens on drupalci - no addressfield in the dependency list.

And your hunch was right: http://cgit.drupalcode.org/project_dependency/tree/project_dependency.dr... (line 524).

We could probably set that to 100 levels and not really be risking anything at all.

Yes, you are correct that the recursion is limited to 4 levels. I have seen that many times in the code and wondered about it but no one has brought it up as a problem until now. We could probably set it to 100 and it would be okay but ... there is always a chance we get an infinite loop and even if there are only 2 dependencies that would be 2^100. I think it probably is a good idea to have some limit but 4 seems rather low. How about changing it to 10?

Okay, I am a little lost here. Are you saying 7.x-2.x-dev is failing?

There are two blocks that appear on the release page that show the components and dependencies of the release. They are only available to certain roles although someone could request them #760890: Display module dependencies on project pages. I will be attaching screenshots with these blocks. Release 7.x-2.-dev shows dependencies including commerce 7.x-1.11 but I do not see addressfield. When I check 7.x-2.0-beta7 no components or dependencies appear. This usually means that there was a problem when the release was created storing this data to database tables. If this is a problem then request someone on Drupal.org to run

drush pdpv 7.x-2.0-beta7

and it should try to recreate the dependencies. Release 7.x-2.0-beta6 looks like it has the dependencies. There may also be a problem with the info files for the latest release but I want to make sure I am on the right track first.

The screenshot is attached. I think I finally am beginning to understand the problem. Commerce does have a dependency on addressfield but yesterday it was not showing up when I grepped for it so I was confused. Now I see the dependency is specified in commerce_customer.info. AuthCache does not have a dependency on Commerce Customer. I have not looked to see if there is an indirect dependency but either you have not specified a dependency on Commerce Customer or that dependency takes more than four levels.

Thanks Trobey, I went ahead and did a drush pdpp authcache as it seemed like many were missing their dependencies, and dependencies seem to be showing up now for authcache.

I re-ran this on drupalci (https://dispatcher.drupalci.org/job/staging_simpletest_job/209/console)

And it's still having problems finding the right dependencies.

the external release dependencies for 1910022 (the 2.x-dev release that we're testing) shows the following:

drush ev 'var_dump(project_dependency_get_external_release_dependencies(1910022));'
array(15) {
  [2321111]=>
  array(4) {
    ["uri"]=>
    string(9) "votingapi"
    ["version"]=>
    string(8) "7.x-2.12"
    ["tag"]=>
    string(8) "7.x-2.12"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2437885]=>
  array(4) {
    ["uri"]=>
    string(6) "entity"
    ["version"]=>
    string(7) "7.x-1.6"
    ["tag"]=>
    string(7) "7.x-1.6"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2592759]=>
  array(4) {
    ["uri"]=>
    string(4) "flag"
    ["version"]=>
    string(7) "7.x-3.7"
    ["tag"]=>
    string(7) "7.x-3.7"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2453319]=>
  array(4) {
    ["uri"]=>
    string(5) "rules"
    ["version"]=>
    string(7) "7.x-2.9"
    ["tag"]=>
    string(7) "7.x-2.9"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2443407]=>
  array(4) {
    ["uri"]=>
    string(5) "token"
    ["version"]=>
    string(7) "7.x-1.6"
    ["tag"]=>
    string(7) "7.x-1.6"
    ["dependency_type"]=>
    string(1) "1"
  }
  [2366815]=>
  array(4) {
    ["uri"]=>
    string(6) "expire"
    ["version"]=>
    string(11) "7.x-2.0-rc4"
    ["tag"]=>
    string(11) "7.x-2.0-rc4"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2531428]=>
  array(4) {
    ["uri"]=>
    string(5) "feeds"
    ["version"]=>
    string(13) "7.x-2.0-beta1"
    ["tag"]=>
    string(13) "7.x-2.0-beta1"
    ["dependency_type"]=>
    string(1) "1"
  }
  [2454883]=>
  array(4) {
    ["uri"]=>
    string(6) "ctools"
    ["version"]=>
    string(7) "7.x-1.7"
    ["tag"]=>
    string(7) "7.x-1.7"
    ["dependency_type"]=>
    string(1) "0"
  }
  [1566814]=>
  array(4) {
    ["uri"]=>
    string(13) "job_scheduler"
    ["version"]=>
    string(14) "7.x-2.0-alpha3"
    ["tag"]=>
    string(14) "7.x-2.0-alpha3"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2549341]=>
  array(4) {
    ["uri"]=>
    string(8) "relation"
    ["version"]=>
    string(7) "7.x-1.0"
    ["tag"]=>
    string(7) "7.x-1.0"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2480259]=>
  array(4) {
    ["uri"]=>
    string(5) "views"
    ["version"]=>
    string(8) "7.x-3.11"
    ["tag"]=>
    string(8) "7.x-3.11"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2243019]=>
  array(4) {
    ["uri"]=>
    string(11) "cacheobject"
    ["version"]=>
    string(13) "7.x-1.0-beta1"
    ["tag"]=>
    string(13) "7.x-1.0-beta1"
    ["dependency_type"]=>
    string(1) "1"
  }
  [2408705]=>
  array(4) {
    ["uri"]=>
    string(8) "commerce"
    ["version"]=>
    string(8) "7.x-1.11"
    ["tag"]=>
    string(8) "7.x-1.11"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2479989]=>
  array(4) {
    ["uri"]=>
    string(13) "flightcontrol"
    ["version"]=>
    string(11) "7.x-1.x-dev"
    ["tag"]=>
    string(7) "7.x-1.x"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2415993]=>
  array(4) {
    ["uri"]=>
    string(6) "panels"
    ["version"]=>
    string(7) "7.x-3.5"
    ["tag"]=>
    string(7) "7.x-3.5"
    ["dependency_type"]=>
    string(1) "0"
  }
}

It has commerce listed as a dependency, but does not list any of commerce's dependencies

drush ev 'var_dump(project_dependency_get_external_release_dependencies(2408705));'
array(5) {
  [2437885]=>
  array(4) {
    ["uri"]=>
    string(6) "entity"
    ["version"]=>
    string(7) "7.x-1.6"
    ["tag"]=>
    string(7) "7.x-1.6"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2453319]=>
  array(4) {
    ["uri"]=>
    string(5) "rules"
    ["version"]=>
    string(7) "7.x-2.9"
    ["tag"]=>
    string(7) "7.x-2.9"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2480259]=>
  array(4) {
    ["uri"]=>
    string(5) "views"
    ["version"]=>
    string(8) "7.x-3.11"
    ["tag"]=>
    string(8) "7.x-3.11"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2454883]=>
  array(4) {
    ["uri"]=>
    string(6) "ctools"
    ["version"]=>
    string(7) "7.x-1.7"
    ["tag"]=>
    string(7) "7.x-1.7"
    ["dependency_type"]=>
    string(1) "0"
  }
  [2582669]=>
  array(4) {
    ["uri"]=>
    string(12) "addressfield"
    ["version"]=>
    string(7) "7.x-1.2"
    ["tag"]=>
    string(7) "7.x-1.2"
    ["dependency_type"]=>
    string(1) "0"
  }
}

ugh. I did *not* delete that screenshot. wth.

Generally I would track this down further, but I think the maximum recursion level needs to be raised and this is as good an excuse any to do it. I committed the patch and created release 7.x-1.0-beta6 with the increased limit. It will need to be installed on Drupal.org.

I marked the issue as needs review because we still need to check if this fixes this particular dependency issue.

I uploaded the screenshot again although it really is not essential.

This has been deployed to Drupal.org.

I have requested an update of my dev so I can debug this. It will take a few days of setting things up and tracing through to find the problem. If you are in a hurry you can explicitly add the dependency for addressfield.

Well, it is slow work getting everything set up and tracing through this but here is a start. From authcache_page_manager.info you have

dependencies[] = authcache_p13n
dependencies[] = page_manager

Since you requested the module page_manager, a search of releases with modules named page_manager comes up with flightcontrol. Module names are not unique and this has been a problem that keeps repeating. What is especially insidious is everything can be working fine and then a completely new release with an identical module name shows up and despite no changes to your project the dependencies are messed up. That is the bad news. The good news is that I have fought for well over a year to provide a way to avoid this problem and 12 days ago the backport to Drupal 7 was committed #2205271: Project namespace for dependencies. So you should be able to change your two lines specifying the dependencies to (assuming you want the page_manager module from the page_manager project):

dependencies[] = authcache_p13n
dependencies[] = page_manager:page_manager

As an aside, I thought I had the code working so it would check for project names with the same name as the module first so I am a bit surprised it is picking up flightcontrol but it could be that the page_manager project does not have a recommended release.

I can add flightcontrol to https://bitbucket.org/drupalorg-infrastructure/drupal.org/src/4e1fac2316...

That is deployed, and it looks like that change will take effect immediately. Any affected modules that have not updated their dependencies with namespaces should also be fixed.

@drumm: Thanks but flightcontrol is a side issue. I tracked down the main issue and will be committing a patch shortly. After it is deployed it will still need to be reviewed to make sure the problem is fixed. It is looking okay on my test system except for the flightcontrol problem.

New release is created (7.x-1.x-beta7) with the change. With the change the maximum recursion level is 5 although limiting it to 4 did not change the dependencies. But it is probably a good thing that the maximum recursion level has been increased.

7.x-1.x-beta7 is now deployed to Drupal.org.

Thanks a ton Trobey!