For testing token features in Google Analytics I need to enable token module. I tried it this way, but the module install fails:

parent::setUp('googleanalytics', 'token');

See http://qa.drupal.org/pifr/test/353373 for details, please. I guess the reason is that token module is not a required module for Google Analytics and only optional, but I cannot change this. How can I test the token stuff now with the d.o testbot? On my local box all works well, but on d.o the testbot does not download/install the token module.

How can I solve this issue without excluding the tests?

Comments

hass’s picture

jthorson’s picture

I believe that the current workaround for this is to create a 'hidden' test sub-module which does depend on token, and (optionally?) move the related feature tests there ... but the dependencies will then need to be rebuilt, which a few projects have been having troubles with. Dependencies are supposed to be rebuilt every 12 hours, but may only be happening upon creation of a release. Since creating a release to 'test' a workaround isn't ideal, if the dependencies aren't rebuilt 12 hours after the commit introducing the submodule, post back when ready and we can try triggering a manual dependency rebuild.

hass’s picture

Component: malfunctioning testbot » Documentation
Category: bug » support
Status: Active » Fixed
hass’s picture

Category: support » bug
Status: Fixed » Active

Ah, no - this only prevent the failures and does not execute the tests now. See http://qa.drupal.org/pifr/test/353923

I don't get it. I cannot create a release as the tests need to pass first to get a patch committed.

hass’s picture

Component: Documentation » malfunctioning testbot
jthorson’s picture

Category: bug » support

1. Create a sub-module in your project with a hard dependency on token.
2. Move the token-specific tests into the the sub-module.
3. Commit to repo.
4. If the test isn't passing 12 hours later (i.e. dependencies haven't been rebuilt), post here asking to have dependencies rebuilt manually.

hass’s picture

Category: support » bug

This bug was already reported in #1670936: Testbot does not detect/download additional dependencies.

Sorry, but I do not need a submodule in the GA module. Why should I add one? For sure, not for the test bot only. The bot need to parse the .test files and extract all module names from setUp('googleanalytics', 'token') and download these modules and the issue is solved. This is a bug in the bot, not my module nor my tests.

rfay’s picture

The way to get modules downloaded is through dependencies in info files. They are calculated recursively, as they would need to be.

The fact is, your module (in its tests) has a dependency on token. That's the fact. So you can either change the test, or add a dependency. The simple workaround is to create a "mock module" in the tests subdirectory that has a dependency on token.

jthorson’s picture

Category: bug » feature
Status: Active » Fixed

Sorry, but I do not need a submodule in the GA module. Why should I add one?

Your initial post was "How can I solve this issue without excluding the tests?"

Question asked, and answered.

If you don't like the answer, then let me provide an alternate one ... Code the feature (they way you want it to work) for the testbot, because today, what you are asking for does not exist. Having the bot parse names from the test files is a change in functionality, and therefore a 'feature request', not a bug.

hass’s picture

Status: Fixed » Active

Ok, than it's a feature, but it looks like the feature is not yet implemented and the case cannot have a status of "fixed".

jthorson’s picture

Component: malfunctioning testbot » Code

That, I can certainly agree to. :)

rfay’s picture

Component: Code » malfunctioning testbot

I really have no idea how this could be implemented.

Note that module_enable() might be called on any string, which could be a part of some undetermined project.

The testbot has to figure out how to check out the projects that are needed. It does that by using the dependency information which is explicitly given.

Even parsing all the code of Drupal + all modules would not be adequate to solve this problem, since the module we want to enable may have dependencies itself, and it may not be a part of a project by the same name.

rfay’s picture

Component: malfunctioning testbot » Code

x-post

hass’s picture

Why not just run a regex on the dependency key and extract the array items? We could solve 98,7%. We could also write a rule that this needs to be the raw module strings in quotes...

jthorson’s picture

The problem is that only gives us the first level of dependencies. For a project that depends on views, the tests will still fail because of the unwritten dependency that views has on ctools. Forcing module authors to specify the complete dependency chain would be laborous and unmanagable.

Alternatively, if you wanted to recursively parse and calculate these after each individual checkout, it would require a complete refactoring of the PIFR process.

hass’s picture

Until today I thought I need to specify all modules... :-) I guess it's better than not being able to run the tests.

rfay’s picture

You know, I forgot - the correct and excellent answer for this is test_dependencies, as proposed in #698932: "test_dependencies" for dependencies / integration tests - it would allow listing in the info file modules that are not required for the module to run, but which could be used by it for tests or whatever.

rfay’s picture

And this is now deployed, so you can use test_dependencies[] = to solve this problem. http://qa.drupal.org/node/198

hass’s picture

Status: Needs work » Active

Cool. Thx. Is it correct that the version in the dependency does not follow the same rule as core dependency? Normally you use (>1.0), but test_dep use syntax with core version (>7.x-1.0) that we already know from core key... Can we remove the core version here? Just for consistency... I guess I'm not testing D8 with D7 :-)

hass’s picture

Status: Active » Needs work

Seems not working, see #1833154: Enable optional token tests on d.o please. Custom variables test are all skipped.

rfay’s picture

Status: Active » Needs work

Please note that:

1. Dependencies are not calculated on patches, ever.
2. Dependencies are generated at release generation time. That means for a dev release, every 12 hours if a commit has happened.
3. We can rebuild dependencies manually on request.

http://drupal.org/project/testbot has some FAQs

rfay’s picture

Status: Needs work » Fixed

Marking fixed until further notice.

rfay’s picture

@hass, #20: The dependency rules should be exactly the ones for core, as documented at http://drupal.org/node/542202#dependencies. They are copied from Drupal 7's work. Since this is currently a D6 module...

hass’s picture

Component: Code » Documentation
Status: Fixed » Needs work

http://drupal.org/node/542202#dependencies needs work:

test_dependencies[] = autoload (>7.x-1.5)

Core version should not specified per my understanding.

You can optionally specify the core version number as well:

dependencies[] = exampleapi (>7.x-1.5)

What is the use case for the "core" version? Mixing D8 and D7 modules? I'm not aware that core supports this.

hass’s picture

Re #22: This is really bad. I have no idea why this limitation exists, but it should be removed. Otherwise attaching a patch to a queue makes no sense as it is not tested. This makes all tests useless as the next needs review patch in queue will fail in automated testing and this is wrong.

If I remember correctly - the idea of tests was - do not let a broken patch in that fails testing...

rfay’s picture

Status: Needs work » Fixed

@hass, #25, please open a core issue or a documentation issue and reference it here. This implementation is just the Drupal implementation.

On #26, please figure out a way to calculate dependencies on patches and provide a patch to PIFT that would do it. But don't break anything.

hass’s picture

@rfay: Does it also work for D6 branches? Looks like GA 7 has been tested properly tonight.

Before I commit this change to D6, I'd like to know.

rfay’s picture

@hass, We just used the D7 code. I don't know the history. You could do a git blame on that section of code to find out what issue changed it and see if there were API changes.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

a