Simpletest Example test should support DrupalCI

I think this is something that will need to be looked at later, but for now we can simulate the same sort of behavior as qa.d.o.

Trying to use a symlink in pretty much the same manner as qa.d.o does still gave the same sort of results.

When we remove the check that ran the failing test we ran into another issue.
Drupal\cache_example\Tests\CacheExampleTestCase 25 passes 17 fails 7 exceptions
Allowed memory size of 419430400 bytes exhausted (tried to allocate 493 bytes)
(this is already a raised number, where we have set the number to 320M)
The original issue I'm not sure why we weren't skipping those tests like on qa.d.o, but that is for another day.

And the second issue is most likely due to a PHP minor version difference
qa.d.o PHP 5.5.27
dci web-5.5 PHP 5.5.23

runningOnTestbot() is a method defined within the simpletest example: http://cgit.drupalcode.org/examples/tree/simpletest_example/src/Tests/Si...

So what's needed is a way to implement that method.

Is there a known environmental variable that can be checked?

The reason we're checking is so that we can show a failing test to the user.

We can modify the check to something like this
return (file_exists("../checkout") || file_exists("../site_under_test") || file_exists("../html/checkout"));

Though even with that check adjusted there are still failures.

Right, it'd be great if there were a way to say

if (getenv('DRUPALCI')) {
  etc.
}

Is there an env variable that's always set under DrupalCI?

This gets us into another conversation. Since devs can run DrupalCI from their local environment, should the test be skipped if they're using DrupalCI?

Yes. :-)

The point is to show what a failed test looks like, but that's secondary to being able to use automated tests for the project.

Also, it's nice to demonstrate how to check for the testbot. Devs might set up special requirements for the testbot.

Based on discussion with mixologic, using environmental variable.

So what happened here is that simpletest_example was modified so it always returns TRUE for runningOnTestbot().

This will be changed back once the new testbot code makes it to production, where we can use the DRUPALCI environmental variable to check whether we're running under the testbot.

This same issue is currently breaking the 7.x-1.x branch of Examples.

New patch which uses the environmental variable.

Derp.

:-)

OK, if this fails then we know the testbot code with the environmental variable still hasn't deployed.

Let's see if the environment variable has been added, yet.

I'm going to update the docs to explain what is happening here, and how someone running through the tutorial can actually get the failures that the tutorial says they'll get. When this is fixed and backported, the tutorial should be updated as well.

Follow-up in the testbot issue queue: #2860941: Expose an environmental variable so that SUT can change behavior

Really should be postponed on that issue.

Can't find SimpleTestExampleTest in the codebase of 4.0.x. And drupalci.yml was removed from the codebase.

No activity for over 6 years. I am closing this. Please feel free to reopen if necessary.

Thanks!