Switching between minor versions of PHP

Pushed branch 2568299-use-DCI_PHPInterpreter to the repo

Needs review for the PHP Interpreter change, which is step 1.

Then should go back to active for the actual switching versions functionality.

Back to 'needs review' for the actual implementation (versus earlier 'pre-requisite' work).

Up-to-date with dev as of March 12th ...

Trying to rebase for dev.. It looks like this removes the testcommand step and adds the run-tests.sh command back to the command step. Also removes the phantomJS work.

It also introduces DCI_PHPInterpreter which needs some coverage from functional tests.

So it needs a functional test where you run a test specifying a minimum version using DCI_PHPInterpreter, and another test with another minimum version. Probably using php 5.5.9 and 5.5.MAX since that's the minimum for Drupal 8.1.x.

Trying to rebase for dev.. It looks like this removes the testcommand step and adds the run-tests.sh command back to the command step. Also removes the phantomJS work.

Nope ... you must have fouled up your rebase:
phantomJS: http://cgit.drupalcode.org/drupalci_testbot/tree/src/DrupalCI/Plugin/Job...
testcommand: http://cgit.drupalcode.org/drupalci_testbot/tree/src/DrupalCI/Plugin/Job...

It also introduces DCI_PHPInterpreter which needs some coverage from functional tests.

Suggested by, manually demonstrated to, and RTBC'd by Archie in Barcelona, as per comment #6. Adding a functional end-to-end test run to prove that abstraction of the php binary via variable substitution works is severe overkill ... the test run either finds PHP or it doesn't. There is no risk of 'subtle' breakage here.

So it needs a functional test where you run a test specifying a minimum version using DCI_PHPInterpreter, and another test with another minimum version. Probably using php 5.5.9 and 5.5.MAX since that's the minimum for Drupal 8.1.x.

The only 'functional test' we can do right now is an end-to-end test run. If we require a new automated end-to-end test run for every variable substitution scenario we come up with, then it won't be long before it takes six hours to run the test suite. :/ In any case, see the attached files (search for php -v).

Updated branch (again).

Adding a functional end-to-end test run to prove that abstraction of the php binary via variable substitution works is severe overkill ... the test run either finds PHP or it doesn't. There is no risk of 'subtle' breakage here.

That's not what I mean.

You have to prove that it works, and the way to do that is to write a test so that we can continue to prove that it works when we change it and change things around it.

There's this line in the definition: - %DCI_PHPInterpreter% -v so you can write a functional test that looks for the PHP version in the command display and verify that it changed.

I proved the DCI_PHPInterpreter addition (along with the rest of the changes) works via the attached log files. I'm opposed to adding a new functional test for every variable substitution, because each of those requires a new test run ... quickly bloating the time it takes to execute the full test suite.

The counter that we need these automated so that we can prevent 'hidden' regressions doesn't hold a lot of weight here, specifically; the only potential break with the DCI_PHPInterpreter substitution commit here (which is what the quoted comment was directed at) is 'PHP not found'. Yes, this substitution needs to be tested, along with every other job definition plugin ... but only to the extent that it proves the variable substitution occurs - a new 'functional' test for every one of these definition preprocessor plugins is what I'm saying is overkill.

In the absence of a framework that enables the definition pre-processor plugin tests today, we should not be encouraging adding minutes to our test suite execution time for every two-line patch; when manual proof has been provided and the risk of 'subtle' regressions is, as it is in this case, essentially zero. (Again, I'm talking specifically about the addition of DCI_PHPInterpreter here.)

Clarifying the title.

Moved to DrupalCI Environments.

We've removed both PHPInterpreter and phpenv, and have individual containers per php version.