Maintain popular CI presets to improve developer experience

Not sure what it is that you're suggesting here, but I *think* you might be saying "lets use TravisCI or CircleCI instead of DrupalCI".

So, some background. A long while back, when DrupalCI was being designed, Drupal's test suite was so big that *none* of the free services were capable of running the entire thing before hitting their timeout walls. The sheer magnitude of the breadth, scope, and depth of the amount of testing that we run ended up being too big for any of those services. And our test suite has only grown bigger. Maybe the services are more performant, but my experience so far is that the larger contrib modules have started moving *back* to drupalci from Travis, because it was considerably faster. That leads me to think that running our test suite on any of the cloud services would likely never be able to qualify for their 'free' layer, and would likely be extremely expensive if we had to use any of their paid services.

Setting up locally:

its not linux specific, it uses virtualbox: https://www.drupal.org/drupalorg/docs/drupal-ci/running-drupalci-locally, but yes, testing happens on linux environments. There hasn't been any demand from anybody to be able to run the test suite on windows or OSX.

Reduce testbot fails noise

: Not sure what you are referring to here. Do you mean when somebody submits a bad patch and it notifies everybody on the issue? Or something else? This doesnt seem to have anything to do with using a cloud service, and is really just a notification preference feature request. If we're using the drupal.org issues then it wouldnt matter if drupalci is the service or if some other cloud is the service. A fail would notify everybody

Let the public services

Again, we dont run just a couple of tests. We'll run out of goodwill really, really quickly with the volume of tests we need to run. AFAIK from the JS maintainers, we dont really *need* to test anything against the extra Node branches, because we aren't running node as part of drupal. If you're suggesting that we add some build tests to validate that its easy for developers to get up and running regardless of what local platform they're trying to develop on, well, thats an entirely new feature, and an entirely new type of testing (build tests, vs product functionality tests).

Make it easy to run single test rather than full test suite.

You can do this now, but its poorly documented. You patch run-tests.sh to set which tests it runs. In the future, there will be a drupalci.yml build file that we can use to specify the same things you can specify on the command line for run-tests or phpunit etc.

Pretty much, my suggestion is that this is Close (wont fix). And that some of these things get separated out into other issues and we could talk about them individually. "Use a bunch of cloud services" adds a whole host of other problems that we probably do not want.

Finally,
There're many rooms can be improved into DrupalCI. As simple as let contributor to pick a test to run first manually, then it helps. (Most developers able to predict what to test and what may fail.) I know there's an issue in DrupalCI project issue thread for this part (find a way to run it automatically). But for a long time, no such function in d.org. It must have a good reason cannot / not to do that. DrupalCI team has a different vision & problem to solve.

This is a brilliant idea!

I totally agree we should make it as easy as possible to run the tests properly locally. In my world the local testrunner would be good enough, if the setup would be as easy as possible as in:

• Download Drupal core
• Work on stuff
• composer run localtest (filename-to-test.php)
• This internally asks you to install vagrant, downloads drupalci_testbot and sets up everything needed

On top of that I also agree it would be nice to enable running tests on travis/circleci ... but well its tricky because you cannot run the full testsuite on those.

Maintain popular CI presets to improve developer experience

First of all, this is an excellent idea. You have no idea how much time I have wasted on this.

But I don't think it should be maintained in core. Someone can create a repo and we can all keep on improving it.