Run tests with GitLab CI

I've been working on the various settings for how we want to support gitlab CI, and have now got it configured to do the following:

1. Merge requests will run a pipeline in the upstream project, against the merged result

https://git.drupalcode.org/project/once/-/merge_requests/4 shows the text "Merged result pipeline #30" -> i.e. its my changes on the forks feature branch, merged in with upstream.

2. Secrets, like the NPM push key variable will only be available to protected branches and tags, and as such are not exposed to contributors, only maintainers.

No jobs will run in forks themselves, we will only support testing against MR's against the upstream -> this is primarily to contain testing costs and ensure we're not doing extraneous testing that does not necessarily give valid results. This is different than we had discussed before, and we can revisit this if theres a workflow consideration we havent accounted for.

In general what this means is that we might run into situations where, say a linter finds an issue with a particular line of code, but that line is now in a different place because the parent project has changed the file. E.g. line 10 in your fork is bad, but after the merge with upstream it gets reported as line 12. So, ideally to avoid situations like that devs will merge the parent project into their fork before testing (it's still going to happen where they forget, of course, but hopefully we'll all get into the habit of keeping up with HEAD in our forks.)

That is sort of the same thing as running the pipeline on the merged result, it's just that the merge is happening in the pipeline instead of on the fork branch.

From github's docs page (https://docs.github.com/en/github/administering-a-repository/about-prote...) They call out

More builds may be required

Which is the scenario that we're trying to avoid.

Gitlab does not have that same exact setting that github has, but I think the settings we're opting to use achieve the same result:

For contributors, we want their changes to run against HEAD, this ensures that whatever change they're working on has to keep up with outside changes as early as possible. The disadvantage is if they let their fork get stale, line number reporting might be off, and might require an extra test after they update their fork.

For maintainers, we want them to be able to confidently commit MR's without having a broken HEAD. There will usually be a delay between when a contribution is tested, and 'ready' to when a maintainer can merge it in. Our current mechanism for handling this is RTBC re-testing.

But gitlab gives us a better way to do this: Merge Trains.

When a maintainer is ready to commit an MR, they push the "add to merge train" button, which re-runs the test against HEAD, and if it passes, performs the merge. This is essentially on-demand RTBC retesting, which saves us the nonsense of retesting patches that are kicking around on a daily basis.

The thing that makes merge trains interesting is if you have long tests, you dont have to wait for the tests to pass to continue a commit spree.

Example: if a maintainer had 3 MR's they wanted to accept, they hit "add to merge train" on all of them and it kicks off one build for each,

HEAD + A
HEAD + A + B
HEAD + A + B + C

And then successively merges them in. If A+B fails for some reason, then it stops all builds with B in it, and retries with A+C and moves on.

There is also an option for the maintainer to just skip the merge train altogether and merge in without an additional test (if, say, the MR was already tested against HEAD, and nothing has happened to HEAD since the MR was tested)

Does that clarify things? Is there some other disadvantage Im not seeing?

I .. thought I already had, but turns out I just enabled it on the dummy project I was experimenting on.

It's on now.

Ah, sorry didn't see this earlier. I don't actually know the answer to that question. I'll coordinate with Neil tomorrow and find out. If it doesn't do that, we will make it do that.

Currently there isn't any connectivity between gitlabCI and the issues, so thats something we'll need to add. So the question in #18 is that it will still show green if it can merge, even if the tests are failing, and that a failed merge train will not alter what is displayed/the status in the issue queue.

That looks great!

Let's get that in and start building on it if there are corrections.