Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Problem/Motivation
Currently, testbot dependencies have more or less the same logic as e.g. a drush dl . It fetches a non-dev release if one is available, even if that is months and years old. That's different to core, where contrib tests are always run against the latest 8.0.x branch (Although that makes me wonder how that's going to work with 8.1.x now, I think we need an option to run test against both the current supported and the next minor version so we can prepare for changes there)
This was especially painful during 8.x development when you had dependencies that already had alpha releases, combined with some weird bugs where testbot dependencies are stuck and don't update for weeks and months after a new release happened. But it's also a problem just between contrib modules, when trying to integrate with new features in another module that aren't yet available as a full release.
Proposed resolution
Change dependency logic to always check out the branch and not a release tag of the latest support major version, unless something else is explicitly specified.
Remaining tasks
Not sure where this needs to change exactly.
Comments
Comment #2
joshuamiWe have this in the plan at #2268449: Run contrib module branch tests against both dev and latest stable core branches
We are also planning to make it possible for someone to test against a feature branch. That is going to require some extra thought for the UI.
Comment #3
BerdirThanks. That issue doesn't mention dependencies of other contrib modules at all however, it just covers my thinking-out-loud part about the core branch but that's not what this issue is about. I don't think the other issue will cover contrib dependencies, that's a completely different topic IMHO. So re-opening for confirmation, please close again if you really think that other issue will do all that.
See https://dispatcher.drupalci.org/job/default/39016/console for example.
That downloads a whole bunch of dependencies. And among many others, it downloads currency 8.x-3.0-rc1 and search_api-8.x-1.0-alpha9. The latter is from 2015-Sep-10, so more than two month old and in the current phase of 8.x contrib modules, it just doesn't make sense to test against something that's so old, not if there is newer code to test against.
Countless times during the 8.0.x phase, I've had broken tests because my tests were using an old alpha version of that project.
I'm suggesting that we should instead test against currency 8.x-3.x-dev and search_api 8.x-1.x-dev.
Comment #4
joshuamiThanks for the clarification!
I chatted with @mixologic a bit about the request and what the difference between the two approaches to testing will mean.
In short, we need both tests against dependent projects that are in development and that are stable. Contrib developers need a means to pick which times of tests are appropriate. One challenge will be limiting the tests to those with the greatest value that we can actually afford to provide. This feature will either take a ton of UI work or a really good integration with Composer. Better integration with composer is probably the more sustainable approach.
We have some Composer work that is starting up right now. I feel like the end result will affect how we can go about running these tests.
So, yes. Let's leave this active and build out the requirement a bit more so we know what we are building.
Comment #5
BerdirBump.
We're now even using 8.1.x by default for testing contrib.. but possibly months old tagged releases of other dependencies. That combination just doesn't make any sense? If you take the latest core then you have to take the latest of all contrib dependencies too. 8.1.x will make changes that will at least change tests, admin UI's and so on. tagged releases will on the other hand not be compatible with that.
Extending it to support testing specific versions just like we support explicitly testing against 8.0.x would be nice, but the default has to be to use the latest dev snapshot. That's really the only sane option IMHO :)
Comment #6
BerdirSee #2653630: Fix failing tests for another example, the pathauto tests are currently failing, and the only way to fix that is doing a new release of token and hoping that it will get picked up soon (which doesn't always work...)
Comment #7
MixologicComment #8
MixologicComment #9
MixologicDrupalCI was deployed today, and part of that deployment is that drupalci uses the composer facade to construct its codebase. As Joshuami mentioned in #4, this is now going to allow us to choose whatever versions of dependencies are required for testing purposes (combined with the ability to select which core branch a project is tested against).
All one needs to do is add a composer.json to your project if you wish to have fine grained control over which versions of dependencies get pulled in when the project is tested and built. Patches that modify composer.json will be detected (*not info.yml), and those changes will also be part of the build that is tested.
Happy Festivus.
Comment #10
Mixologic