Problem/Motivation

Today, the default theme when writing tests is classy, which is likely deprecated in the future #3050378: [meta] Replace Classy with a starterkit theme. For this, and various other reasons Classy is decoupled from the testing infrastructure: #2352949: Deprecate using Classy as the default theme for the 'testing' profile. Until now, we assume there isn't a default value in Drupal 9 and it is required for all test cases not specifying installation profile to specify which theme they use in their tests. However, “no default” is just one of the possible choices.

The 3 possible choices:

Theme Pros Cons When best fit?
None
  • Explicit: no surprises, full control
  • Higher chance the test authors choose the most appropriate theme for the test case
 Boilerplate: every test must specify it N/A
stark No boilerplate Higher probability of breaking between minor releases
  • Contrib / custom tests that DO NOT depend on core markup
  • Core tests because they should be updated at the time when markup changes occur
stable9 No boilerplate Higher probability of breaking between major releases
  • Contrib / custom tests that DO depend on core markup
  • Contrib / custom tests that are not using markup for testing

(In Drupal 9, the default would be stable9, in Drupal 10, it would be stable10, and so on.)

Default: opt-in vs opt-out?

There might be a way to be both explicit and provide a default value. We could set a default value that is set only if the test case has explicitly opted into the default. One option would be provide TESTING_DEFAULT_THEME constant which would allow test cases to use the current default value. This would remove the need to manually update testing theme on major releases.

Drupal 9
const TESTING_DEFAULT_THEME = ‘stable9’;
const CORE_TESTING_DEFAULT_THEME = ‘stark’;
Drupal 10
const TESTING_DEFAULT_THEME = ‘stable10’;
const CORE_TESTING_DEFAULT_THEME = ‘stark’;

Proposed resolution

TBD

Remaining tasks

  • Decide what is the default theme for testing in Drupal 9 if any
  • Decide if the default theme is set explicitly or implicitly

User interface changes

None

API changes

TBD

Release notes snippet

Comments

lauriii created an issue. See original summary.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Issue tags: +DX (Developer Experience), +Drupal 9, +DTT issue
wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

#2352949: Deprecate using Classy as the default theme for the 'testing' profile landed. Let's continue here!

xjm’s picture

xjm
she/her
Primary language English
commented
Parent issue: » #2659890: [Policy] [Plan] Drupal 9 and 10 markup and CSS backwards compatibility
xjm’s picture

xjm
she/her
Primary language English
commented

Of these, I think Stark is the best option.

  • Updating tests for an individual patch that breaks the markup those tests were expecting is basically the same as what we already do when we change a UI element (e.g., a button label or etc.).
  • If we go with stableN, then we have a potentially huge pile of technical debt every major release.
  • If we require a theme to be specified each time, we're putting a decision on the test author that has a 99% chance of being irrelevant to what they're actually trying to test, so we'll get bad decisions as well as a collection of random things.

So my suggestion is to use stark, see how many tests already pass with that, and then stick classy on whatever still fails, with a @todo to #3083275: [meta] Update tests that rely on Classy to not rely on it anymore.

xjm’s picture

xjm
she/her
Primary language English
commented

Looks like #3082655: Specify the $defaultTheme property in all functional tests is already taking an approach consistent with that proposal.

lauriii’s picture

lauriii
he/him
Primary language Finnish
Location Finland
commented

I agree that using stark for testing makes sense for core in all test cases except ones specifically testing other themes, therefore it should be the default theme for all core tests. However, this issue was opened because we weren't quite not convinced that using stark for contrib tests would make sense. Using stark as the default testing theme for contrib would add burden to minor release upgrades, as well as potentially disincentivize writing tests.

If we go with stableN, then we have a potentially huge pile of technical debt every major release.

+1, we definitely want to avoid this in core.

If we require a theme to be specified each time, we're putting a decision on the test author that has a 99% chance of being irrelevant to what they're actually trying to test, so we'll get bad decisions as well as a collection of random things.

Is this statement against requiring test authors to explicitly opt-in for using a default theme?

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

#5: "@todo to #3083275" looks like it's not the issue you wanted to link? :)

xjm’s picture

xjm
she/her
Primary language English
commented

@Wim Leers, no, it definitely is. Edit: What my comment says is what I mean. Yours is a different number. :)

Every time we set the profile to classy to get the test to pass, we're adding technical debt that needs to be cleaned up before we remove Classy. So each time we do that, there should be a @todo to the meta for removing said coupling.

It would be nice for both core and contrib to have a non-boilerplate option. Contrib will after all just copy core and end up setting stark even if we make the default value stable9 to be "nice" to them. Also, we're still adding major technical debt for contrib if we do this, because we're still implicitly encouraging them to couple their tests to markup and then have to update to the new stable all at once each major.

We should summon actual contrib maintainers for input also. I suggest Berdir as I know he's run into the same problem of tests starting to fail when we change button labels or such.

Version: 9.x-dev » 9.0.x-dev

The 9.0.x branch will open for development soon, and the placeholder 9.x branch should no longer be used. Only issues that require a new major version should be filed against 9.0.x (for example, removing deprecated code or updating dependency major versions). New developments and disruptive changes that are allowed in a minor version should be filed against 8.9.x, and significant new features will be moved to 9.1.x at committer discretion. For more information see the Allowed changes during the Drupal 8 and 9 release cycles and the Drupal 9.0.0 release plan.

xjm’s picture

xjm
she/her
Primary language English
commented

What are the next steps here? Do we need to implement this? Is it D9-only?

dww’s picture

dww
we/he/they
commented

I think we've decided. Stark seems like the best (default) choice, and we should use that whenever feasible. I think #3083275: [meta] Update tests that rely on Classy to not rely on it anymore is where all the work needs to happen now. I vote to mark this fixed and move on to #3083275...

jaykandari’s picture

jaykandari
Primary language Hindi
commented

Subscribe

Version: 9.0.x-dev » 9.1.x-dev

Drupal 9.0.10 was released on December 3, 2020 and is the final full bugfix release for the Drupal 9.0.x series. Drupal 9.0.x will not receive any further development aside from security fixes. Sites should update to Drupal 9.1.0 to continue receiving regular bugfixes.

Drupal-9-only bug reports should be targeted for the 9.1.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.2.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.1.x-dev » 9.3.x-dev

Drupal 9.1.10 (June 4, 2021) and Drupal 9.2.10 (November 24, 2021) were the last bugfix releases of those minor version series. Drupal 9 bug reports should be targeted for the 9.3.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

quietone commented
Status: Active » Fixed
Issue tags: -

I agree with #12 that this is fixed and the change record was published in 2019. The implementation was moved to other issue.

I've updated credit and moving straight to Fixed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.