Problem/Motivation

Since the introduction of Simpletest, we've been plagued with issues around keeping the parent (testing) process in sync with changes in the child (tested) processes.

Even now $this->container \Drupal::container() and the container in the child processes aren't necessarily the same thing.

Proposed resolution

Refactor tests to be either pure integration tests with no browser requests, or pure browser requests with no API access. You'd be fine to use the API during setup since that all happens before the child site is requested, but once you start making browser requests, should stick to that (which means losing some shortcuts for assertions generally).

Remaining tasks

Decide if this is the course of action we want to take, then start identifying and re-factoring problematic tests.

User interface changes

API changes

Data model changes

Comments

catch created an issue. See original summary.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

I think this would be great for sanity - it's going to be hard to implement because this is not simply about the container it is about the test runner not have any access to any drupal methods.

neclimdul’s picture

neclimdul
He/Him
commented

Man oh man yes. Separation of the testing environment from the site being tested has always been a dream.

However... "only" implies we can't check the database to confirm values are stored correctly. It would require quite a bit of effort to see that a hidden field the logged in user doesn't have access to is stored correctly. Or that some computed internal value is correctly triggered off a form submit. Probably other cases but those comes to mind as things we might want to consider.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

geek-merlin’s picture

geek-merlin
Primary language German
Location Freiburg, Germany
commented

We might want to use json:api and other rpc methods to replace the api calls.

mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented
mile23’s picture

mile23
Primary language English
Location Seattle, WA
commented

We're working on a more formal black-box testing type here #3031379: Add a new test type to do real update testing

It will be easier to make that work than to try to add the formality to the existing BTB type.

Also: No idea why I just made three comments by loading this page.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

joachim’s picture

joachim commented
Status: Active » Postponed (maintainer needs more info)
Issue tags: +Needs issue summary update

> Since the introduction of Simpletest

Is this issue still relevant now we're using PHPUnit?

catch’s picture

catch
he/him
Primary language English
commented
Status: Postponed (maintainer needs more info) » Closed (duplicate)
Related issues: +#2066993: Use magic methods to sync container property to \Drupal::getContainer in functional tests

It's still relevant, we have exactly the same issues between the parent child and the child site with functional tests. However, this is now a duplicate of #2066993: Use magic methods to sync container property to \Drupal::getContainer in functional tests.