Problem/Motivation

We want to get rid of simpletest.module in Drupal 9. We want to prepare the latest Drupal 8 development branch as much as possible before opening the Drupal 9 development branch (see #2608062: [policy, no patch] Pre-requisites for opening Drupal 9). Therefore we need to convert all web tests to BrowserTestBase or UnitTestBase or KernelTestBase and deprecate WebTestBase.

Proposed resolution

1) Convert a small part of Drupal core web tests to Browser tests to get them battle tested. See all the conversion child issues that have been opened here before 2016-Sept-01.
2) Implement a backward compatibility layer as AssertLegacyTrait used by BrowserTestBase. Ideally a test conversion should only be the change of the namespace and used base class. Everything else should be covered by backwards compatible methods as far as possible/reasonable.
3) Prepare one big bang patch of test conversions in #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits. Commit it during code freeze of the next Drupal minor release (likely March 2017 for 8.3).
4) In the meantime open child issues to convert web tests for each core module. While doing so we should also check if the test cases actually require a browser or can be converted to unit/kernel tests. They will be postponed until #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits is committed.

Remaining tasks

Resolve all the child issues. Convert web tests in child issues per module/component in Drupal core.

Instructions for converting a Simpletest web test:

  • Read https://www.drupal.org/phpunit and all subpages, especially https://www.drupal.org/node/2783189 .
  • Move the test file to a tests/src/Functional folder in the module.
  • Don't bulk move the tests but move and work on 1 test at the time. Regularly upload a patch to see if the tests pass on the testbot as well.
  • Rename the namespace, make the class extends BrowserTestBase and fix the use statement.
  • Try to run the single test with phpunit, example for HelpTest (replace the path with the file you converted):
    cd core
    ../vendor/bin/phpunit modules/help/tests/src/Functional/HelpTest.php --stop-on-failure
    
  • Fatal error: undefined function assertNoLink(): not all legacy assertion methods are implemented yet, check #2750941: Additional BC assertions from WebTestBase to BrowserTestBase to borrow from there.
  • If there is a test method which does not perform a web request (for example no ->drupalGet() or ->drupalPostForm() calls) then that method should be extracted to a unit test or kernel test. $this->setRawContent() generally means the test should be converted to a Kernel test
  • $this->drupalPostAjaxForm(); means the test should be converted to a Javascript test. There most likely already is an issue for it, see #2809161: Convert Javascript/AJAX testing to use JavascriptTestBase
  • Otherwise try to change as little things as possible in the converted web test. The conversion should be easy to review and further cleanup can be done in a follow-up issue.
  • If you get stuck checkout the other child issues that do conversions - they can give you a hint what needs to be changed.
  • If you have tricky conversions, please read out to people. They might have an issue. Occasional it could be also helpful to skip tricky tests.
Progress report
8.2
$ find core/modules -print | grep "src/Tests" | wc -l
    1152
8.2
$ find core/modules -print | grep "src/Tests" | wc -l
    888

Conversions

Note: don't convert the following bits:

  • Tour tests
  • Update path tests
  • Views tests
  • WizardTestBase tests

Needed conversions per module

Completed

Tips: Common conversions

  • Simpletest allowed typecasting of elements. BrowserTestBase does not. Instead of (string) $element use $element->getText() or $element->getHtml();
  • Views tests need the new constructor format with $import_test_views = TRUE as the first parameter in the setUp() method.
  • Attributes are fetched with $item->getAttribute('value'); now, previously it was $item->value or $item->attributes()->value.

Issue creation

Please add the tag 'phpunit initiative' to every issue and add the issue to the correct Component (if it's a test for the comment module, add it to comment.module).

User interface changes

None.

API changes

Simpletest and WebTestBase will be deprecated.

Data model changes

None.

Comments

klausi created an issue. See original summary.

catch’s picture

Just a note when doing this we should also consider whether the test would be better as a kernel or unit tests - neither of which we had for the first several years we were adding web tests.

klausi’s picture

Title: Convert all Simpletest web tests to BrowserTestBase » Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase)
Issue summary: View changes

Good idea.

dawehner’s picture

Open child issues to convert web tests for each core module.

Not sure whether this is the best idea. Conceptual similar tests, aka. tests with similar scope, is often a better idea, because its easier to figure out what is going on.

catch’s picture

Yes that gets tricky as well. There's a fair bit of duplication between the test coverage for modules like node/taxonomy and entity system tests. Similarly between tests in Views module vs. tests in views-data-providing modules. That's no fun to pick apart but would be good if we did.

So issues by theme, then sub-issues for those sounds good.

jibran’s picture

Yes yes yes! but unfortunately it is not as straight forward as KTBNG conversion. I'm still struggling to convert DER WTB tests to BTB #2469609: Convert all WebTestBase to BrowserTestBase.
We, as Drupal developers, have to change our mentality about browser base testing. We don't have to think page as a giant html string or form as an giant array to pass to a function. We have to write tests in a way we test them on a browser. Like visit a link, click a link, click a form element. Form is not big piece of array we have to set. Input of the fields should be set in a way we set them while browsing.

For a start BTB::submitForm() is super flawed.
Exhibit A from DER #2712545: Field setting form is broken after 2412569 http://cgit.drupalcode.org/dynamic_entity_reference/tree/tests/src/Funct... and http://cgit.drupalcode.org/dynamic_entity_reference/tree/src/Tests/Dynam...
In WTB

    $this->drupalPostForm(NULL, array(
      'cardinality' => FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED,
      'settings[exclude_entity_types]' => FALSE,
      'settings[entity_type_ids][]' => ['user', 'entity_test'],
    ), t('Save field settings'));

BTB equivalent

    $page = $this->getSession()->getPage();
    $entity_type_ids_select = $assert_session->selectExists('settings[entity_type_ids][]', $page);
    $entity_type_ids_select->selectOption('user');
    $entity_type_ids_select->selectOption('entity_test', TRUE);
    $assert_session->selectExists('cardinality', $page)
      ->selectOption(FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED);
    $page->uncheckField('settings[exclude_entity_types]');
    $this->submitForm([], t('Save field settings'), 'field-storage-config-edit-form');
dawehner’s picture

@jibran
Can you elaborate why we have to? There is submitForm() for this kind of work.

jibran’s picture

And then there is a way we assert stuff on the page in WTB and BTB.
For example, before #2711963: Modernized ToolbarIntegrationTest ToolbarIntegrationTest.php was asserting stuff like WTB, treating page as giant piece of html string, and after #2711963: Modernized ToolbarIntegrationTest ToolbarIntegrationTest.php has a nice flow to it.

I'm still trying to find a perfect balance with this apporoch as well in #2702233: Add Javascript tests for Form API's #states. How to assert stuff on a page with form? Should we add FormWebAssert like WebAssert or not, as pointed in #2702233-30: Add Javascript tests for Form API's #states? Or we shouldn't do this at all as suggested in #2702233-25: Add Javascript tests for Form API's #states.

jibran’s picture

@jibran
Can you elaborate why we have to?

Short answer BTB is designed to work with different type of drivers and GoutteDriver sets element differently then PhantomJSDriver.

jibran’s picture

mpdonadio’s picture

Given that KTB/UTB tests typically live in a different namespace than WTB tests, how stupid of an idea would it be to do this as

1. Write equivalent KTB/UTB for existing WTB classes and similar tests that can run in a simpler space.
2. Make old tests as @deprecated
3. Run with both sets of tests for a while and see if we have regressions that one test catches that the other doesn't
4. Remove the @deprecated test before 9.0.0.

My fear is that that WTB has some implicit side effect coverage going on, and if we simplify (for lack of a better term) we may actually be losing some test coverage.

(this thought is somewhat half baked, but I have click save and head to a meeting)

dawehner’s picture

@mpdonadio
I think this could be an interesting strategy for some subset of the tests. Maybe some which are really side wide.

klausi’s picture

That was fun: #2735199: Convert web tests to browser tests for help module

I had to implement many assert*() methods on BrowserTestBase and made the following decisions:
* Mink does not support custom messages for the built in assertions. So I just ignore a potential $message parameter in that case.
* I completely removed $group support for assertions. I don't think it makes sense to keep them, I never needed that when working/debugging with Simpletest.

The good news is that I had to change very few things in the web tests of help module to make this work. I'm hopeful that this can be a moderately easy conversion process.

We should probably create dedicated test coverage for all the new stuff on BrowserTestBase, I was just lazy for now to quickly see what we actually need.

klausi’s picture

Next one: #2736109: Convert web tests to browser tests for action module.

The porting process is very straight forward, no pain points there - just copying and modifying all the helper methods from WebTestBase.

klausi’s picture

Good news regarding test execution time: I was worried that BrowserTestBase would take longer to execute the same test, but that is not the case.

Simpletest:

$ time php run-tests.sh --dburl mysql://drupal-8:drupal-8@localhost/drupal-8 --url http://drupal-8.localhost --file core/modules/action/src/Tests/BulkFormTest.php 

Drupal test run
---------------

Tests to be run:
  - Drupal\action\Tests\BulkFormTest

Test run started:
  Sunday, May 29, 2016 - 22:54

Test summary
------------

Drupal\action\Tests\BulkFormTest                              72 passes                                      

Test run duration: 31 sec


real	0m32.197s
user	0m8.016s
sys	0m0.860s

PHPUnit Browsertest:

$ time ../vendor/bin/phpunit modules/action/tests/src/Functional/BulkFormTest.php 
PHPUnit 4.8.11 by Sebastian Bergmann and contributors.

.

Time: 30.46 seconds, Memory: 4.00Mb

OK (1 test, 28 assertions)

real	0m30.508s
user	0m7.184s
sys	0m0.876s

The converted Browser test is 1 second faster! I assume because of the complicated simplest module bootstrapping we have to do in run-test.sh.

larowlan’s picture

Sweet, and the second one includes all of phpunit's discovery processes, which are high for a single test, but for a full-testsuite or group are lower

dawehner’s picture

@larowlan
Are you sure this is not triggered by less tests at the moment? What would happen if we have the same amount of tests as we have with simpletest now?

benjy’s picture

I don't find the debugging experience as good with BTB as WTB, the UI is very helpful for finding the rendered HTML closest to the failure or request you want to inspect. IMO, we should fix that before we replace everything with BTB.

klausi’s picture

I think the debugging experience is better with PHPUnit and BrowserTestBase. Once you have copied core/phpunit.xml.dist to phpunit.xml and enabled BROWSERTEST_OUTPUT_DIRECTORY + configured printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter" you get very useful links when running the test.

When a test fails phpunit stops execution immediately, so the last printed link to the HTML response from your test is the place where it broke. No fiddling around in the Simpletest UI and desperately searching for the line where it actually broke and what might be the bad response. For me that is a lot better than Simpletest UI.

Sure - there is still room for a lot of developer experience improvements, but I think we should not hold up conversion to BrowserTestBase because of that. Can be parallel efforts.

dawehner’s picture

When a test fails phpunit stops execution immediately, so the last printed link to the HTML response from your test is the place where it broke. No fiddling around in the Simpletest UI and desperately searching for the line where it actually broke and what might be the bad response. For me that is a lot better than Simpletest UI.

I agree, this is something we really need to teach people. We fail early, because in simpletest, further failures could be totally misleading once the system was in a broken state at some point.

Sure - there is still room for a lot of developer experience improvements, but I think we should not hold up conversion to BrowserTestBase because of that. Can be parallel efforts.

Oh yeah totally. Please open issues if you have ideas!

larowlan’s picture

larowlan’s picture

Found in #2737805: Convert web tests to browser tests for forum module that submitting admin/modules page to uninstall/install a module kicks out the logged in user. That one will be fun to debug...

dawehner’s picture

klausi’s picture

#2735045: Convert StandardTest to BrowserTestBase by introducing a AssertLegacyTrait and new assertions on the WebAssert could use a review, it sets the foundation for all the other conversion issues.

alexpott’s picture

We need to fix #2575725: Run PHPunit tests one at a time via UI to avoid exception when selecting too many before doing too much BTB conversion since the fact they take some time to run means that the approach of trying to run all the PHPUnit tests first and in the form submission is quite flawed.

dawehner’s picture

klausi’s picture

We are still in the process of bringing BrowserTestBase fully up to speed with the deprecated WebTestBase, here is another issue that could use a review: #2751875: BrowserTestBase must enable cache tags in responses

dagmar’s picture

I would like to help with the dblog tests conversion, I been working on several issues of this module for the last 2 months and I feel quite comfortable to help with this task. However, reading the comments, and specially #6 it seems this is not easy to implement.

I think it could be a good idea to update the summary issue to reflect what are the main changes need to be made in order to convert a WebTestcase into Unit or Kernel Test base.

So in the section "Proposed resolution" in addition to "create child issues" we should have more details regarding when a Kernel Test applies, or when a Unit test applies. I'm pretty sure this documentation already exists but we need links to this docs, otherwise I don't see how other people can join to help in this plan.

klausi’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Added instructions.

We are not quite there yet for very smooth conversions, since we are still uncovering missing legacy methods.

Also https://www.drupal.org/phpunit does not even mention BrowserTestBase, a section we will have to write.

And https://api.drupal.org/api/drupal/core!core.api.php/group/testing/8.2.x is outdated, we need to replace the Functional part instructions with BrowserTestBase and JavascriptTestBase.

So I think there are no real docs yet - we should copy https://drupalcommerce.org/blog/45322/commerce-2x-unit-kernel-and-functi... to a drupal.org doc page.

Thanks for helping!

klausi’s picture

I updated the docs at https://www.drupal.org/node/2116263 now to tell you how to run browser tests.

dagmar’s picture

Amazing. Thanks!

dawehner’s picture

Thanks a ton klausi!

klausi’s picture

Issue summary: View changes
FileSize
1.12 KB

I created a script to help convert web tests to browser tests for a core module, see attachment. Copy it to the root of your Drupal 8 checkout and rename it to browsertest-convert.php. Usage:

php browsertest-convert.php node

... will convert all web tests in node module to browser tests with primitive string replacements, so that you don't need to do the easy work manually.

@todo This probably does not work with web tests in subfolders.

klausi’s picture

Issue summary: View changes
dawehner’s picture

I still believe we should go through the individual tests and identify places which could be rather a kernel test.

klausi’s picture

Yes, of course, this script only helps you get started.

klausi’s picture

Yay, #2750941: Additional BC assertions from WebTestBase to BrowserTestBase was committed! That is a big win of legacy compatibility functions, so that it is easier to convert an old web test.

Ideally our conversions should look as trivial as this: #2755991: Convert web tests to browser tests for telephone module (please review!)

A general note on conversions: we should touch the test cases as little as possible if they stay as browser test to make the conversion easier to review. We can convert the deprecated assert function calls later, our goal right now is to get off of SimpleTest as quickly as possible.

pfrenssen’s picture

When adding new, untested, assertions to AssertLegacyTrait, please double check the visibility. If a method is added that is public instead of protected it will cause fatal errors in real life BrowserTestBase implementations in contrib.

See for example: #2744089: Fix visibility of AssertLegacyTrait::assertLink(), #2752315: Fix visibility of AssertLegacyTrait::assertNoEscaped(), #2757139: Fix visibility of AssertLegacyTrait::buildXPathQuery().

I personally think the AssertLegacyTrait should not be included by default in BrowserTestBase at all, instead it should be included in the converted legacy tests.

klausi’s picture

Issue summary: View changes

Fixed script link.

benjy’s picture

When a test fails phpunit stops execution immediately, so the last printed link to the HTML response from your test is the place where it broke.

This is great when you have a nice fast unit test, run fail, fix and run again, great. But when you're talking about a web tests that can run for 3-10minutes, if it fails after 5 minutes on an assertion, then you run it again for another 5 minutes and it fails on the next line, that's just annoying. Sometimes it's better to know all the failures so you can attempt to fix more than one at a time. stopOnFailure="false" would be a better default for longer running browser tests IMO.

Sure - there is still room for a lot of developer experience improvements, but I think we should not hold up conversion to BrowserTestBase because of that. Can be parallel efforts.

The problem here is, we seem to favour doing conversions and getting to the new shiny rather than adding the polish it needs. The verbose output only works for drupalGet() and drupalPost(), which is ironic given one of the benefits of BTB is familiar API's (mink) but if you use Mink directly, you don't get any verbose output. Also, other BTB functions like clickLink() do requests without capturing verbose output. IMO, we need a reliable way to capture output whenever a request happens in the test.

klausi’s picture

Thanks for the feedback.

When an assertion fails in PHPunit then an exception is thrown, so there is no way to continue with the test. The --stop-on-failure configuration only helps you when running multiple tests. Then it will stop on the first test that fails.

Verbose output: we already hacked the HTTP client for mink to catch errors in #2664150: Expand BrowserTestBase with error handling support, so we should do something similar to always generate verbose output.

benjy’s picture

The --stop-on-failure configuration only helps you when running multiple tests

Ahh, i see that's a shame.

Verbose output: we already hacked the HTTP client for mink to catch errors

How come we override the client rather than the driver? They look pretty similar looking at them.

klausi’s picture

Opened #2763401: PHPunit browser tests should log all Mink requests to discuss and implement HTML output logging for all requests, let's continue there.

dawehner’s picture

Just to point out the failure discussion ... when there is a failure, anything could happen afterwards, much like you can prove anything when you start with "0 = 1". Given that the only way to fix tests properly is to fix the first failure.

benjy’s picture

Sure anything can happen but most of the time you can infer quite well what went wrong. Take for example the following code, it would be helpful to see all failures, for example if the array was reversed vs being in a random order, it could help you find the issue right away. There are many situations like this imo.

$this->assertEqual(1, $files[0]->id()); // Fail, 1 does not equal 4.
$this->assertEqual(2, $files[1]->id());
$this->assertEqual(3, $files[2]->id());
$this->assertEqual(4, $files[3]->id());
dawehner’s picture

Sure anything can happen but most of the time you can infer quite well what went wrong. Take for example the following code, it would be helpful to see all failures, for example if the array was reversed vs being in a random order, it could help you find the issue right away. There are many situations like this imo.

Mh fair, but its seriously something the design of PHPUnit will never be able to support. The alternative for this usecase though would be to use something like the following:

$file_ids = array_map(function ($file) { return $file->id();}, $files);
$this->assertEquals([1, 2, 3, 4], $file_ids);
klausi’s picture

@benjy: continuing tests after the first assertion already failed can be a huge time waster. Since anything can happen after that failed assertion a PHP fatal error is likely in a completely different location, distracting you from the first fail. Or worse: our testbot is not even capable to display the failed assertions when a PHP fatal error has happened later. If you cannot reproduce that fail locally you need to start with weird robustness fixes to prevent the fatal error from happening just to see which assertion failed. Example: MigrateUpgrade7Test which I have to modify to even get an idea what went wrong on the testbot in #2712647: Update Symfony components to ~3.2.

So I think stopping a test case as soon as the first assertion fails is a very good thing in PHPUnit.

mpdonadio’s picture

I pretty strongly disagree with #49 and #46. It really depends on what you are doing.

Within a test, if doing something sequential, then I can see stopping at the first failure. But, in other cases, seeing all the fails can be beneficial. If feeding a test with a data provider, then I want to see all of the fails at once: I can likely address several of the fails at once. I was working on an integration test last night, where seeing all of the fails at the same time pointed to the true cause of my problem.

When doing a full test run (which outside of just the UnitTestCase is mostly infeasible on a local machine) aborting after the first fail (even within a particular test) will probably end up with more full runs on the bot.

I think evaluating whether we can downgrade tests, which will shorten runs, will be the biggest benefit both for local dev and load on the bots.

Mile23’s picture

Issue tags: +Needs Documentation

@klausi #20: I think the debugging experience is better with PHPUnit and BrowserTestBase. Once you have copied core/phpunit.xml.dist to phpunit.xml and enabled BROWSERTEST_OUTPUT_DIRECTORY + configured printerClass="\Drupal\Tests\Listeners\HtmlOutputPrinter" you get very useful links when running the test.

Can we make sure that's in the docs somewhere if it isn't already?

claudiu.cristea’s picture

New issue/bug that blocks some test conversions #2767655: Allow WebAssert to inspect also hidden fields.

claudiu.cristea’s picture

I provided a patch that is green in #2767655: Allow WebAssert to inspect also hidden fields. That needs review.

anavarre’s picture

claudiu.cristea’s picture

xjm’s picture

I posted #2770549: [plan] Discuss desired API and test suite following deprecation of SimpleTest to help address some of the architectural and long-term questions without derailing the conversions.

klausi’s picture

@Mile23: yes, this is mentioned in the documentation at https://www.drupal.org/node/2116263

What we still need for documentation: how to write browser tests, similar to https://www.drupal.org/simpletest-tutorial-drupal7 - same for javascript tests.

xjm’s picture

@klausi agreed, that would be very valuable.

xjm’s picture

Hmm, also, -1 for doing this on a per-module basis. I'd recommend trying #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits and seeing what shakes out there, not only in terms of the API but also in terms of DrupalCI compatibility. E.g. there is a recent random fail in forum tests that may be caused by a difference between SimpleTest and BTB, but when it's on a per-module basis it becomes difficult to debug or track down. Or, see https://www.drupal.org/pift-ci-job/390573

And honestly I'd also kind of feel we need to have the documentation mentioned above before we do conversions, because we've missed the mark on the documentation gate so far for BTB. I wouldn't yet recommend reverting the 3 or 4 per-module conversions that have gone in, but it is something we might need to consider.

dawehner’s picture

I'm all for not backporting actual test changes, this is not adding that much value.

Backporting test framework changes though have a couple of big advantages:

  1. People more likely will see those improvements and leverage it as part of their daily workflow
  2. Contrib authors potentially have less of an issue to keep their tests working on the two current important branches of core
claudiu.cristea’s picture

claudiu.cristea’s picture

Yet another one that is required by more than one test conversion #2773733: Fix AssertLegacyTrait::assert(No)Text to handle html encoded strings (part 3). So I opened a dedicated issue.

benjy’s picture

Ran across this today https://elliot.land/7-reasons-to-use-concise-on-top-of-phpunit thought it might be interesting following on from the convo in #47, it provides a verify() option so it doesn't quick fail the test. Also, the tool claims to have a better CLI/runner than PHPUnit. Could be worth an issue to discuss if anyone is interested.

dawehner’s picture

@benjy
Sure open up an issue, but its unrelated with BTB, because concise is on a lower level.

For example when I review those patches I actually look at the converted tests and see whether they could be fundamentally approved.

Mile23’s picture

I suggest beginning by converting per test base class.

In the entire hierarchy of WebTestBase, there are 930 classes. 403 of those are direct subclasses. 31 are *TestBase classes, which means ~500 subclass from test bases. 31 issues for 500 classes.

Here's our first round: Make an issue for each of these test base classes. Convert the base class, pick up the pieces on the subclasses.

Found 31 matches of extends webtestbase in 31 files.	
AggregatorTestBase.php	
abstract class AggregatorTestBase extends WebTestBase {      [position 13:35]	
BlockTestBase.php	
abstract class BlockTestBase extends WebTestBase {      [position 11:30]	
BlockContentTestBase.php	
abstract class BlockContentTestBase extends WebTestBase {      [position 12:37]	
CommentTestBase.php	
abstract class CommentTestBase extends WebTestBase {      [position 16:32]	
ContentTranslationTestBase.php	
abstract class ContentTranslationTestBase extends WebTestBase {      [position 14:43]	
FieldTestBase.php	
abstract class FieldTestBase extends WebTestBase {      [position 12:30]	
FileFieldTestBase.php	
abstract class FileFieldTestBase extends WebTestBase {      [position 14:34]	
FileManagedTestBase.php	
abstract class FileManagedTestBase extends WebTestBase {      [position 13:36]	
ImageFieldTestBase.php	
abstract class ImageFieldTestBase extends WebTestBase {      [position 25:35]	
MenuWebTestBase.php	
abstract class MenuWebTestBase extends WebTestBase {      [position 10:32]	
MigrateUpgradeTestBase.php	
abstract class MigrateUpgradeTestBase extends WebTestBase {      [position 13:39]	
NodeTestBase.php	
abstract class NodeTestBase extends WebTestBase {      [position 12:29]	
PathTestBase.php	
abstract class PathTestBase extends WebTestBase {      [position 10:29]	
RESTTestBase.php	
abstract class RESTTestBase extends WebTestBase {      [position 13:29]	
SearchTestBase.php	
abstract class SearchTestBase extends WebTestBase {      [position 11:31]	
ShortcutTestBase.php	
abstract class ShortcutTestBase extends WebTestBase {      [position 13:33]	
InstallerTestBase.php	
abstract class InstallerTestBase extends WebTestBase {      [position 17:34]	
StatisticsTestBase.php	
abstract class StatisticsTestBase extends WebTestBase {      [position 10:35]	
AjaxTestBase.php	
abstract class AjaxTestBase extends WebTestBase {      [position 10:29]	
CacheTestBase.php	
abstract class CacheTestBase extends WebTestBase {      [position 10:30]	
PageCacheTagsTestBase.php	
abstract class PageCacheTagsTestBase extends WebTestBase {      [position 12:38]	
DatabaseWebTestBase.php	
abstract class DatabaseWebTestBase extends WebTestBase {      [position 11:36]	
ToolkitTestBase.php	
abstract class ToolkitTestBase extends WebTestBase {      [position 11:32]	
MenuTestBase.php	
abstract class MenuTestBase extends WebTestBase {      [position 7:29]	
ModuleTestBase.php	
abstract class ModuleTestBase extends WebTestBase {      [position 14:31]	
SystemConfigFormTestBase.php	
abstract class SystemConfigFormTestBase extends WebTestBase {      [position 14:41]	
UpdatePathTestBase.php	
abstract class UpdatePathTestBase extends WebTestBase {      [position 39:35]	
TaxonomyTestBase.php	
abstract class TaxonomyTestBase extends WebTestBase {      [position 11:33]	
TourTestBase.php	
abstract class TourTestBase extends WebTestBase {      [position 10:29]	
UpdateTestBase.php	
abstract class UpdateTestBase extends WebTestBase {      [position 26:31]	
ViewTestBase.php	
abstract class ViewTestBase extends WebTestBase {      [position 19:29]	
Berdir’s picture

Per base class is something that we also did more or less in kernel tests although I often did more than one there except for the really big ones. I think that does work fairly well.

One thing there is that contrib/custom code might also extend from those (extending from others is also possible but less likely. I sometimes do that for different backends, e.g. redis queue testing). what we did for kernel tests is mostly keeping the old base classes around, so contrib has some time to shift from the old to the new base classes and doesn't break every time we commit a new patch.

xjm’s picture

I actually would like to get as close as possible to making the conversion of core a single, scheduled change, that we would do during a beta or RC phase, or immediately when a minor branch opens. Edit: The work already going on to add BC layers will work well with that approach.

(Editing to remove comments from the wrong issue, sorry).

claudiu.cristea’s picture

(...) making the conversion of core a single, scheduled change...

@xjm, this is not possible for some of the tests, especially for those that are testing JS features in forms by setting hidden fields. This is because of Mink architecture that tests everything from the perspective of a human. An example is #2757007: Convert all book web tests to BrowserTestBase and I'm sure we'll find others.

xjm’s picture

We should try to get as close to possible to converting the whole test suite in one massive, scheduled patch with a single scope once we have provided as much BC as possible, but it may be a few massive patches (one for the simplest scope to just change which class is extended, another for a specific change throughout core that is impossible to provide a BC layer for, another different change that is impossible to provide a BC layer for, etc). It would be during a beta/RC phase or immediately after a new minor branch is opened. This mitigates the intermittent small disruptions and makes it something we can announce in advance and plan for, once we're sure the documentation and the DrupalCI support and the BC layer and everything else is in really great shape.

There are a number of reasons for this, many outlined in https://www.drupal.org/core/scope. This was also pointed out in #4 on this issue by @dawehner. A few specifics that are of concern for me in this case:

  • Documentation. We have comprehensive documentation for SimpleTest in the handbook and API docs, but comprehensive docs for BTB are pretty much missing. @dawehner has a starter patch in #2469713: Step 2: Create a JavaScriptTestBase using PhantomJs Driver/Binary but there is still a long way to go.
  • Contrib may extend base classes, as @Berdir points out. If we provide enough BC to make core test conversions as close as possible to a single scope, the contrib pain will be eased a lot. Let's say we want 60%-80% of web tests to work just by changing the class it extends. If we also have a clear guide on how to convert the remaining 20%, then contrib developers can plan well in advance that we will do it on a scheduled date.
  • Consistency. Non-core developers are confused, put off, and sometimes actually distressed by the fact that core inconsistently uses different APIs for the same thing. The closer we can get to changing core all at once, the more this DX issue is mitigated.
  • As-yet unknown issues and infrastructure support. There are still some things happening on DrupalCI that seem to be exposed by BTB tests and not simpletest. Having one conversion of 5 test classes introduce 75% random failures on secondary environments is a huge problem. If we make the conversion patches more conceptually scoped (and bigger), we can make sure they are tested on all environments and also give enough heads up to ensure d.o infrastructure resources are available to fix blockers. The same is true for bugs we may not yet know about in our BTB test running, etc. within core itself.
xjm’s picture

@claudiu.cristea, I crossposted, but hopefully the latest comment clarifies. Thanks!

xjm’s picture

I'd modify @Mile23's suggestion as follows:

  1. Add BC layers to make as much as possible pass in #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits until we are out of ideas on how to safely add BC. Test it extensively on all supported environments.
  2. Identify specific changes we can't provide BC for, but that could be scripted.
  3. Meanwhile, make sure things like documentation etc. get completed in parallel.
  4. Then go down the route of changing base classes for remaining things.

We'd stack a series of a few large but individually conceptually simple issues together as a scheduled conversion, and then possibly work out the stragglers, rather than the other way around.

Mile23’s picture

@xjm #71.1: If #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits isn't just exploratory but is supposed to be part of a process it really needs an IS update for how it fits in.

Agreed on stacking large but simple issues, but I'm not sure what the atomic unit is here. We'd want to be able to review each piece without going insane. :-)

If we care that contrib is using our simpletest base classes, then we can make a BTB version of the base classes alongside the WTB ones, with deprecation info. We leave the WTB ones alone and modify the code core uses.

OTOH, there's already a bunch of work done in conversion per-module, so perhaps while the documentation is coming together we could grab those? We could postpone any issue for a module that defines a test base class.

xjm’s picture

Thanks @Mile23!

The work so far in the per module issues is definitely still useful; it gives us diffs for the comprehensive change record/conversion docs and proves BC layers that immediately work across core and not just in that module. I just don't think we should actually commit them individually at this time.

dawehner’s picture

Per module patches also help with reviews, as maintainers of the particular modules can review their space.

Could we agree on the following approach maybe? @xjm
This gives quick progress now while still adding the BC layers we might want.

  1. Take #2770921-10: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits and remove every failing test
  2. Add BC layers like described in #2770921-11: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits
  3. Try to move all use statements again
  4. Goto Step 2
  5. At some point identify tricky tests, Update path etc.
mpdonadio’s picture

Haven't seen this in any of the related issues. A lot of tests have assertions like this:

    $this->drupalPostForm(NULL, $edit, t('Save'));
    preg_match('|entity_test/manage/(\d+)|', $this->url, $match);
    $id = $match[1];
    $this->assertText(t('entity_test @id has been created.', array('@id' => $id)), 'Entity was created.');

Has a precedence been set yet for how to handle this pattern ($this->url isn't in BrowserTestBase)? Or do we need to add $this->url?

neclimdul’s picture

Seems it would be better to assert before the post that no items in the database matches the posted criteria then load the item using the same criteria afterwards. You can then assert both that the entity was created and you have the id you're looking for to assert the message on the page.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Eric_A’s picture

xjm’s picture

catch’s picture

klausi’s picture

Ladies and gentlemen, I proudly present: the PHPUnit browser test tutorial doc page! https://www.drupal.org/node/2783189

Please improve as you see fit :)

dawehner’s picture

Thanks so much klausi++

larowlan’s picture

Thanks @klausi. Made some minor changes. Great work.

klausi’s picture

dawehner’s picture

Opened #2784159: Remove CURL timeout in BTB for some better debugging experience

klausi’s picture

klausi’s picture

dawehner’s picture

xjm’s picture

So glad to see the handbook page! Thanks @klausi!!

The summary could definitely use an update here, for the handbook page, my comment in #69 and the discussions we had after it, etc.

larowlan’s picture

Found one that should have been a kernel test, #2791955: Convert \Drupal\node\Tests\NodeAccessTest to a KernelTest - reviews welcome

klausi’s picture

Title: Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase) » PHPUnit initiative: Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase)
Issue summary: View changes
Issue tags: -Needs issue summary update

Labelling this as a core initiative now :-)

@xjm: I think we cannot do one big patch now, because we are still in the phase where we iron out problems with BrowserTestBase. We can only fix problems while we convert smaller parts of Drupal core and actually commit them. For example #2737805: Convert web tests to browser tests for forum module showed that we have random test fails with browser tests, but we would have never uncovered that if we had not made the core commit and reverted it. We cannot avoid this painful phase where we figure out the edge cases and shortcomings of our current browser tests. We have to convert small parts of Drupal core to get it battle tested.

So I propose that we continue with the conversion issues that are open right now, make them ready and commit them to core. Any conversion issue that is opened after 2016-Sept-01 is postponed. Then we make #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits ready as big patch. There is a risk that we will have to do endless rerolling in that issue and realistically it will not be ready by Oct 2016. So we will have to postpone it to March 2017 (when we are in code freeze again for 8.3). That will frustrate me and I will simply keep working on other postponed conversion issues.

Actually I would prefer to keep going with incremental conversions because people already have to live with 2 test systems in Drupal core. Seeing the conversion work committed sooner to core also motivates contributors.

Anyway - let's plan for the big bang patch approach for now and if we are faster as expected with the preliminary conversions then we can re-plan if necessary.

chx’s picture

If this is an initiative it surely would be nice if "each test runs in a completely new Drupal instance, which is created from scratch for the test" -- would get an option to not do that. Exercising an existing site is helpful and simpletest 7.x-2.x had an option for that while I know that's contrib, well, there's no place to put that now as simpletest is being removed and many contrib was core'd in D8.

Sam152’s picture

We are already using the feature described in #92 with BTB and JTB for testing client projects by copying the whole setUp method, not calling parent::setUp and omitting anything which installs drupal or assumes a new environment. A very handy feature which would be fantastic if it was made more accessible and easier to get started with.

klausi’s picture

@chx and @Sam152: that would make sense, can you open a feature request for that and link it here?

chx’s picture

I can't but I am sure @Sam152 can.

Sam152’s picture

Sure.

Also credit to @larowlan for doing the legwork to make this possible for our internal projects. Probably worth getting your thoughts on how we could achieve this.

dawehner’s picture

dawehner’s picture

@chx
Please next time just open up a new issue, as this kind of stuff derails other discussions. Thank you!

chx’s picture

Before we ditch simpletest (which was running *.test files until very recently) we need to solve the following: When developing a test, you run phpunit path/to/test.php and the test passes but there is no warning the test won't be picked up when running a group of tests because it's not called *Test.php, because it violates PSR0/4 or whatever else it needs.

Alternatively we need to ditch phpunit and use a modern framework. Fantasy.

klausi’s picture

@chx: test discovery will not be a blocker to further adapt PHPUnit. This is also not PHPUnit's fault, if you think our discovery needs improving please look at #2605664: [Needs change record updates] Align TestDiscovery and bootstrap.php's non-permissive loading of PSR-4 namespaces for traits or open a new issue if that one does not match.

PHPUnit is the most widely used test framework in PHP, any particular alternative you have in mind?

dawehner’s picture

Note: #2296635: Evaluate performance impact of limiting test discovery to *Test.php filename suffix actually fixes the problem described by chx by failing in case a test class doesn't line up the with the expectations.

dawehner’s picture

So well, can someone please decide the way forward?

IMHO going on with individual issues is still not a bad idea, because, well, we have found a tons of things along the way, while being able to work in parallel. Unlike codestyle issues, where the outcome is sort of predictable, its not on those conversion issues IMHO.

But well, whatever we do, please @xjm, @alexpott decide on something, because this is obviously better than waiting.

Mile23’s picture

@chx: When developing a test, you run phpunit path/to/test.php and the test passes but there is no warning the test won't be picked up when running a group of tests because it's not called *Test.php, because it violates PSR0/4 or whatever else it needs.

phpunit --filter MyTest does full discovery and then filters with a regex. It's a bit slower but you can do sanity check with it when you add the test file.

neclimdul’s picture

@chx: When developing a test, you run phpunit path/to/test.php and the test passes but there is no warning the test won't be picked up when running a group of tests because it's not called *Test.php, because it violates PSR0/4 or whatever else it needs.

For my understanding is this a complaint against phpunit or how Drupal run's tests? The described behaviors seem to be consistent with how tests work in simpletest unless I'm missing something.

catch’s picture

I've opened #2794285: [meta] Add backwards compatibility layer to browsertestbase to ease conversion from webtestcase to track the bc layer for webtestbase - please add additional child issues to that if you know of them.

klausi’s picture

Yesterday in IRC we discussed a bit with alexpott because he has postponed preliminary phpunit conversion issues until there is agreement on the proposal in the issue summary here.

Note that the issue summary is just the best compromise proposal I know of for now, it was not my intent to override or dismiss the concerns xjm has raised. @xjm: can you agree to commit the handful of preliminary browser test conversions that are currently open? If not we will come up with alternative proposals, just let us know :)

webchick’s picture

Issue tags: +Drupal core ideas

Hi there! This is an issue that we'd ideally like to move to the new "Drupal core ideas" queue when/if #2785735: [policy, no patch] Agile process for bigger changes in core (Part A: Ideation) goes through (hoping for the next week or two). If you could read that issue and provide your thoughts on that, that would be great!

klausi’s picture

Hey webchick! I think this is not a core "idea" in the sense that it would need buy-in from the community or core maintainers - we already have their blessing and decided many years ago that we want to go with PHPUnit as a replacement for Simpletest. In fact, most of the core tests are now PHPUnit tests already :-)

What we are discussing here is not the "what" or "why", just the "how". There is no risk in wasted effort where people work on something which later gets rejected - any PHPUnit conversion people work on will sooner or later be committed to Drupal core. It is only a question of when, that's why we are bikeshedding the process here a bit :)

So I would remove the "Drupal core ideas" tag again if that is ok with you?

webchick’s picture

Issue tags: -Drupal core ideas

Actually, yeah, good call. Sorry I was just quickly running down the list of meta plans looking for things that seemed to fit.

Still, if you have thoughts, feel free to add them in that issue! :)

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

dawehner’s picture

chx’s picture

> that it would need buy-in from the community or core maintainers - we already have their blessing

You left out "Some of".

dawehner’s picture

dawehner’s picture

dawehner’s picture

jhodgdon’s picture

I have a question about this issue: Can you still make Verbose output in BrowserTestBase?

The reason I am asking is that we are using the Simpletest UI (which I also understand you want to get rid of, with as far as I can tell no currently viable alternative, which I commented on, on the other issue) to make automated screen shots for the User Guide project.

EDIT: What I meant by "other issue" here was: #2750461: Remove Simpletest UI because we don't want to maintain a graphical test runner

What I did was make a method that works like verbose output, that puts out a cleaner HTML file with some injected JavaScript in it, and then I use a combination of a Greasemonkey script and Image Magick to grab screenshots of these HTML output files and trim them down to nice images for the User Guide.

Will this be possible to do in BrowserTestBase, do you think? So far I have not been able to produce verbose output from the command line at all, and I would hate for the User Guide to die in Drupal 9 due to not being able to keep automating the screen shots.

klausi’s picture

Verbose output is being implemented in #2763401: PHPunit browser tests should log all Mink requests.

klausi’s picture

#2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits is ready now, so it would be super helpful to get core committer feedback here or in the other issue if there is still something we can do for the 8.2 release.

We also haven't decided if the proposal in the issue summary is how we want to proceed, so some core committer approval or rejection on that would also be highly appreciated :) There is some confusion what we should work on, for example #2755991: Convert web tests to browser tests for telephone module was postponed although it was opened before 2016-Sept-01.

dawehner’s picture

When one would ask me, I would just commit it to 8.3.x, also all the other conversions, but keep adding the BC layers / additional assertions to 8.2.x as well.

We certainly killed the momentum completely which we had.

mpdonadio’s picture

I'd love to see this proceeding, too, so that I can staging refactoring tests / split out kernel/unit tests at the same time.

catch’s picture

#128 seems like a good idea to me, one caveat is whether git handles the renames when cherry-picking patches between 8.3.x and 8.2.x, if it doesn't that'd be a pain.

klausi’s picture

The problem with only committing to 8.3.x is that your tests diverge and makes it harder to provide patches for both branches. We already had problems in the past where we committed the kernel test conversion only to one branch, so people working on security issues had a harder time.

Of course we would leave all test base classes in place and just deprecate them, but otherwise test cases are not part of our API compatibility promise. I think we avoid pain when we have tests in the exact same form on both branches.

Anyway: let's get back momentum in this initiative in the meantime. We have the easy conversion patch ready now at #2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits. Let's un-postpone the harder conversions per module again where we need to tweak the tests a bit. When such an issue is ready we add the "phpunit ready for conversion" tag instead of settings it to RTBC. That way we have lots of conversion patches ready until we set the date here.

claudiu.cristea’s picture

claudiu.cristea’s picture

xjm’s picture

(Linking the issue that apparently fixed the random fails related to the Forum tests, which @dawehner had me find.)

klausi’s picture

We had a meeting a Drupalcon Dublin and made a plan to launch this as official Drupal core initiative. #2807237: PHPUnit initiative

Gábor Hojtsy’s picture

Project: Drupal core » Drupal core ideas
Version: 8.3.x-dev »
Component: simpletest.module » Active Initiative

Moving to the ideas queue as an active initative as per https://www.drupal.org/core/roadmap

klausi’s picture

Title: PHPUnit initiative: Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase) » Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase)
Project: Drupal core ideas » Drupal core
Version: » 8.3.x-dev
Component: Active Initiative » simpletest.module

Nope, this is just a plan issue to organize stuff. The initiative issue is #2807237: PHPUnit initiative.

Gábor Hojtsy’s picture

Ok, this was listed as an initiative issue at https://www.drupal.org/core/roadmap. Also that it is a plan could also put it in the ideas queue, especially if it needs some signoffs, but if you believe this is the best place, let's keep it here.

Gábor Hojtsy’s picture

Note that since that page now links to ideas queue lists, this being in the core queue will decrease the visibility as it will not be listed anymore.

klausi’s picture

#2807237: PHPUnit initiative is the issue in the idea queue, so we are listed just fine :)

mpdonadio’s picture

I hope I am wrong about this, but in WTB, assertText() is case sensitive:

\Drupal\simpletest\AssertContentTrait

  protected function assertTextHelper($text, $message = '', $group = 'Other', $not_exists = TRUE) {
    if (!$message) {
      $message = !$not_exists ? SafeMarkup::format('"@text" found', array('@text' => $text)) : SafeMarkup::format('"@text" not found', array('@text' => $text));
    }
    return $this->assert($not_exists == (strpos($this->getTextContent(), (string) $text) === FALSE), $message, $group);
  }

Note the strpos in the actual assert().

In BTB, assertText is case insensitive. The BC later eventually calls \Behat\Mink\WebAssert

   public function responseContains($text)
    {
        $actual = $this->session->getPage()->getContent();
        $regex = '/'.preg_quote($text, '/').'/ui';
        $message = sprintf('The string "%s" was not found anywhere in the HTML response of the current page.', $text);

        $this->assert((bool) preg_match($regex, $actual), $message);
    }

Note the 'ui' modifier on the regex.

If this is true, what does this mean for WTB to BTB conversions or our current BC layer?

EDIT: I traced this out wrong. The deprecated BTB assertText eventually calls the BTB version of assertTextHelper(), which does a strpos. But, the docs say to use the Mink equivalents, which are case insensitive.

mpdonadio’s picture

Status: Active » Needs review
FileSize
1.56 KB

My last comment in demo test form. Note which of the BTB assertions is the first fail, which should be the one on line 28. Just setting NR for a bot run.

Status: Needs review » Needs work

The last submitted patch, 142: 2735005-142.patch, failed testing.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jhedstrom’s picture

I couldn't find an existent issue on d.o., but prior to converting all of these, it would be really good to have a solution, or a documented workaround to running xdebug within page requests made from functional tests.

There is a PHPUnit PR from @dawehner to PHPUnit to fix this, but that would need to be pulled into core to utilize.

juampynr’s picture

Interesting, I can debug both test code and web requests in functional tests with PHPStorm. Isn't this problem solved by increasing the number of connections in the PHPStorm XDebug settings @jhedstrom? If I am misunderstanding you, please post an example where the debugger won't kick in.

jibran’s picture

Yes, I can confirm that xdebug is working for Unit/Kernal/Browser/JavaScript test but only from cli(phpunit) or phpstrom not sure about run-tests.sh. Just to be on the safe side I raised the number of connections to 5.

jhedstrom’s picture

@juampynr @jibran I've opened #2850891: xdebug breakpoints not working in web requests from BrowserTestBase to discuss further since we are having varying results :)

klausi’s picture

dawehner’s picture

dawehner’s picture

Issue summary: View changes
dawehner’s picture

Issue summary: View changes
dawehner’s picture

Issue summary: View changes
mpdonadio’s picture

Issue summary: View changes
dawehner’s picture

Issue summary: View changes
Lendude’s picture

nlisgo’s picture

mpdonadio’s picture

Issue summary: View changes

I think one of the recent edits lost some changes.

dagmar’s picture

nlisgo’s picture

nlisgo’s picture

scuba_fly’s picture

Issue summary: View changes
Jo Fitzgerald’s picture

Jo Fitzgerald’s picture

GoZ’s picture

Issue summary: View changes
GoZ’s picture

Issue summary: View changes
GoZ’s picture

Issue summary: View changes
GoZ’s picture

Issue summary: View changes
dawehner’s picture

Issue summary: View changes
dawehner’s picture

Issue summary: View changes
slv_’s picture

Issue summary: View changes
nlisgo’s picture

GoZ’s picture

nlisgo’s picture

GoZ’s picture

GoZ’s picture

mpdonadio’s picture

I think this only affects the RDF tests, #2864201: XPath in WebTestBase and BrowserTestBase use different roots. Need to see the lang attribute on <html> is tested .

GoZ’s picture

Statistics and many modules require the conversion of \Drupal\simpletest\AssertContentTrait::assertNoPattern(). Follow #2864257: Convert web tests AssertNoPattern to Browser Test

GoZ’s picture

GoZ’s picture

GoZ’s picture

Issue summary: View changes

fix wrong markup list

GoZ’s picture

Issue summary: View changes
GoZ’s picture

Issue summary: View changes
GoZ’s picture

Issue summary: View changes
nlisgo’s picture

dawehner’s picture

If I understand this correctly we have like 890 to go:

$ find core/modules -print | grep "src/Tests" | wc -l
893
mpdonadio’s picture

quietone’s picture

Issue summary: View changes

Added details for migrate modules.

Jo Fitzgerald’s picture

michielnugter’s picture

Issue summary: View changes

Added note on Wizard tests to IS, separate issue created: #2867777: Convert views WizardTestBase tests to phpunit

michielnugter’s picture

Issue summary: View changes

Added various issues for module conversions in the issue summary

michielnugter’s picture

Issue summary: View changes

All module conversion issues created!

dawehner’s picture

michielnugter Thanks a lot!

michielnugter’s picture

Issue summary: View changes

Link fix.

All new issues that are postponed contain the tests in the IS that it is postponed on.

michielnugter’s picture

Issue summary: View changes

Updated issue summary with the current best approach and some hints/tips on common conversions.

dawehner’s picture

Issue summary: View changes
GoZ’s picture

dawehner’s picture

Issue summary: View changes
dawehner’s picture

We start to make good progress, see the progress report in the issue queue.

mpdonadio’s picture

Issue summary: View changes
mpdonadio’s picture

michielnugter’s picture

Issue summary: View changes
Issue tags: +phpunit initiative

Added 'phpunit initiative' tag to this issue and all it's sub-issues. Will add it to every related issue so we can use the tag to track all issues related to the initiative.

Updated IS to state the tag usage.

dawehner’s picture

Impressive work, seriously!

michielnugter’s picture

Issue summary: View changes

Updating the todo list, good progress!