Apparently functional tests are now the preferred way of testing the UI. We should therefore try to port our existing web tests to those. (According to Joris, it can incredibly simple in a lot of cases.)

Then we can also think about expanding those tests with JS checks in the course of #2753807: Add functional Javascript tests.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

drunken monkey created an issue. See original summary.

borisson_’s picture

Title: Port Simpletest web tests to functional Javscript tests » Port Simpletest web tests to BrowserTestBase tests
Issue summary: View changes

Web tests should extend from the BrowserTestBase base class.

niko-’s picture

Status: Active » Needs review
FileSize
25.12 KB
borisson_’s picture

Status: Needs review » Needs work

I don't think that's a move of all the tests? Also, it'd be great to make the patch with git diff -M > patchname.path, as that makes the patch much easier to grok.

Thanks for working on this!

itsekhmistro’s picture

Hi guys,

Checked the patch from nico comment #3 https://www.drupal.org/files/issues/initial-functional-tests-2753815-3.p...
and slightly updated it, I would like to outline some details here:

* (1) Fixed checkForMetaRefresh() method in WebTestbase - now it does as supposed to be = checks for redirects in recursion (allows to pass batches).
* (2) There is really an issue with "Index now" form, due to
<summary role="button" aria-controls="edit-index" aria-expanded="true" aria-pressed="true">Index now</summary>
this element is confusing for submitForm() method in BrowserTestBase.
$this->drupalPostForm(NULL, array(), $this->t('Index now'));
The real submit button is
<input data-drupal-selector="edit-index-now" type="submit" id="edit-index-now" name="index_now" value="Index now" class="button js-form-submit form-submit">
* (3) Ported LanguageIntegrationTest (passes 100%) and IntegrationTest , this is not final patch there are still several Tests to port.

For (2) fixed by renaming element title to "'Start indexing now". What do you think about it?

The alternative would be to pass element name into
$this->drupalPostForm(NULL, array(), 'index_now');
But it doesn't sound good as for me.

borisson_’s picture

Status: Needs work » Needs review

Setting to NR for the bots, will respond to #5 later.

Status: Needs review » Needs work

The last submitted patch, 5: 2753815-functional-tests.patch, failed testing.

niko-’s picture

Status: Needs work » Needs review
FileSize
30.59 KB
87.76 KB

Thanks for your help.

Updated patch includes:
CacheabilityTest
HooksTest
IntegrationTest - not finished
LanguageIntegrationTest
OverviewPageTest
ProcessorIntegrationTest

WebTestBase was renamed to SearchApiBrowserTestBase
ExampleContentTrait was moved to namespace Drupal\Tests\search_api

Status: Needs review » Needs work

The last submitted patch, 8: 2753815-8.patch, failed testing.

niko-’s picture

All search_api simpletest tests are moved to functional tests.
Current tests status:
All existing Kernel, Unit tests have passed with ../vendor/bin/phpunit

Functional tests status:
CacheabilityTest - passed
HooksTest - passed
IntegrationTest - Passed
- all things that need detail review was marked with @todo
- calls drupalPostAjaxForm - need discussing and port to javascript fucntional tests (commented now)
LanguageIntegrationTest - passed
OverviewPageTest - passed
LocalActionsWebTest - passed
ProcessorIntegrationTest - passed
ViewsTest - not passed
- need proper migration of installModulesFromClassProperty method that not exists in BrowserTestBase

niko-’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 10: 2753815-10.patch, failed testing.

borisson_’s picture

  1. +++ b/tests/src/Functional/IntegrationTest.php
    @@ -289,31 +295,33 @@ class IntegrationTest extends WebTestBase {
    +    // @todo Do we really need this if?
    +    if ($index instanceof IndexInterface) {
    

    Not really, we could just remove that, and the else

  2. +++ b/tests/src/Functional/IntegrationTest.php
    @@ -335,20 +343,21 @@ class IntegrationTest extends WebTestBase {
    +    // @todo Commented now. need move to js fuctional tests?
    +    // $this->drupalPostAjaxForm(NULL, $edit, array('datasources_configure' => t('Configure')));
    +    // $this->drupalPostForm(NULL, $edit, $this->t('Save'));
    +    // $this->assertSession()->pageTextContains($this->t('The machine-readable name is already in use. It must be unique.'));
    

    Is there no other way to test this behavior other than a js test? That'd be very unfortunate, JS tests take a lot longer.

  3. +++ b/tests/src/Functional/IntegrationTest.php
    @@ -393,32 +403,35 @@ class IntegrationTest extends WebTestBase {
    +    // @todo commented now. Need move to js tests?
    +    // $this->drupalPostAjaxForm(NULL, $edit, 'tracker_configure');
    

    See above. Not testing these would lead to a significant regression in the tests-suite as it is right now.

  4. +++ b/tests/src/Functional/IntegrationTest.php
    @@ -746,24 +772,30 @@ class IntegrationTest extends WebTestBase {
    +    // @todo Do we need this if?
    +    if (!empty($fields['uid'])) {
    ...
    +    // @todo Do we need this if?
    +    if (!empty($fields['status'])) {
    

    I think we need to keep those

I'll go trough the rest of the patch after work.

borisson_’s picture

I just ran the integration test locally and that did seem to pass. So not sure why the bots are so confused about this patch.

We do need to find a way to replace the drupalPostAjaxForm calls. Not sure how/if the drupal core tests have done that when they converted to BTB.

niko-’s picture

Status: Needs work » Needs review
FileSize
162.19 KB
92.75 KB

Status: Needs review » Needs work

The last submitted patch, 15: 2753815-15.patch, failed testing.

borisson_’s picture

Woo - no longer errors on drupal-ci! Now all we need to do is fix the remaining fails.

niko-’s picture

Status: Needs work » Needs review
FileSize
162.43 KB
5.29 KB

Details:
- LocalActionsWebTest was marged into OverviewPageTest and removed
- @todos was processed according #13
- search_api_db test was moved to Functional
-

Drupal\Tests\search_api\ExampleContentTrait

was moved to

Drupal\Tests\search_api\Functional\ExampleContentTrait

because run-tests.sh can't preper configure autoload. It seems

drupal_phpunit_get_extension_namespaces()

not called and this issue led to CI error.
- convert arrays to short syntax

Remaining tasks:
ViewsTest - not passed
- need proper migration of installModulesFromClassProperty method that not exists in BrowserTestBase

Status: Needs review » Needs work

The last submitted patch, 18: 2753815-18.patch, failed testing.

borisson_’s picture

Yeah, the autoloader is a known issue; there's a core issue available. (I don't remember the issue by heart, but I pinged @dawehner on irc to ask about that).
I'll have a look at installModulesFromClassProperty later.

niko-’s picture

Status: Needs work » Needs review
FileSize
10.32 KB
189.67 KB

installModulesFromClassProperty() migration.
It seems all simpletest tests was migrated to functional one.

niko-’s picture

modules/search_api_db/search_api_db_defaults/src/Tests/IntegrationTest.php moved to
modules/search_api_db/search_api_db_defaults/tests/src/Functional/IntegrationTest.php
Found strange thing with this test \Drupal\Tests\search_api_db_defaults\IntegrationTest::testInstallAndDefaultSetupWorking() is protected, not sure is it ok or not?

modules/search_api_db/tests/src/Functional/IntegrationTest.php moved to modules/search_api_db/tests/src/FunctionalJavascript/IntegrationTest.php

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

I think these changes look good - the diff is really big. @drunken monkey how are you feeling about these changes?

drunken monkey’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
28.02 KB
198.72 KB

Amazing job, thanks a lot! Looks really good, on the whole.

Since the tests now use PhpUnit, and thus abort at the first fail anyways, no, we don't need those ifs anymore.

Had a few other remarks, corrections and nit-picks. Should now all be good, more or less. I only have one error, in the DB Defaults module's integration test: there, the form submit for enabling the modules takes more than 30 seconds, thus terminating in a timeout and throwing an exception. I tried to find out how I can increase the timeout, but since the architecture seems to be "BrowserTestBase -> Mink -> BrowserKit -> Goutte -> Guzzle -> Curl" I couldn't really see a way to do that. (Sometimes, D8 is just insane.)
Let's see, how the test bot fares. Otherwise, maybe we could try splitting that up into three separate form submissions, one for each module.

Integrating LocalActionsWebTest into OverviewPageTest is a good idea, thanks!

Having to override installDrupal() in our ViewsTest is really ugly, but I don't really see any alternative either. At least it seems Core is already working on it: #2796105: Move similar methods in BrowserTestBase / WebTestBase to a trait; untangle installDrupal().

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

  • drunken monkey committed f294693 on 8.x-1.x authored by niko-
    Issue #2753815 by niko-, drunken monkey, itsekhmistro, borisson_: Ported...
drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

Great, thanks a lot for reviewing!
Committed.
Thanks again for your excellent work here, niko-!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.