Problem/Motivation
In some cases, tests are wrong because similar links exists.
Example with Tour module, in TourHelpPageTest::verifyHelp(), assertNoLink() on 'Translation' will return 'Interface Translation' and make assert result FALSE, even if the 'Translation' link is not here.
In WebAssert::linkExists() and WebAssert::linkNotExists(), we should call $this->session->getPage()->findAll(); with 'named_exact' instead of ''named'.
This can be done by default for linkNotExists(), but i think some tests with linkExists() expect to search on approximative label.
Proposed resolution
- Replace 'named' by 'named_exact' in
WebAssert::linkNotExists().
- Add option to
AssertLegacyTrait::assertLink() and WebAssert::linkExists() to filter on 'named_exact' in place of 'named'.
Comments
Comment #2
goz commentedComment #3
klausiI think we should use named_exact in our WebAssert because we really want to check that the link name is there standalone, not as part of some other link.
Comment #4
goz commentedEven for
AssertLegacyTrait::assertLink()andWebAssert::linkExists()?Previously trying this made some existing tests fail. See #2767275-16: Convert web tests to browser tests for tour module
Comment #5
goz commentedComment #6
klausiYes, let's try to use named_exact for both and see where the fails are. From that issue I only see that tour tests were failing which are not converted yet anyway.
Comment #7
goz commentedComment #9
goz commentedLet tests define if we want named_exact or named.
Comment #10
klausiI looked into the test fails from #7 and the problem is for example that a link
Manage fields <span class="visually-hidden">(active tab)</span>of course does not match with named_exact when only "Manage fields" is passed in.That makes me think that our current behavior of linkExists() is correct. There might be other parts in a link label (icons, images, hidden spans for accessibility) that are not relevant for testing. So just using the "named" selector which does partial matches seems fine to me.
So we could just change the behavior of linkNotExists() to named_exact, but that also feels wrong. It really should work the same as linkExists() except of being reversed.
Comment #11
dawehnerI would make exact the default, its always better to be strict than less strict. If you don't need the strict behaviour then you'll see it in your test.
In other others explicit > implicit. I believe this is exactly what @klausi wrote before.
Comment #12
goz commentedMaking exact default will break existing tests (see #7). Are we ok with that ?
Comment #13
dawehnerMh, that's tricky, I guess we cannot risk that now.
Another idea I just had, have a method
linkExistsExactto make it a bit easier to read. Any thoughts/opinions about that?Comment #14
mile23The dockblock says: "Passes if a link with the specified label is found." This implies the exact label.
How about we reverse this?
$exactshould default toTRUE, since the expectation is that we're looking for the exact label.Then for special-case failing legacy cases, we should add
FALSEto the assertion with a todo/follow-up.Comment #15
goz commentedWith default to
TRUE, we need to change a lot of tests, which is not the purpose to of #2767275: Convert web tests to browser tests for tour module. Seems we are agree with dawehner in #12 and #13 to avoid that.Dockblock can be fixed to match with new argument.
It's an option.
To also take care of klausi comment #10, may be we should do the same for linkNotExists.
Let's make a patch with this one, so we move a step forward.
Comment #16
dawehnerI like it "obviously". Do you mind adding some test coverage for those?
Comment #17
goz commentedYes, i'll add tests
Comment #18
goz commentedAnd here are tests
Comment #19
dawehnerYou can use
setExpectedExceptionhere. Any reason why not?Comment #20
goz commentedNo reasons except i miss this one.
Comment #21
dawehnerNote: You can specify which exception you expect. Let's do that as we did before :)
Comment #22
mile23Comment #23
Anonymous (not verified) commentedYeah, like this:
Also maybe replace names:
on
It will be better to match the name of the tested method:
linkNotExistsExact().Comment #24
jofitzSpecified expected exception.
Edited function names to match tested method.
Comment #25
Anonymous (not verified) commentedThanks, looks nice! All feedback has been addressed.
Comment #26
dawehnerNice! I think this is good to go now
Comment #27
larowlandded @dawehner to issue credits for numerous reviews.
Added @klausi for investigating test fails and providing reviews.
Committed as e562b4e and pushed to 8.4.x.
Unpostponed #2767275: Convert web tests to browser tests for tour module