Problem/Motivation
core/tests/Drupal/Nightwatch/Tests/claroAutocompleteTest.js passes in our Nightwatch job on GitLab.
Yet, when we look at it's console log in the saved artifact after the test, we see:
"http://localhost/subdirectory/form-test/autocomplete-1?q=123 - Failed to load resource: the server responded with a status of 404 (Not Found)"
The original IS below already found the solution for the 404 problem:
In core/modules/system/tests/modules/form_test/form_test.routing.yml, how is it possible that these invalid route definitions (use of controller: with no underscore instead of _controller:) don't cause test failures when form_test is used in general and when the autocomplete test is performed in particular?
The test itself isn't affected by the 404. Since it's only testing the layout of the "processing" phase of the autocomplete, it doesn't need a response. In fact this situation is beneficial, since the "processing" phase is currently endless, due to the lack of any response that would end it.
Nevertheless, this is not code we want to have as an example in core.
Also, this form might be used elsewhere someday, by somebody expecting it to actually works.
So let's fix it.
Steps to reproduce
Go to the Nightwatch job of a GitLab CI run, browse the Job artifacts and under nightwatch_output/consoleLogs/Tests/claroAutocompleteTest find the JSON file.
It will contain the key message with value "http://localhost/subdirectory/form-test/autocomplete-1?q=123 - Failed to load resource: the server responded with a status of 404 (Not Found)".
I've attached an example JSON from a random run to the issue.
Proposed resolution
- Fix the invalid route definitions
- Test the return of the autocomplete in claroAutocompleteTest
- Add some comments in claroAutocompleteTest to explain what's actually being tested.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
| Comment | File | Size | Author |
|---|---|---|---|
| #22 | Test-Claro-autocomplete_PASSED_Sat-Jan-06-2024-13_41_33-GMT+0000-(Coordinated-Universal-Time)_console.json_.txt | 284 bytes | spokje |
Issue fork drupal-2651798
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 2651798-claroautocompletetest-passes-but
changes, plain diff MR !6053
Comments
Comment #2
cilefen commentedThe issue with this commit was #2056627: Form API autocomplete is broken for routes.
Comment #3
cilefen commentedPoking around, it looks as though no tests ever try those routes:
Comment #4
cilefen commentedWell if we bother patching unused test code (see #3), here it is...
Comment #6
surbz commentedPatch applies clean on 8.2.x-dev.
Comment #7
alexpottWell the routes are used:
So they should be working - in fact maybe the test should be changed to a FunctionalJavascriptTest so we can test the auto complete calling beyond just the existence of the links and autocomplete.js being loaded.
I think we should do that here instead so the trivial fix is actually tested.
Comment #15
tr commentedComment #19
quietone commentedTests were asked for n 2016, adding tag.
Comment #21
spokjeClosed #3412895: claroAutocompleteTest passes, but log shows a 404 as a duplicate and going to repurpose this issue to preserve the original credits.
Comment #22
spokjeComment #23
spokjeComment #24
spokjeComment #25
spokjeComment #27
spokjeComment #28
spokje@alexpott in #7
I disagree.
I would _love_ to convert every Nightwatch test to a FunctionalJavascriptTest, and drop Nightwatch altogether, but that's not very feasible.
Basically, we're a bit spoiled by FunctionalJavascriptTests, which not only test the (JS)-Frontend, but also look at backend errors.
Nightwatch doesn't. If the webserver spews errors in the range from 500 to 911, is on fire and with its last dying breath serves a page that meets the assertations, it's perfectly happy.
That's why
claroAutocompleteTestdoesn't care about the invalid routes, it only asserts certain specific layout features of a autocomplete which is in progress. The result of the autocomplete is irrelevant and as proven, not even necessary.The test is not about the inner workings of autocomplete, nor dependent on it even working.
Like it or not, somebody somewhere decided to use a Nightwatch test for this, and I think we should stick with that.
Having said that: We should fix the routing, make the autocomplete work and test this in the Nightwatch test itself.
The latter part didn't exist in the test until now and is added.
I hope this explanation makes any kind of sense?
Comment #29
spokjeComment #30
smustgrave commentedConfirmed the issue by going by going to the last ticket merged
https://git.drupalcode.org/issue/drupal-3409587/-/jobs/505831/artifacts/...(Coordinated-Universal-Time)_console.json
Checked this MR https://git.drupalcode.org/issue/drupal-2651798/-/jobs/587545/artifacts/...(Coordinated-Universal-Time)_console.json
And no issue.
Comment #31
alexpottCommitted and pushed a4d53f09f6 to 11.x and e265a01324 to 10.2.x. Thanks!
Backported to 10.2.x since this only involved tests.