Updated: Comment #N
Problem/Motivation
In several recent issues, we've come up with areas of Search module functionality that were not covered by the existing tests. I've also noticed that some of the test code we have is redundant.
Proposed resolution
a) Make a list of search functionality that should be tested, including:
- Base search.module functionality
- Node search
- User search
This needs to pay special attention to:
- permissions
- languages
b) Verify that this list of functionality is tested, and that there are not multiple tests for the same functionality. Add and remove tests as needed to make sure the list of existing tests match the list of needed tests.
c) Fix bugs identified by the new tests.
Inventory of functionality to be tested
Here is a list of search functionality that we need to verify has been tested.
Status: I think the list of functionality is pretty complete. Test coverage needs to be filled in.
| Functionality | Description | Tested in | OK or Needs work |
|---|---|---|---|
| Plugin system overall | Modules can define plugins that define search pages |
|
Test coverage is OK. Things to fix:
These are part of #2183951: Fix up some search tests. |
| Search page config/CRUD | Verify that you can use the UI to add/configure/enable/disable/delete search pages based on plugin types, choose the order, and choose a default page. Verify that if you go to /search you get to the set default search page. Also that you can click through to the other tabs, that disabled pages are not shown, and that the pages are in the correct order |
|
Needs tests to cover all the bases - now part of #2183989: Add tests for search admin |
| Search admin permissions | Verify that users without administer search permissions cannot do the search page config/CRUD operations or other search admin operations | Needs tests - now part of #2183989: Add tests for search admin | |
| Search admin page | Verify that you can find the admin page link on admin/config and admin/config/search | Needs tests - now part of #2183989: Add tests for search admin | |
| Search block | Verify that a search from the block goes to the chosen default search page | SearchBlockTest: verifies you go to search/node and that both positive and negative searches work; doesn't test what happens if the default has been changed. | Needs tests for going to the default page - now part of #2183989: Add tests for search admin |
| Search block empty | Verify that if you don't enter keywords you get an appropriate message | SearchBlockTest verifies this. | OK |
| Search block admin | Verify that the search block is available on the Blocks admin page and can be placed | SearchBlockTest: Verifies that the search block is available at admin/structure/block. Uses the drupalPlaceBlock() method to place it, and verifies the block appears on the page. | OK |
| Indexing/querying system functionality | Plugins can add text to the search index and then search with it |
|
OK |
| Indexing API -- search_reindex() function | Should be able to use it to clear one item or the whole index. | Needs tests - now part of #2184003: Add tests for the search_reindex() function | |
| UI: reindex button | Verify that the reindex button from the Search config admin page works | SearchConfigSettingsFormTest tests this | OK |
| Indexing API -- search_mark_for_reindex() function | Verify that it updates the timestamp on an entry in the index |
|
OK |
| Cron overall | Verify all indexable search plugins' updateIndex() methods are called during each cron run. | Some of the tests do run cron and would fail if NodeSearch was not being indexed. Some of the tests call the plugin updateIndex methods directly. | OK |
| Keyword relevance for index | Verify that for searches using the index, ordering by keyword relevance works. |
|
OK |
| Indexing/searching text preprocessing | Verify that the same preprocessing (search_simplify() function) is used to process keywords during indexing and searching, with the following processing steps:
Phew! |
|
SearchNumberMatchingTest and SearchNumbersTest should probably be combined; otherwise this is probably pretty thorough. This is part of #2183951: Fix up some search tests now. |
| Indexing config -- minimum word length and CJK handling | Verify they can be set from the Search Settings page and that the settings work correctly during indexing and during searching | SearchConfigSettingsFormTest - tests that invalid values for word length (not an integer) cannot be saved, and that the default values allow the form to be submitted. Doesn't test that other values can be saved or that they do anything useful. | Needs tests - now part of #2183989: Add tests for search admin |
| Indexing config -- cron throttle | Verify the # of nodes per cron run setting can be set (may be on the search config page or the node page config page), and that if it is set it affects the cron run. |
|
Needs tests to use the UI for the throttle. This is now part of #2178643: Indexing status can show a negative percentage of content as having been indexed |
| API function search_excerpt() | Verify that an excerpt with keywords/phrases highlighted is created:
|
SearchExcerptTest runs a number of test cases, verifying that HTML tags are removed and HTML entities work, that simple keywords are highlighted at the beginning, middle, and end of the text, that it works with multi-byte characters, and that it works with preprocessing-only matches. | Probably OK |
| Node search permissions | Verify that users with/without search content/access content permissions can/cannot search nodes using any configured NodeSearch pages via the UI. Verify that users with/without advanced search permissions cannot use advanced search via the UI or via directly using advanced GET query params. Verify that node access is respected. |
|
Needs tests #2183971: Add tests for Node search and advanced search is for adding access tests. Also, I think the method on SearchNodeAccessTest::testPhraseSearchPunctuation() should be moved elsewhere and we should either then delete this class or make it actually test node access? (see #2183951: Fix up some search tests for that) |
| Node search functionality | Verify that keyword searching in nodes works via the UI, and that a non-matching query gives an appropriate message. |
|
OK |
| Node search Advanced functionality | Verify that the advanced search functionality works correctly via the UI. Should test: positive "or" keywords, phrases, negative keywords, node type filter, language filter |
|
Needs tests for the other Advanced filters.#2161067: Advanced search with Language filter does not work takes care of language filtering. #2183971: Add tests for Node search and advanced search is for the other filters that need testing. |
| Node search international | I'm actually not sure what the correct behavior is supposed to be here? Need to figure out what was intended |
|
Needs definition and test I think we can leave this to #2161067: Advanced search with Language filter does not work |
| Node search with comments | Not sure what should be listed here? |
|
Should add a test for "Add new comment" text not getting into the index or results for the case where anonymous users can make comments. This test may fail: see #1113832: [PP-1] Comment module renders "reply" and other links in search index/results |
| Node search results ordering | The UI can be used to set rankings and they correctly influence search results | SearchRankingTest tests that ranking by sticky, promoted to home page, keyword relevance, most recent, most comments, and most views works. They are each tested as the sole source of rankings, and there is also one combined ranking test. However, due to #893302: Search ranking based on the factor "number of comments" & "No of Page Views" is broken the tests for comments and views are currently omitted from testing. The rankings are set in the UI; search is done by calling methods on the NodeSearch plugin. | OK (once related issue is committed) |
| User search permissions | Verify that users with/without search content/access user profiles permissions can/cannot search users via substrings on user account names, via any configured UserSearch pages via the UI. Verify that users with/without administer users permissions can/cannot search users via email address substrings. | UserSearchTest (in User module lib) tests that users with/without admininister users permission can/cannot search by email, and that users with/without admin permissions can/cannot see blocked users in search results | Needs more tests Now part of #2183963: Add tests for User search |
| User search name functionality | Verify that keyword searching for user account names works via the UI, and that a non-matching query gives an appropriate message. |
|
Needs test Now part of #2183963: Add tests for User search |
| User search email functionality | Verify that keyword searching for user account email addresses works via the UI, and that a non-matching query gives an appropriate message. | Needs test Now part of #2183963: Add tests for User search | |
| Misc. Bug reversion tests |
|
OK |
Remaining tasks
Finish the audit. Update the tests.
User interface changes
None.
API changes
None.
Comments
Comment #1
jhodgdonAdded a couple of notes to the task list
Comment #2
jhodgdonWorking on a list of functionality that needs testing (see summary).
Comment #3
jhodgdonUpdated the list. I think the list of functionality is probably complete. Next step is to inventory the tests.
Comment #4
jhodgdonAdded some to the table. More to do but I don't want to lose this.
Comment #5
jhodgdonMore additions... I'm going through the tests and have tests with names starting with SearchL* through SearchT* left to inventory, plus SearchExcerptTest which I skipped. Probably all I'll get to this week.
Comment #6
jhodgdonFound a little more time. Only 4 more to go through to see what they're testing:
SearchRankingTest
SearchSetLocaleTest
SearchSimplifyTest
SearchTokenizerTest
Comment #7
jhodgdonFinished making an inventory of existing tests, and figured out where we need more testing to be done.
Comment #8
jhodgdonAdded some child issues.
Comment #9
jhodgdonAdded some more child issues. I think this inventory has been done and the required issues have been filed now. I am going to go ahead and mark this meta-issue as "fixed".
Comment #10
jhodgdonDoh! I realized that the User module had a search tests under its lib directory (the ones I had inventoried were all in the Search module lib directory). Also checked the Node lib but I didn't see anything there.