Problem/Motivation
Now that #3390193: Add a drupalGet() method to KernelTestBase is in, it's possible to convert straightforward functional tests (no form submissions, no session data) to kernel tests.
The biggest advantage of this is test speed, which can easily be ten times quicker, but kernel tests can also be easier to debug, because you don't have separate http requests with their own php execution to worry about.
On the other hand we need to make sure we're not losing important test coverage in the conversions, e.g. where the separate http request and full Drupal bootstrap is important.
Steps to reproduce
Proposed resolution
I think we should pick a small number of tests to convert first, to have the biggest impact, we can choose some slow tests with lots of test methods, this will help us to get an idea how much we can reduce pipeline times, and no particular reason those tests should be easier or harder to convert than any others.
In the process of that, we can see how much change is required for each test to convert, and that should help inform how to scope things when we're converting a larger number of tests.
Comments
Comment #2
catchHere's some potential candidates, I did not vet them for form submissions or session data, although from memory most should be OK.
The run I used for times is https://git.drupalcode.org/issue/drupal-3581111/-/pipelines/783025
This will require a kernel test version of
ResourceTestBase- but would allow us to convert a lot of slow tests more or less in one go.Comment #3
andypostComment #4
andypostnot a parent
Comment #5
joachim commentedComment #6
joachim commentedDrupal\Tests\layout_builder\Functional\LayoutBuilderTest is testing form submissions, so not suitable.
Drupal\Tests\update\Functional\UpdateSemverCoreBaselineTest uses checkForMetaRefresh(), which we can *probably* add to the kernel test UI trait?
Drupal\Tests\jsonapi\Functional\UserTest will need #3582228: add $options and $headers parameters to HttpKernelUiHelperTrait::drupalGet()
Drupal\Tests\rest\Functional\Views\StyleSerializerTest - same, needs to pass options to the request.
Drupal\Tests\block\Functional\BlockTest needs looking at -- some of the inline comments in the test make it sound like the admin form is being tested as well as the result -- which means the test should be in BlockUiTest instead. Needs unpicking.
JsonApiRegressionTest - the test methods which make GET requests should be ok.
Comment #7
mstrelan commentedThere are two I opened while reviewing the original issue.
#3519393: Convert functional tests in navigation module to kernel tests
#3517299: Convert functional tests in help module to kernel tests
Happy if someone wants to rebase those and see if they're suitable.
Comment #8
dwwDo we want to make this a meta and add child issues? I guess this is already 'Plan', so we're not intending to actually move anything in here.
Comment #9
catchYeah child issues here is good, just as above I think it would help everyone if we focus on getting a small number of issues in first to iron things out before opening a large backlog.
Comment #10
dwwSpun off #3582580: add checkForMetaRefresh() method to HttpKernelUiHelperTrait as a blocker to some of these...
Comment #11
dwwAnd opened #3582581: Move UpdateContribTest.php to a Kernel test as a child of this. Starting to look into some of the details, I think that's a better candidate for the initial update.module Functional test to move than
UpdateSemverCoreBaselineTest(which is all tangled up with other tests viaUpdateSemverTestBaselineTrait😬).Comment #12
mstrelan commentedOpened #3583587: Convert some functional tests in system module to kernel tests which converts 11 tests and has a 79.4% reduction in run time on my local env
Comment #13
joachim commentedMade a start on #3587035: convert jsonapi\Functional\UserTest to a kernel test.
Comment #14
larowlanWhat about stuff that extends
https://git.drupalcode.org/project/drupal/-/blob/main/core/modules/rest/...
Those are a big chunk and don't have forms
Similar for jsonapi
Comment #15
mstrelan commented@larowlan I don't think there are any drupalGet calls there? Do you suggest we should use the new trait instead of ApiRequestTrait? I suspect most child classes would need a fair amount of setup to be converted to a kernel test, so we'd need a duplicate of ResourceTestBase, EntityResourceTestBase, etc, to start working on them.
Comment #16
joachim commented> I suspect most child classes would need a fair amount of setup to be converted to a kernel test, so we'd need a duplicate of ResourceTestBase, EntityResourceTestBase, etc,
Yup, I've started digging into that at #3587035: convert jsonapi\Functional\UserTest to a kernel test and there's a lot to do. But what we figure out for the jsonapi module tests should translate to the rest module tests.
Comment #17
joachim commentedI'm writing up some rough notes on the process to help others who may want to try this conversion on other tests:
1. Copy the class rather than move it - while working, you may need to run the original tests to understand how they are doing something.
2. Change the base class to KTB. If the test uses a specialised base class (as in other than BTB), you might want do one of these instead:
-- add a new base class too
-- copy things you find you need from the base class into the test class itself.
-- create a new trait that can be used by both the browser tests you're not changing and the new kernel test
In all approaches, I prefer to copy methods across as I find I need them, rather than everything at once. You may find you change which approach you take, e.g. if you find that most of the helper methods need changing as well.
3. remove the $defaultTheme
4. review the modules - you will need to explicitly add dependencies. Simplest thing is to pick one test method and run it and see how it crashes ;)
5. add setup. you will likely need to set up entity types, tables, and config. Tests can sometimes expect default config settings, for example. See https://www.drupal.org/docs/develop/automated-testing/phpunit-in-drupal/... for a crib sheet. Again, run the test and failures will often show what you're missing -- missing tables, entity types, plugins, services, etc.
Test failures, I have found, are generally due to one of the following:
- missing setup that the browser test got and the kernel test needs to be done explicitly. If values are not as expected, it can be missing config settings, for example.
- SUT code that is remembering state between requests. If a class has a $this->something = a config value or something based on the request, then that won't be getting reset in a kernel test, because we're all in the same request. This will need changing -- either don't persist the value, or add a method that clears it. It's something that will need fixing for #2218651: [meta] Make Drupal compatible with persistent app servers like ReactPHP, PHP-PM, PHPFastCGI, FrankenPHP, Swoole anyway!
- things to do with the change in the browser being used. With simple drupalGet() calls this shouldn't be a problem, but more complex uses might need adapting.
Comment #18
mstrelan commentedOpened #3588363: Convert some tests in Drupal\FunctionalTests namespace to kernel tests