Problem/Motivation

Now that #3390193: Add a drupalGet() method to KernelTestBase is in, it's possible to convert straightforward functional tests (no form submissions, no session data) to kernel tests.

The biggest advantage of this is test speed, which can easily be ten times quicker, but kernel tests can also be easier to debug, because you don't have separate http requests with their own php execution to worry about.

On the other hand we need to make sure we're not losing important test coverage in the conversions, e.g. where the separate http request and full Drupal bootstrap is important.

Steps to reproduce

Proposed resolution

I think we should pick a small number of tests to convert first, to have the biggest impact, we can choose some slow tests with lots of test methods, this will help us to get an idea how much we can reduce pipeline times, and no particular reason those tests should be easier or harder to convert than any others.

In the process of that, we can see how much change is required for each test to convert, and that should help inform how to scope things when we're converting a larger number of tests.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Comments

catch created an issue. See original summary.

catch’s picture

Here's some potential candidates, I did not vet them for form submissions or session data, although from memory most should be OK.

The run I used for times is https://git.drupalcode.org/issue/drupal-3581111/-/pipelines/783025

 167.763s Drupal\Tests\jsonapi\Functional\UserTest                                 18 passed

This will require a kernel test version of ResourceTestBase - but would allow us to convert a lot of slow tests more or less in one go.

106.464s Drupal\Tests\rest\Functional\Views\StyleSerializerTest                    6 passed
  98.031s Drupal\Tests\jsonapi\Functional\JsonApiRegressionTest                    16 passed
 181.637s Drupal\Tests\layout_builder\Functional\LayoutBuilderTest                 16 passed
 119.434s Drupal\Tests\update\Functional\UpdateSemverCoreBaselineTest               6 passed
 111.763s Drupal\Tests\block\Functional\BlockTest                                  15 passed
andypost’s picture

joachim’s picture

Title: Convert suitable functional tests to kernel tests » Convert functional tests with suitably simple HTTP requests to kernel tests
joachim’s picture

Drupal\Tests\layout_builder\Functional\LayoutBuilderTest is testing form submissions, so not suitable.

Drupal\Tests\update\Functional\UpdateSemverCoreBaselineTest uses checkForMetaRefresh(), which we can *probably* add to the kernel test UI trait?

Drupal\Tests\jsonapi\Functional\UserTest will need #3582228: add $options and $headers parameters to HttpKernelUiHelperTrait::drupalGet()

Drupal\Tests\rest\Functional\Views\StyleSerializerTest - same, needs to pass options to the request.

Drupal\Tests\block\Functional\BlockTest needs looking at -- some of the inline comments in the test make it sound like the admin form is being tested as well as the result -- which means the test should be in BlockUiTest instead. Needs unpicking.

JsonApiRegressionTest - the test methods which make GET requests should be ok.

mstrelan’s picture

There are two I opened while reviewing the original issue.

#3519393: Convert functional tests in navigation module to kernel tests
#3517299: Convert functional tests in help module to kernel tests

Happy if someone wants to rebase those and see if they're suitable.

dww’s picture

Do we want to make this a meta and add child issues? I guess this is already 'Plan', so we're not intending to actually move anything in here.

catch’s picture

Yeah child issues here is good, just as above I think it would help everyone if we focus on getting a small number of issues in first to iron things out before opening a large backlog.

dww’s picture

dww’s picture

And opened #3582581: Move UpdateContribTest.php to a Kernel test as a child of this. Starting to look into some of the details, I think that's a better candidate for the initial update.module Functional test to move than UpdateSemverCoreBaselineTest (which is all tangled up with other tests via UpdateSemverTestBaselineTrait 😬).

mstrelan’s picture

Opened #3583587: Convert some functional tests in system module to kernel tests which converts 11 tests and has a 79.4% reduction in run time on my local env

joachim’s picture

larowlan’s picture

What about stuff that extends

https://git.drupalcode.org/project/drupal/-/blob/main/core/modules/rest/...

Those are a big chunk and don't have forms

Similar for jsonapi

mstrelan’s picture

@larowlan I don't think there are any drupalGet calls there? Do you suggest we should use the new trait instead of ApiRequestTrait? I suspect most child classes would need a fair amount of setup to be converted to a kernel test, so we'd need a duplicate of ResourceTestBase, EntityResourceTestBase, etc, to start working on them.

joachim’s picture

> I suspect most child classes would need a fair amount of setup to be converted to a kernel test, so we'd need a duplicate of ResourceTestBase, EntityResourceTestBase, etc,

Yup, I've started digging into that at #3587035: convert jsonapi\Functional\UserTest to a kernel test and there's a lot to do. But what we figure out for the jsonapi module tests should translate to the rest module tests.

joachim’s picture

I'm writing up some rough notes on the process to help others who may want to try this conversion on other tests:

1. Copy the class rather than move it - while working, you may need to run the original tests to understand how they are doing something.
2. Change the base class to KTB. If the test uses a specialised base class (as in other than BTB), you might want do one of these instead:
-- add a new base class too
-- copy things you find you need from the base class into the test class itself.
-- create a new trait that can be used by both the browser tests you're not changing and the new kernel test
In all approaches, I prefer to copy methods across as I find I need them, rather than everything at once. You may find you change which approach you take, e.g. if you find that most of the helper methods need changing as well.
3. remove the $defaultTheme
4. review the modules - you will need to explicitly add dependencies. Simplest thing is to pick one test method and run it and see how it crashes ;)
5. add setup. you will likely need to set up entity types, tables, and config. Tests can sometimes expect default config settings, for example. See https://www.drupal.org/docs/develop/automated-testing/phpunit-in-drupal/... for a crib sheet. Again, run the test and failures will often show what you're missing -- missing tables, entity types, plugins, services, etc.

Test failures, I have found, are generally due to one of the following:

- missing setup that the browser test got and the kernel test needs to be done explicitly. If values are not as expected, it can be missing config settings, for example.
- SUT code that is remembering state between requests. If a class has a $this->something = a config value or something based on the request, then that won't be getting reset in a kernel test, because we're all in the same request. This will need changing -- either don't persist the value, or add a method that clears it. It's something that will need fixing for #2218651: [meta] Make Drupal compatible with persistent app servers like ReactPHP, PHP-PM, PHPFastCGI, FrankenPHP, Swoole anyway!
- things to do with the change in the browser being used. With simple drupalGet() calls this shouldn't be a problem, but more complex uses might need adapting.