Problem/Motivation

Through #3046959: Convert RelationshipNodeTermDataTest into a Kernel test I/we discovered that there are Functional tests that make no HTTP requests but are still a Functional test.

The overhead for a Functional test is way more than for a Kernel test, so converting tests that are eligible will make the overall test time go down.

Steps to reproduce

I've created the below list with eligible candidates by comparing the list with tests and number of passed assertions on top of each of the seven Functional test jobs on Gitlab (Test summary) with the list of *.counter files in /artifacts/sites/simpletest/browser_output of the artifact of the same testjob.

Reasoning here is that when there's a *.counter file for a Functional test, HTML output was logged and thus a HTTP request was made.

Present in the list of tests that ran, but no *.counter file means no HTTP request was made and thus the test might be eligible for converting into a Kernel test.

Eligible Functional test for conversion Action taken
Drupal\FunctionalTests\Core\Config\SchemaConfigListenerTest This tests that Functional tests have schema checking turned on by default, so needs to be a Functional test.
Drupal\FunctionalTests\Core\Container\ServiceDeprecationTest From class PHPDoc: "Note: this test must be a BrowserTestBase so the container is properly compiled.
Drupal\FunctionalTests\Core\Test\ModuleInstallBatchTest #3045736: Convert ModuleInstallBatchTest into a Kernel test
Drupal\FunctionalTests\FolderTest Tests hook_install() during setup of image module. Needs to run on serialized container.
Drupal\FunctionalTests\GetTestMethodCallerExtendsTest Explicit test for BrowserTestBase::getTestMethodCaller() in child classes. Needs to extend BrowserTestBase.
Drupal\FunctionalTests\GetTestMethodCallerTest Explicit test for BrowserTestBase::getTestMethodCaller() Needs to extend BrowserTestBase.
Drupal\FunctionalTests\Installer\DrupalFlushAllCachesInInstallerTest Uses drupal_flush_all_caches(). Needs to run on serialized container.
Drupal\FunctionalTests\Installer\InstallerDeprecatedDriverNameTest Tests a deprecation which will be removed in 11.0. Also looks like it actually needs a serialized container. Most probably not worth the effort trying to convert..
Drupal\FunctionalTests\Installer\InstallerPerformanceTest This tests an actual install, so needs to be a Functional test.
Drupal\FunctionalTests\Installer\TestingProfileInstallTest This tests an actual install, so needs to be a Functional test.
Drupal\FunctionalTests\MailCaptureTest #3042994: MailCaptureTest provides the same test coverage as AssertMailTraitTest and should be removed
Drupal\FunctionalTests\Routing\LazyRouteProviderInstallTest Tests that the lazy route provider is used during a module install. Needs to run on serialized container.
Drupal\Tests\block_content\Functional\Views\FieldTypeTest #3414259: Convert FieldTypeTest into a Kernel test
Drupal\Tests\config\Functional\ConfigInstallProfileOverrideTest
Drupal\Tests\config\Functional\ConfigOtherModuleTest #3415612: Convert ConfigOtherModuleTest into a Kernel test
Drupal\Tests\contact\Functional\Rest\MessageXmlAnonTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\contact\Functional\Rest\MessageXmlBasicAuthTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\contact\Functional\Rest\MessageXmlCookieTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\editor\Functional\Rest\EditorJsonAnonTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\image\Functional\FileMoveTest #3415038: Convert FileMoveTest into a Kernel test
Drupal\Tests\image\Functional\Rest\ImageStyleXmlAnonTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\image\Functional\Rest\ImageStyleXmlBasicAuthTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\image\Functional\Rest\ImageStyleXmlCookieTest All classes extending \Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase should be Functional tests.
Drupal\Tests\locale\Functional\LocaleNonInteractiveDevInstallTest Test class is skipped due to due to major version-specific logic. See https://www.drupal.org/project/drupal/issues/3359322. Otherwise it would do a HTTP request.
Drupal\Tests\locale\Functional\LocaleNonInteractiveInstallTest est class is skipped due to due to major version-specific logic. See https://www.drupal.org/project/drupal/issues/3359322. Otherwise it would do a HTTP request.
Drupal\Tests\menu_ui\Functional\MenuUninstallTest Tests that uninstalling menu does not remove custom menus. Needs to run on serialized container.
Drupal\Tests\mysql\Functional\RequirementsTest #3414957: \Drupal\Tests\mysql\Functional\RequirementsTest is always skipped
Drupal\Tests\node\Functional\NodeTemplateSuggestionsTest #3414499: Convert NodeTemplateSuggestionsTest into a Kernel test
Drupal\Tests\node\Functional\Views\PathPluginTest #3414481: Convert PathPluginTest into a Kernel test
Drupal\Tests\node\Functional\Views\RowPluginTest #3415557: Convert RowPluginTest into a Kernel test
Drupal\Tests\options\Functional\OptionsDynamicValuesApiTest #3414501: Convert OptionsDynamicValuesApiTest into a Kernel test
Drupal\Tests\options\Functional\OptionsDynamicValuesValidationTest #3414557: Convert OptionsDynamicValuesValidationTest into a Kernel test
Drupal\Tests\pgsql\Functional\Database\PostgreSqlSequenceUpdateTest This test will only run in jobs that are using PostgreSQL as a DB. I've checked and then it makes HTTP calls.
Drupal\Tests\search\Functional\SearchSetLocaleTest #3414589: Convert SearchSetLocaleTest into a Kernel test
Drupal\Tests\system\Functional\Cache\ClearTest Tests our clearing is done the proper way. Needs to run on serialized container.
Drupal\Tests\system\Functional\Common\AlterTest #3415060: Convert AlterTest into a Kernel test
Drupal\Tests\system\Functional\Entity\EntityReferenceSelection\EntityReferenceSelectionAccessTest #3414776: Move EntityReferenceSelectionAccessTest to Kernel namespace and directory
Drupal\Tests\system\Functional\FileTransfer\FileTransferTest #3419114: Convert FileTransferTest into a Kernel test
Drupal\Tests\system\Functional\Mail\HtmlToTextTest #3414914: Convert HtmlToTextTest into a Unit test
Drupal\Tests\system\Functional\System\SitesDirectoryHardeningTest Needs to run on serialized container.
Drupal\Tests\system\Functional\Theme\TwigDebugMarkupTest Uses \Drupal\Core\Test\FunctionalTestSetupTrait::rebuildContainer
Drupal\Tests\system\Functional\Theme\TwigLoaderTest #3414969: Convert TwigLoaderTest into a Kernel test
Drupal\Tests\system\Functional\UpdateSystem\DependencyHookInvocationTest Uses update_retrieve_dependencies which needs to run on serialized container.
Drupal\Tests\system\Functional\UpdateSystem\DependencyMissingTest Uses update_resolve_dependencies which needs to run on serialized container.
Drupal\Tests\system\Functional\UpdateSystem\DependencyOrderingTest Uses update_resolve_dependencies which needs to run on serialized container.
Drupal\Tests\taxonomy\Functional\Views\RelationshipNodeTermDataTest #3046959: Convert RelationshipNodeTermDataTest into a Kernel test
Drupal\Tests\taxonomy\Functional\Views\RelationshipRepresentativeNodeTest #3414664: Convert RelationshipRepresentativeNodeTest into a Kernel test
Drupal\Tests\taxonomy\Functional\Views\TaxonomyTermFilterDepthTest #3414667: Delete TaxonomyTermFilterDepthTest Functional test
Drupal\Tests\views\Functional\Handler\FieldTimeIntervalTest #3414699: Convert FieldTimeIntervalTest into a Kernel test
Drupal\Tests\views\Functional\Handler\HandlerAllTest #3041755: Convert HandlerAllTest into a kernel test
Drupal\Tests\views\Functional\Plugin\CacheTagTest #3041813: Convert CacheTagTest into a kernel test

Proposed resolution

Look at each Functional test in the list and determine if it can be converted in a Kernel test.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

Spokje created an issue. See original summary.

spokje’s picture

Issue summary: View changes
spokje’s picture

Title: [META] Convert eligible Functional tests into Kernel tests » [META] Convert Functional tests which make no HTTP requests into Kernel tests
Issue summary: View changes
Parent issue: » #3041700: [META] Convert some tests into Kernel or Unit tests
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
longwave’s picture

I wrote a test patch a while back over in https://www.drupal.org/files/issues/2022-02-02/btb-visitcount_0.patch that might be useful to introduce for real once we have solved all these.

spokje’s picture

Nice, but not sure all of the listed ones are solvable, there are already 2 'won't fixes' in the list.

Do we want to force making a HTTP call even when the test doesn't really need it?
Might be a good plan, the extra "effort" might be minimal and we would prevent a lot of Functional-tests-that-are-actually-Kernel-ones to slip in.

spokje’s picture

Issue summary: View changes
longwave’s picture

If we use a similar solution to the test patch we could just do $this->visitCount++; to fake an HTTP request (a bit like we call $this->addToAssertionCount(1); which I'm not sure we actually need any more)

spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
catch’s picture

#10 are the won't fixes because those tests do make an http request after all, or for some other reason?

spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

spokje’s picture

#10 are the won't fixes because those tests do make an http request after all, or for some other reason?

Other reason, mostly that the test needs to run on an a serialized container.

spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
mstrelan’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
hardik_patel_12’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
hardik_patel_12’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

Title: [META] Convert Functional tests which make no HTTP requests into Kernel tests » [META] Convert Functional tests classes which make no HTTP requests into Kernel tests
spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes
spokje’s picture

spokje’s picture

Issue summary: View changes
spokje’s picture

Issue summary: View changes

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

catch’s picture

I think NodeAccessGrantsCacheContextTest is a candidate here - possibly for a unit test rather than a kernel test.