Make tests more resilient to changes in Drupal core

Some initial thoughts here, first looking at the code base I think the best approach would be to address testing in a way that relies as little on core as possible. Right now the tests use real json output, which can result in problems. If we better test individual components then we can probably avoid many of the high level validation requirements. For example using Unit and Kernel tests to target the key functions and validate that the json logic is being built correctly, would make tests more flexible.

There are overall improvements to the module like leveraging plugins for setting up routing/endpoints. Then tests could be built to validate the key components of the resulting schema, and individual tests for the jsonapi or rest components that may be unique. Taking that approach has some larger requirements and refactoring that would need to be completed prior to solving tests, and should be handled after the initial test refactor.

I think adding and testing our own entity type is a safe approach as it lets us focus on the resulting structure more then relying on a core feature to show all functions of the module. However, yes there are still cases where core may differ between versions, such as the security section you mentioned, and we need to account for those. I think having the differentiated types of tests can get us around many of those rigid requirements.

Still trying to flush out a few thoughts about this. I am sure you have more experience with tests in general, so if you have any suggestions, would be good to hear them.

Took a first pass over the tests. The functionality I targeted first with the host and base path definitions. The use of string replacements was causing problems running on my https dev box. Ultimately php has the parse_url function for doing this, so its silly for us to attempt this in a way that imposes limitations. I converted both the host and base path detection logic to use it. There are also cases in the Generators where we may wish to use the same logic, but we should come back to that after the tests.

With the improved test cases, it seemed silly to store {host} in the expectation files since we will just be replacing it. The same applies to {base_path}. The updated test would not use string replacement here, but prepend the base path. When exporting the expectations, the basePath gets stripped from the generated url.

The tests still need work. Uploading the patch so far to test against DrupalCI.

Attempt at fixing sub directory.

Taking another pass at the tests. I took an initial stab at supplying our own entity type. The patch includes a new module with a basic entity type with bundles. I replaced nodes in the tests with the new entity type. We still have to think about Taxonomy Terms and Vocabularies. They probably can be removed, since i have added the new entity bundle definition to the api as well.

The updated patch includes updated expectation files as well. Lets see if it passes ci.

@tedbow, any thoughts on where to go from here with the tests?

Taking another stab at cleaning this up. I did some major refactoring between this and the previous patch, to account of the changes in drupal 8.4 to 8.5.

Here is a quick breakdown:

• Improved validation of path and host to ensure they match expected.
• Added verification of existence of swagger, schemes, info, and security sections.
• Verified that Paths have valid tags, schemes, and security methods.
• Reduced expectations files to eliminate all the above "dynamic" type data, and only track the more static path's and definitions.

The attached patch contains the refactored test logic, which hopefully will pass for both 8.4 and 8.5.

Actually running sudirectory tests locally now cause I suck.

For #2930810-11: Make tests more resilient to changes in Drupal core.1, we are creating our own field here, once on the custom entity type, once on taxomony_term. The comment doesn't seem relevant and should probably be changed to simply "Added test fields". Seems like the current comment is most likely the result of copy/paste.

For #11.2, I am not familiar with how `@dataProvider` works. I do think tracking the list of types and bundles here is a little clunky in terms of the test though. I wouldn't be against changing them to be sourced from elsewhere.

For #11.3, I think the requirement to address jsonapi differently then core rest stems from a larger problem which should be solved by providing those keys as part of the generator's jsonapi schema. Even if those options are not supported, we still should probably be providing the lack of data there in order to be consistent. In the mean time, I think that its safe to wrap these in the #api_module !== 'jsonapi' for now, with a plan to revise it later.

The changes in #14, #15, and #16 look good. Additionally added the suggestion for #11.3 and created issues for the @todos added in the current patch.

Previous patch passed tests. My interdiff caused it to fail.

talked with @tedbow, this is a good improvement to the module tests and functionality. Marking RTBC.