Implementing additional tests for ctools:

- New function ctools_reset_page_tokens() to reset the page token store (drupal_static()).

- Unit test coverage of the Object Cache plugins simple.inc and export_ui.inc, and the function ctools_cache_find_plugin, mostly checking the plugins are well-formed.

- Unit test coverage of the page_token functionality.
- ctools_set_page_token()
- ctools_set_variable_token()
- ctools_set_callback_token()
- ctools_page_token_processing()

Improving function documentation for:

- ctools_set_page_token()
- ctools_set_variable_token()
- ctools_set_callback_token()

CommentFileSizeAuthor
#64 2827100-64.patch884 bytesTR
#62 2827100-62-partial-revert.patch6.95 KBjoelpittet
#52 ctools-n2827100-52.interdiff.txt415 bytesDamienMcKenna
#52 ctools-n2827100-52.patch51.14 KBDamienMcKenna
#49 ctools-n2827100-49.patch51.08 KBDamienMcKenna
#49 ctools-n2827100-49.interdiff.txt614 bytesDamienMcKenna
#47 ctools-n2827100-46.patch53.55 KBrivimey
#47 interdiff-44-46.txt12.24 KBrivimey
#44 ctools-n2827100-44.interdiff.txt4.92 KBDamienMcKenna
#44 ctools-n2827100-44.patch42.69 KBDamienMcKenna
#40 ctools-n2827100-40.interdiff.txt1.88 KBDamienMcKenna
#40 ctools-n2827100-40.patch42.1 KBDamienMcKenna
#37 ctools-n2827100-37.interdiff.txt6.21 KBDamienMcKenna
#37 ctools-n2827100-37.patch41.42 KBDamienMcKenna
#35 ctools-n2827100-35.patch40.94 KBDamienMcKenna
#35 ctools-n2827100-35.interdiff.txt2.33 KBDamienMcKenna
#33 ctools-n2827100-33.interdiff.txt4.99 KBDamienMcKenna
#33 ctools-n2827100-33.patch40.89 KBDamienMcKenna
#30 ctools-n2827100-30.patch39.77 KBDamienMcKenna
#30 ctools-n2827100-30.interdiff.txt2.66 KBDamienMcKenna
#28 ctools-n2827100-28.patch39.39 KBDamienMcKenna
#24 ctools-n2827100-24.interdiff.txt966 bytesDamienMcKenna
#24 ctools-n2827100-24.patch38.29 KBDamienMcKenna
#22 ctools-n2827100-22.interdiff.txt3.4 KBDamienMcKenna
#22 ctools-n2827100-22.patch38.28 KBDamienMcKenna
#20 ctools-n2827100-20.interdiff.txt415 bytesDamienMcKenna
#20 ctools-n2827100-20.patch38.19 KBDamienMcKenna
#17 ctools-n2827100-17.patch38.19 KBDamienMcKenna
#17 ctools-n2827100-17.interdiff.txt28.4 KBDamienMcKenna
#15 ctools-n2827100-15.patch23.15 KBDamienMcKenna
#11 Screen Shot 2016-11-18 at 22.48.55.png184.2 KBrivimey
#10 page-css-uuid.csv_.txt7.15 KBrivimey
#10 base.csv_.txt6.47 KBrivimey
#7 improve_test_coverage-2827100-7.patch23.68 KBrivimey
#6 improve_test_coverage-2827100-6.patch20.37 KBrivimey
#4 improve_test_coverage-2827100-4.patch14.51 KBrivimey
#2 improve_test_coverage-2827100-2.patch12.39 KBrivimey
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rivimey created an issue. See original summary.

rivimey’s picture

Added new function ctools_reset_page_tokens() for use by test suite, and modify ctools_set_page_token() to use drupal_static to enable ctools_reset_page_tokens(). Otherwise tests accumulate changes rather than resetting them for each test.

rivimey’s picture

This patch includes changes to ctools.info that are included in 2808985 so it should be applied afterwards.

rivimey’s picture

FileSize
14.51 KB

Adding a token processing test in CtoolsPageTokens

rivimey’s picture

Issue summary: View changes
rivimey’s picture

Added new tests for callback tokens and clarify docs for that.

Improve tests for CSS handling, especially check the negative cases.

rivimey’s picture

FileSize
23.68 KB

Last one for now; test the uuid functionality.

rivimey’s picture

Some code-coverage info from the code_coverage module and xdebug. However, note there's a lot of code that is only ever parsed, so files with 100% coverage are likely to be more like 0% :(

For the record, current ctools 7.x-1.x has reported Total coverage of 75% (vs 77% from this patch)

DamienMcKenna’s picture

I question the test coverage data in comment #8, but I'm happy to see the test coverage itself improve. Great work, rivimey!

rivimey’s picture

FileSize
6.47 KB
7.15 KB

Uploaded base (ctools 7.x-1.x) and to most recent patch (as page-css-uuid) both as simple csv files. Easier to manipulate than table html!

rivimey’s picture

DamienMcKenna, the coverage data appears to only be present for functions that were called at least once. If a function was never, ever called, there is no data for it, and it 'disappears' from the coverage data as well, being considered no more than a comment.

An example: this is from the colorized output from the stylizer code. Some functions were called by php at some point, but there is no test for them, so the lines are red. The other functions appear in light grey, indicating not included (I think). A tested line would be black.

Many of the plugins are all grey and their coverage is indicated as 100% :(

rivimey’s picture

Status: Active » Needs review

Marking needs review: there is plenty more to do on this, but I think this patch is big enough!

rivimey’s picture

rivimey’s picture

Assigned: rivimey » Unassigned
DamienMcKenna’s picture

FileSize
23.15 KB

Rerolled.

Status: Needs review » Needs work

The last submitted patch, 15: ctools-n2827100-15.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
28.4 KB
38.19 KB

Some minor tidying up.

Status: Needs review » Needs work

The last submitted patch, 17: ctools-n2827100-17.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review

Darnit, the patch isn't applying anymore via testbot, even though it applies perfectly locally.

DamienMcKenna’s picture

FileSize
38.19 KB
415 bytes

Fixed a typo in math_expression.test.

Status: Needs review » Needs work

The last submitted patch, 20: ctools-n2827100-20.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
38.28 KB
3.4 KB

Minor fixes to object_cache.test, object_cache_unit.test and uuid.test.

Status: Needs review » Needs work

The last submitted patch, 22: ctools-n2827100-22.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
38.29 KB
966 bytes

Two PHP 5.3 -compatibility fixes.

DamienMcKenna’s picture

The following error is being logged by the testbot:

PHP Fatal error: Call to undefined function ctools_include() in /var/www/html/sites/all/modules/ctools/tests/object_cache_unit.test on line 97

Why would it fail to enable the CTools module, which is where that function exists?

Status: Needs review » Needs work

The last submitted patch, 24: ctools-n2827100-24.patch, failed testing.

DamienMcKenna’s picture

The exception thrown by uuid.test is:

Constant UUID_PATTERN already defined

Given it's wrapped around an if(!defined()) I'm not sure how this is possible. Argh.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
39.39 KB

The UUID handling had to be split up into two classes, otherwise it was failing when the UUID module was loaded.

Status: Needs review » Needs work

The last submitted patch, 28: ctools-n2827100-28.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
2.66 KB
39.77 KB

Not sure what's going on, the tests run correctly locally.

Status: Needs review » Needs work

The last submitted patch, 30: ctools-n2827100-30.patch, failed testing.

DamienMcKenna’s picture

Darnitall :-\ Not sure what's going on :(

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
40.89 KB
4.99 KB

Some minor tweaks, shouldn't make any difference.

Status: Needs review » Needs work

The last submitted patch, 33: ctools-n2827100-33.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
2.33 KB
40.94 KB

Trying to disable the UUID tests for later.

Status: Needs review » Needs work

The last submitted patch, 35: ctools-n2827100-35.patch, failed testing.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
41.42 KB
6.21 KB

Additional minor tweaks.

Status: Needs review » Needs work

The last submitted patch, 37: ctools-n2827100-37.patch, failed testing.

DamienMcKenna’s picture

That "Call to undefined function ctools_include() in object_cache_unit.test on line 97" error is continuing to cause it to fail, not sure what is causing that.

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
42.1 KB
1.88 KB

Running the tests locally, here's where it's failing:

Fail      Other      object_cache_unit  118 CtoolsUnitObjectCachePlugins->testF
    The Export Cache plugin: has a clear function
Exception Notice     object_cache_unit   52 CtoolsUnitObjectCachePlugins->asser
    Undefined index: cache clear
Fail      Other      object_cache_unit  118 CtoolsUnitObjectCachePlugins->testF
    The Export Cache plugin: clear is executable

This is because Export UI doesn't have a 'cache clear" method.

This patch updates export_ui.inc with a @todo comment, and disables the assertions that were failing but clarifies the @todo to say that the API problem should be fixed.

Status: Needs review » Needs work

The last submitted patch, 40: ctools-n2827100-40.patch, failed testing.

DamienMcKenna’s picture

... what?

00:00:47.798 Test summary
00:00:47.798 ------------
00:00:47.798
00:00:48.013 PHP Fatal error: Call to undefined function ctools_include() in /var/www/html/sites/all/modules/ctools/tests/object_cache_unit.test on line 99
00:00:48.015
00:00:48.015 Fatal error: Call to undefined function ctools_include() in /var/www/html/sites/all/modules/ctools/tests/object_cache_unit.test on line 99
00:00:48.017 UUID handing, without UUID module 0 passes, 0 fails, and 0 exceptions
00:00:48.018 UUID handing, with UUID module 0 passes, 0 fails, and 0 exceptions
00:00:48.203 FATAL CtoolsUUIDWithUUID: test runner returned a non-zero error code (255).
00:00:52.664 Keywords substitution 14 passes, 0 fails, and 0 exceptions
00:00:52.842 Get plugin info 15 passes, 0 fails, and 0 exceptions
00:00:52.873 Math expressions stack 17 passes, 0 fails, and 0 exceptions
00:00:53.176 Object cache storage (UI tests) 9 passes, 0 fails, and 0 exceptions
00:00:53.280 CSS cache 4 passes, 0 fails, and 0 exceptions
00:01:04.121 Page token processing 16 passes, 0 fails, and 0 exceptions
00:01:04.617 CSS Tools tests 16 passes, 0 fails, and 0 exceptions
00:01:04.768 Math expressions 36 passes, 0 fails, and 0 exceptions
00:01:19.522 Export CRUD 27 passes, 0 fails, and 0 exceptions

So it's still failing on "Call to undefined function ctools_include()" for the first line of testFindSimpleCachePlugin(), which makes no sense as ctools.module should be loaded by that point.

As for "FATAL CtoolsUUIDWithUUID: test runner returned a non-zero error code (255)." I have no idea.

rivimey’s picture

I included the tests of export_ui and decided (based on the api comments by merlinofchaos) that the lack of 'clear' method was a code failure, not a test failure, so I left it in place.

It is likely this scenario (i.e. test finds a bug) will happen again: did I do the wrong thing?

DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
42.69 KB
4.92 KB

Adding ctools as a dependency on each getInfo() definition.

DamienMcKenna’s picture

Status: Needs review » Needs work

Ok, talking in IRC the problem with object_cache_unit.test is that it uses DrupalUnitTestCase(), which can't install modules thus ctools_include() is never made available.

The last submitted patch, 44: ctools-n2827100-44.patch, failed testing.

rivimey’s picture

FileSize
12.24 KB
53.55 KB

Main points:
- in object_cache_unit.test: now extends DrupalWebTestCase
- in uuid_with_uuid.test changed CtoolsUUIDWithUUID: now defines uuid as a dependency.

Also got carried away a bit; did some more docblock changes too.
Note that in the docblocks I decided not to add 'array' hints to the functions, in case that causes problems with actual code. We can leave it for another time :-)

rivimey’s picture

Status: Needs work » Needs review
DamienMcKenna’s picture

FileSize
614 bytes
51.08 KB

Great improvements!

We're not ready to add any dependencies on 'uuid' yet, so lets leave that for later.

DamienMcKenna’s picture

rivimey’s picture

@DamienMcKenna, The patch in #49 has additional space at the end of one of the lines in math_expression.test causing git apply to complain, which got me looking at that file, and the result is I've updated the file as a whole. Now thinking that work would be better off as a new Issue rather than tacked onto this one -- do you agree?

The extra space is on line 922 **of the patch file** for function testBuildInFunctions(), approx line 67 in the php.

Edit: Created as #2830393: Improve test coverage and function docs: math_expression

DamienMcKenna’s picture

FileSize
51.14 KB
415 bytes

Fixed the comment on the testBuildInFunctions() method, which is where the whitespace error came from.

DamienMcKenna’s picture

RTBC in my book ;)

rivimey’s picture

Status: Needs review » Reviewed & tested by the community

  • japerry committed 7be2c12 on 7.x-1.x authored by DamienMcKenna
    Issue #2827100 by DamienMcKenna, rivimey: Improve test coverage and...
japerry’s picture

Status: Reviewed & tested by the community » Fixed

Looks good to me. Committed.

DamienMcKenna’s picture

Thanks for finally committing this, japerry. BTW the commit authorship should have gone to rivimey - would you mind reverting & recommitting it with that corrected?

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

joelpittet’s picture

Considering the disabled tests and the dependency on uuid which causes a chain in breaking 5.3 compatibility. I'm considering reverting this patch in parts (mostly the @todo parts).

#2942581: ctools.test breaks Feeds tests on PHP 5.3 because of short array syntax
#2783113: Using trait without PHP 5.4 requirement breaks Drupal CI of all modules that require UUID

+++ b/tests/css.test
@@ -1,65 +1,103 @@
+   * @todo Test  db-row-exist but file-not-exist case in ctools_css_retrieve().

+++ b/tests/object_cache_unit.test
@@ -0,0 +1,141 @@
+    // @todo Clear is required by the spec (cache.inc:40..48): but export_ui
...
+    // @todo Break is optional acc'd to spec but does anything implement it?
...
+    // @todo Finalize is optional so don't fail if not there??

+++ b/tests/uuid_with_uuid.test
@@ -0,0 +1,55 @@
+    // @todo Add UUID as a test dependency and then add comment out this line.
+    // $modules[] = 'uuid';
...
+  public function DISABLED_testMakeNewUUID() {
+    // drupal_get_filename will report even if the module is disabled.

+++ b/tests/uuid_without_uuid.test
@@ -0,0 +1,101 @@
+   * @todo Fix this so testbot doesn't fail.
...
+   * @todo Fix this so testbot doesn't fail.
...
+  public function DISABLED_testMakeNewUUIDRaw() {
...
+   * @todo Fix this so testbot doesn't fail.
...
+  public function DISABLED_testMakeNewUUIDWrapper() {
...
+   * @todo Fix this so testbot doesn't fail.

Snuck in @todos

joelpittet’s picture

Status: Closed (fixed) » Needs review
Issue tags: -Needs tests
Parent issue: #2828925: Plan for CTools 7.x-1.13 release » #2941916: Plan for CTools 7.x-1.14 release
joelpittet’s picture

joelpittet’s picture

Here's what I propose we revert.

Status: Needs review » Needs work

The last submitted patch, 62: 2827100-62-partial-revert.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

TR’s picture

Status: Needs work » Needs review
FileSize
884 bytes

I think it should be sufficient to just comment out the test cases and the test_dependency in ctools.info - locally at least run-tests.sh won't discover the tests if this is done. I'm not sure exactly if that's also true with the testbot.

One way to find out would be to commit this attached patch, THEN try to test CTools against PHP 5.3 and see if it works. If not, revert the patch.

(I'm testing against 5.4 here because patches are applied AFTER the dependencies are calculated, so PHP 5.3 tests will fail even if the patch is correct - that's what happened in #62. The 5.4 test will simply ensure the patch applies, nothing else.)

joelpittet’s picture

The tests aren't finished or working, I'd rather remove them. I don't want to leave @todo in production code, that can stay in the issue queue.

The last submitted patch, 62: 2827100-62-partial-revert.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

  • joelpittet committed dd29dee on 7.x-1.x
    Issue #2827100 by DamienMcKenna, rivimey, joelpittet, TR: Partial Revert...
joelpittet’s picture

Status: Needs review » Fixed

#62 was committed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.