Support from Acquia helps fund testing for Drupal Acquia logo

Comments

quietone created an issue. See original summary.

quietone’s picture

Status: Active » Needs review
FileSize
17.34 KB

This is just part of the conversion. Still to do is d6/CommentTest, d6/CommentSourceWithHighWaterTest.php and d6/CommentTestBase.php which needs the high water set. How to do that?

chipway’s picture

Status: Needs review » Needs work

I would add here what alexpott suggested in order to help to review then commit it:
See #2807879-14: Convert Contact's Migrate source tests to new base class - let's use better array in/out keys to make it easier to understand what is going on.

quietone’s picture

Status: Needs work » Needs review
FileSize
17.9 KB
5.4 KB

Added better array keys as suggested in #3 and added some missing doc blocks.

quietone’s picture

Status: Needs review » Needs work

Back to needs work for :

Still to do is d6/CommentTest, d6/CommentSourceWithHighWaterTest.php and d6/CommentTestBase.php which needs the high water set. How to do that?

erozqba’s picture

Status: Needs work » Needs review
FileSize
25.54 KB

Adding d6/CommentTest, I don't know how to set the high water, so still to do d6/CommentSourceWithHighWaterTest.php, any ideas?

phenaproxima’s picture

Assigned: Unassigned » phenaproxima
phenaproxima’s picture

Status: Needs review » Needs work

I think that adding high-water support will involve changing the MigrateSourceTestBase class -- which is totally fine. :) I'd recommend adding a new parameter to testSource(), call it $high_water_mark or similar, and default it to NULL. Then, in that method, if $high_water_mark is set, set the value in the high-water storage service (which is referenced in SourcePluginBase::getHighWaterStorage(), I believe). This should all be done before iterating over the source to assert the results.

Something like this should do the trick, before the $expected_count is asserted:


    // If there is a high-water mark, set it in the high-water storage.
    if (isset($high_water_mark)) {
      $this->container
        ->get('keyvalue')
        ->get('migrate:high_water')
        ->set($this->migration->id(), $high_water_mark);
    }
quietone’s picture

Status: Needs work » Needs review
FileSize
28.94 KB
13.38 KB

This patch implements phenaproxima's suggestion in #8. Unfortunately CommentSourceWithHighWaterTest fails to pass. It fails in setting the high water value in the key value storage with 'Illegal offiset type' in this code.

  $this->container
        ->get('keyvalue')
        ->get('migrate:high_water')
        ->set($this->migration->id(), $high_water_mark);
    }

Some debugging shows that $this->migration->id() returns an object even though the setup does

 $this->migration->id()->willReturn(
      $this->randomMachineName(16)
    );

Status: Needs review » Needs work

The last submitted patch, 9: 2807875-9.patch, failed testing.

phenaproxima’s picture

Heh, I just realized my mistake -- try changing this:

$this->container
        ->get('keyvalue')
        ->get('migrate:high_water')
        ->set($this->migration->id(), $high_water_mark);

...to this:

$this->container
        ->get('keyvalue')
        ->get('migrate:high_water')
        ->set($this->migration->reveal()->id(), $high_water_mark);
quietone’s picture

Status: Needs work » Needs review
FileSize
29.39 KB
1.5 KB

Thx. I think I know enough now that I should have seen that. I'll blame it on a bit of jet lag...

So, here is the progress after adding the 'reveal'.

1) The count test fails. The expected count is 1 but the assertCount returns an actual count of 2, which is the size of the source. Based on the comment in #8 about adding the code before the $expected_count is asserted, I thought that the actual count would be 1.

2) Putting that aside for a while, I commented out that test so the next test would run. And again, 2 rows were being returned. Eventually found that the query condition to test the high water was not being added to the query in SqlBase. Conditions on the query are added only if $condition_added is true. So setting condition_added to true in the high water if block fixed that. But then, why do the existing high water tests pass? Because conditions_added is set to true when the map is joinable. And the map is joinable for those tests, but not CommentSourceWithHighWaterTest.

3) I also found it possible to get false positives from MigrateSourceTestBase. If the high_water is set but the value is not put in storage, null is used. That gives a condition of 'timestamp > null', which returns 0 rows. And even though the expected count is > 0, MigrateSourceTestBase doesn't check that it has, in fact, iterated over that many rows. Perhaps, a simple check on the would be sufficient, something like $this->assertSame($expected_count, $i);

Status: Needs review » Needs work

The last submitted patch, 12: 2807875-12.patch, failed testing.

quietone’s picture

Status: Needs work » Postponed

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

Status: Postponed » Needs review
FileSize
31.57 KB
12.55 KB

In the duplicate, #2848058: Update comment migration tests, CommentTestBase was removed and that needs to be done here. This just makes those changes and some coding standard fixes.

Might as well test it, so setting to NR.

Status: Needs review » Needs work

The last submitted patch, 16: 2807875-16.patch, failed testing.

quietone’s picture

Status: Needs work » Postponed

For the commiter: please credit Jo Fitzgerald. I've added work to this issue that he did in the duplicate #2848058: Update comment migration tests. Thx.

quietone’s picture

Although postponed this was on my mind. Posting this patch for later review.

jofitz’s picture

Thanks for the nod, @quietone.

mikeryan’s picture

Status: Postponed » Needs review

Highwater issue is in, unpostponing and testing latest patch.

Status: Needs review » Needs work

The last submitted patch, 19: 2807875-19.patch, failed testing.

jofitz’s picture

Status: Needs work » Needs review
FileSize
30.9 KB

Re-rolled.

Status: Needs review » Needs work

The last submitted patch, 23: 2807875-23.patch, failed testing.

jofitz’s picture

Status: Needs work » Needs review
FileSize
27.59 KB

Helps if I roll it against an up-to-date codebase...

Status: Needs review » Needs work

The last submitted patch, 25: 2807875-25.patch, failed testing.

quietone’s picture

Status: Needs work » Needs review
FileSize
27.81 KB

And put CommentTypeTest in the correct directory.

phenaproxima’s picture

+++ b/core/modules/migrate/tests/src/Kernel/MigrateSourceTestBase.php
@@ -158,8 +158,8 @@ public function testSource(array $source_data, array $expected_data, $expected_c
+    // countable and there is no high water mark.
+    if (is_numeric($expected_count) && !isset($high_water)) {
       $this->assertInstanceOf('\Iterator', $plugin);
       $this->assertSame($expected_count, iterator_count($plugin));
     }
@@ -185,6 +185,10 @@ public function testSource(array $source_data, array $expected_data, $expected_c

@@ -185,6 +185,10 @@ public function testSource(array $source_data, array $expected_data, $expected_c
         }
       }
     }
+
+    if (isset($high_water)) {
+      $this->assertSame($expected_count, $i);
+    }

I find this stuff a little strange. Firstly, won't the expected count be influenced by the high-water value? Why would we only want to assert the count if there's no high-water mark?

Second, I'm a bit uncomfortable asserting anything against $i -- it's just a counter value to loop through the expected data and probably shouldn't be treated as any kind of canonical value to assert against.

quietone’s picture

@phenaproxima, yes that is wrong and is now removed. Particularly now that highwater has been improved, #2824267: Highwater condition isn't added (on remote databases).

phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community

Everything else was great. Proceed!

alexpott’s picture

Version: 8.4.x-dev » 8.3.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed bdcd497 and pushed to 8.4.x. Thanks!

I think we should backport this to 8.3.x to keep everything the same in test land.

+++ /dev/null
@@ -1,80 +0,0 @@
-abstract class CommentTestBase extends MigrateSqlSourceTestCase {

Experimental - this is okay.

  • alexpott committed bdcd497 on 8.4.x
    Issue #2807875 by quietone, Jo Fitzgerald, erozqba, phenaproxima:...
alexpott’s picture

Status: Patch (to be ported) » Fixed

Committed 102cffe and pushed to 8.3.x. Thanks!

  • alexpott committed 102cffe on 8.3.x
    Issue #2807875 by quietone, Jo Fitzgerald, erozqba, phenaproxima:...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.