I successfully migrated my site from D7 to D8, but all my forum containers was imported as forums.

Solution:
The patch implements the solution suggest in #19

CommentFileSizeAuthor
#72 interdiff-2903007-51-72.txt3.83 KBmaxocub
#72 2903007-72.patch7.48 KBmaxocub
#61 d8-new-container-created.png156.75 KBdipakmdhrm
#61 d8-create-new-container.png145.65 KBdipakmdhrm
#61 d8-migrated-forums.png156.37 KBdipakmdhrm
#61 d7-forums.png138.5 KBdipakmdhrm
#51 interdiff-2903007-48-51.txt801 bytesrakesh.gectcr
#51 2903007-51.patch5.94 KBrakesh.gectcr
#48 interdiff-2903007-45-48.txt11.11 KBrakesh.gectcr
#48 2903007-48.patch5.94 KBrakesh.gectcr
#45 interdiff-2903007-42-45.txt2.4 KBdipakmdhrm
#45 2903007-45.patch7.43 KBdipakmdhrm
#42 interdiff-2903007-41-42.txt7.4 KBdipakmdhrm
#42 2903007-42.patch7.42 KBdipakmdhrm
#41 interdiff-2903007-37-41.txt3.32 KBrakesh.gectcr
#41 2903007-37.patch3.51 KBrakesh.gectcr
#37 2903007-37.patch3.51 KBrakesh.gectcr
#37 interdiff-2903007-35-37.txt2.76 KBrakesh.gectcr
#35 2903007-35.patch1.27 KBrakesh.gectcr
#28 2903007-28.patch1.26 KBrakesh.gectcr
#22 2903007-22.patch1.35 KBdipakmdhrm
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dillix created an issue. See original summary.

cilefen’s picture

cilefen CreditAttribution: cilefen commented
Component: forum.module » migration system

Thank you for reporting an issue. Let's move this to the migration system first so as to get attention from the migrate maintainers.

dillix’s picture

dillix CreditAttribution: dillix commented
Issue tags: +migrate-d7-d8, +Migrate critical
quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented

I'm pretty sure this was fixed in, #2669030: D6 Forum vocabulary is migrated to the wrong D8 field name, which was only committed yesterday to 8.5.x. Can anyone confirm that? Especially, maxocub, who wrote the patch.

dillix’s picture

dillix CreditAttribution: dillix commented

@quietone, I will check tomorrow if patch from #2669030 fixes this issue.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Active » Postponed (maintainer needs more info)

Postponing while we wait.

dillix’s picture

dillix CreditAttribution: dillix commented
Status: Postponed (maintainer needs more info) » Active

I tried to migrate in fresh 8.4-dev (catch commited patch also for 8.4), but instead containers i've got forums. Also I can't manually create new containers, after migration drupal create forums instead containers.

See screenshot: http://joxi.ru/LmG8Ba9seG4Ll2

dillix’s picture

dillix CreditAttribution: dillix commented

@heddn how can I help to fix this issue?

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented

Maybe I don't understand the wording used here. When I hear the word containers, I think of something besides Drupal. What are containers in the context of comments and forums?

dillix’s picture

dillix CreditAttribution: dillix commented

@heddn When I install new site from scratch on D8 I can create containers and move forums to it. Here is screenshot from admin: http://joxi.ru/xAeJKLgFpKWEEr
And this one from frontend: http://joxi.ru/DrlNXvghvDMg72

But when I migrate from D7 to D8 my containers migrated as forums with child subforums. And when I try to create new container I also get forum instead.

dillix’s picture

dillix CreditAttribution: dillix commented

I investigated forum & taxonomy migrations and there isn't containers support.

xjm’s picture

xjm
she/her
Primary language English
CreditAttribution: xjm at Acquia commented
Title: Can't create container after migration from D7 » Can't create forum container after migration from D7
xjm’s picture

xjm
she/her
Primary language English
CreditAttribution: xjm at Acquia commented
Priority: Major » Critical

Per discussion with @catch, we're promoting the "Migrate critical" issues to critical given the importance of a stable migration path at this point in the release cycle. (The framework and release managers will confirm this in a later triage for individual issues.) Assuming the feature of forum containers still exists in D8 (edit: which is confirmed in #10), this is actually a data integrity issue.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented

@dillix was container support available before the migration? Meaning, is container support available in D8, and there a migrate issue or is the feature not available? I'm not very familiar with forum, so you are the expert here...

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

I'll have some time on Friday this week to troubleshoot this further unless others can get it solved already earlier.

Cheers,
Markus

dillix’s picture

dillix CreditAttribution: dillix commented

@heddn, yes support is available on fresh D8 site before migration (both in templates & admin), I attached links with screenshots in #10, but after migration it breaks. This bug is critical for our site and we can't migrate to D8 now.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented
Assigned: Unassigned » masipila

I investigated this a bit in the bus on my way to work.

Drupal 7
https://api.drupal.org/api/drupal/modules%21forum%21forum.admin.inc/7.x

It seems to me that both forums and forum containers are taxonomy terms. The difference is that we have a persistent variable forum_containers which seems to be an array ot term ids that should be treated as containers instead of forums.

Assigning to myself, I'll continue with this during this week.

Markus

Edit: typos

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

I had a bit of time yesterday evening on this. I need a bit of guidance on the approach.

1. Digging the source is-this-forum-container setting
We can easily do the determination in thde Term source plugin

If the taxonomy term is forum relevant, we can add a source property forum_container with the correct value.

2. What is the most elegant way to map this for forum relevant terms only?
This the part where I could use some guidance. I don't think we want to add the "forum_container is false" value to all non-forum related terms. Or is this safe to do in the sence that this mapping sould be automatically ignored for the non-forum related terms?

Cheers,
Markus

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

Based on my own cursory research, here's what I'm seeing.

In D8, forum_container is a configurable boolean field attached to taxonomy terms from the 'forums' vocabulary. It should therefore be completely harmless for us to do something like this: the source plugin could set an is_container property which indicates if the term being migrated is a container (which, as you know, can be found out by looking at the value of a particular D7 variable). Then we can map that directly to the forum_container destination property, which (I think) will simply be discarded for terms which don't have that field.

Should be pretty straightforward.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

Then we can map that directly to the forum_container destination property, which (I think) will simply be discarded for terms which don't have that field.

Thanks, this answers the only thing I was wondering i.e. does it harm if we map this to all terms. All clear now.

Markus

maxocub’s picture

maxocub CreditAttribution: maxocub for Acquia commented
Issue tags: +Vienna2017

Tagging.

dipakmdhrm’s picture

dipakmdhrm CreditAttribution: dipakmdhrm as a volunteer commented
Status: Active » Needs review
Issue tags: +Needs tests
FileSize
2903007-22.patch1.35 KB

Status: Needs review » Needs work

The last submitted patch, 22: 2903007-22.patch, failed testing. View results

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented

Working on the tests

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented
Assigned: masipila » Unassigned

Unassigning myself since others had time to write the patch already before me. The solution is exactly how I was about to do this (not surprise since this is straight forward).

maxocub’s picture

maxocub CreditAttribution: maxocub for Acquia commented

Beside the failing lint to be fixed, here's my review:

  1. +++ b/core/modules/taxonomy/src/Plugin/migrate/source/Term.php
@@ -92,6 +92,16 @@ public function prepareRow(Row $row) {
+    // Set 'is_container' as a property.
+    // This is will be TRUE for terms which are forum containers, but
+    // will be FAlSE for all the rest.

    What about something simpler: "Set the 'is_container' to TRUE if the tid is in the 'forum_container' variable." 

  2. +++ b/core/modules/taxonomy/src/Plugin/migrate/source/Term.php
@@ -92,6 +92,16 @@ public function prepareRow(Row $row) {
+      in_array($current_tid, $forum_container_tids) ? TRUE : FALSE;

    in_array already return TRUE or FALSE, so the " ? TRUE : FASLE" part is not needed. 

  3. +++ b/core/modules/taxonomy/src/Plugin/migrate/source/Term.php
@@ -92,6 +92,16 @@ public function prepareRow(Row $row) {
+    $row->setSourceProperty(
+      'is_container',
+      in_array($current_tid, $forum_container_tids) ? TRUE : FALSE;
+    );

    In retrospect, this could look cleaner with a one-liner.

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Assigned: Unassigned » rakesh.gectcr
rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Status: Needs work » Needs review
FileSize
2903007-28.patch1.26 KB

Done all the comment @maxocub has addressed.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

Once the automatic tests pass, I can do a manual test still this evening and RTBC this after that.

Markus

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented

Talked to rakesh.gectr about minor nits in a comment, the variable name didn't match what was in the code. He has made changes that will be uploaded in next patch.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

Just realized that this can't be RTBCd yet. We're still missing automated test coverage for this...

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Status: Needs review » Needs work
quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented

Yes, forget to mention that rakesh.gectr was working on the test.

maxocub’s picture

maxocub CreditAttribution: maxocub for Acquia commented 
+++ b/core/modules/taxonomy/src/Plugin/migrate/source/Term.php

This change is in the wrong file.
Drupal\taxonomy\Plugin\migrate\source\Term is deprecated, we need to move that to Drupal\taxonomy\Plugin\migrate\source\d7\Term.

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Status: Needs work » Needs review
FileSize
2903007-35.patch1.27 KB

@maxcoub Thank God and Maxi, We did find out that .... :) Here i am attaching that patch now, Still I am working on test. :)

Status: Needs review » Needs work

The last submitted patch, 35: 2903007-35.patch, failed testing. View results

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
FileSize
interdiff-2903007-35-37.txt2.76 KB
2903007-37.patch3.51 KB

Lets hit again.

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 37: 2903007-37.patch, failed testing. View results

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented 
  1. +++ b/core/modules/taxonomy/src/Plugin/migrate/source/d7/Term.php
@@ -70,6 +70,11 @@ public function prepareRow(Row $row) {
+    // Set the 'is_container' to TRUE if the tid is in the 'forum_container' variable.

    I prefer a comment that says what are trying to achieve. And the code here is simple enough. So, Something like
    'Determine if this is a forum container.' 

  2. +++ b/core/modules/taxonomy/tests/src/Kernel/Plugin/migrate/source/d7/TermTest.php
@@ -159,6 +165,7 @@ public function providerSource() {
+        'is_container' => FALSE,

    is_container is defined only for the expected results of tests[0]. Are the other tests returning the expected results for is_container?

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Status: Needs work » Needs review
FileSize
2903007-37.patch3.51 KB
interdiff-2903007-37-41.txt3.32 KB

Added the tests,

According to quietone's comment,

1) Done

2) Honestly, I am not having convincible answer now. :(

dipakmdhrm’s picture

dipakmdhrm CreditAttribution: dipakmdhrm as a volunteer commented
Issue tags: -Needs tests
FileSize
2903007-42.patch7.42 KB
interdiff-2903007-41-42.txt7.4 KB

This patch contains all the needed tests (migration and source plugin).
This also address @quiteone's point #2 in comment #40 i.e. this patch now has expected results for is_container for all tests in TermTest.php

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented

Was trying to figure out, 

-  }
 
-  /**
-   * Tests the Drupal 7 taxonomy term to Drupal 8 migration.
-   */
-  public function testTaxonomyTerms() {
-    $this->assertEntity(1, 'General discussion', 'forums', '', NULL, 2);
-    $this->assertEntity(2, 'Term1', 'test_vocabulary', 'The first term.', 'filtered_html', 0, [], NULL, 3);
-    $this->assertEntity(3, 'Term2', 'test_vocabulary', 'The second term.', 'filtered_html');
-    $this->assertEntity(4, 'Term3', 'test_vocabulary', 'The third term.', 'full_html', 0, [3], 6);
-    $this->assertEntity(5, 'Custom Forum', 'forums', 'Where the cool kids are.', NULL, 3);
-    $this->assertEntity(6, 'Games', 'forums', '', NULL, 4);
-    $this->assertEntity(7, 'Minecraft', 'forums', '', NULL, 1, [6]);
-    $this->assertEntity(8, 'Half Life 3', 'forums', '', NULL, 0, [6]);
+    if (isset($entity->forum_container)) {
+      $this->assertEqual($expected_container_flag, $entity->forum_container->value);
+    }
   }

Why the lines are getting removed and trying to apply the patch got the following result.

$ git apply -v 2903007-42.patch
2903007-42.patch:9: trailing whitespace.
  forum_container: is_container 
Checking patch core/modules/taxonomy/migration_templates/d7_taxonomy_term.yml...
Checking patch core/modules/taxonomy/src/Plugin/migrate/source/d7/Term.php...
Checking patch core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php...
Checking patch core/modules/taxonomy/tests/src/Kernel/Plugin/migrate/source/d7/TermTest.php...
Applied patch core/modules/taxonomy/migration_templates/d7_taxonomy_term.yml cleanly.
Applied patch core/modules/taxonomy/src/Plugin/migrate/source/d7/Term.php cleanly.
Applied patch core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php cleanly.
Applied patch core/modules/taxonomy/tests/src/Kernel/Plugin/migrate/source/d7/TermTest.php cleanly.
warning: 1 line applied after fixing whitespace errors

Patch is good.

A small nit pick
+ forum_container: is_container

trailing whitespace.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs review » Needs work 
  1. +++ b/core/modules/taxonomy/migration_templates/d7_taxonomy_term.yml
@@ -17,6 +17,7 @@ process:
diff --git a/core/modules/taxonomy/src/Plugin/migrate/source/d7/Term.php b/core/modules/taxonomy/src/Plugin/migrate/source/d7/Term.php

+++ b/core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php
@@ -121,6 +111,20 @@ protected function getParentIDs($tid) {
+    $this->assertEntity(1, 'General discussion', 'forums', '', NULL, 2);

    There are 7 parameters for this method and 6 are being passed. Therefor the test isn't doing what you expect.

  2. +++ b/core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php
@@ -72,7 +72,7 @@ protected function setUp() {
+  protected function assertEntity($id, $expected_label, $expected_vid, $expected_description = '', $expected_format = NULL, $expected_weight = 0, $expected_parents = [], $expected_field_integer_value = NULL, $expected_term_reference_tid = NULL, $expected_container_flag = 0) {

    $expected_container_flag defaults to '0' but then checked with isset, but it will always be set if this is done. The source plugin sets it to TRUE or FALSE, so it will always have a value.

dipakmdhrm’s picture

dipakmdhrm CreditAttribution: dipakmdhrm as a volunteer commented
Status: Needs work » Needs review
FileSize
2903007-45.patch7.43 KB
interdiff-2903007-42-45.txt2.4 KB

Fixed the whitespace.

Also discussed the observations in #44 with @quiteone.

We decided that both observation were not valid for this patch. However, there indeed was a patch needed for fixing the tests where we were using integer values 0 & 1 instead of boolean TRUE & FALSE.

dillix’s picture

dillix CreditAttribution: dillix commented

After migration to D8 I can't create containers anymore. When I try to create one it saves as forum. On a fresh install I can create containers.

maxocub’s picture

maxocub CreditAttribution: maxocub for Acquia commented

@dillix: Do you mean you tried the latest patch from #45 and it still didn't worked?

I did manually tested the latest patch on a fresh D8 install and the containers were migrated and I was able to create new containers after the migration.

If you did something different, please share the steps to reproduce.

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
FileSize
2903007-48.patch5.94 KB
interdiff-2903007-45-48.txt11.11 KB

A small nit pick need to update the comment doc block for the function

protected function assertEntity

Noticed that we are removed the is_container flag from 

// The expected results.
    $tests[0]['expected_data'] = [

And still i am trying to figure out [#43] y the first part is happening / y git is acting wierd.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs review » Needs work

Yes, dipakmdhrm is correct. My comments in #44 aren't very useful. Hope about I blame that as a side effect of breathing all this northern hemisphere air?

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

A couple of nitpicks:

  1. +++ b/core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php
@@ -91,6 +93,9 @@ protected function assertEntity($id, $expected_label, $expected_vid, $expected_d
+    if (isset($entity->forum_container)) {

    Let's use $entity->hasField('forum_container') instead. 

  2. +++ b/core/modules/taxonomy/tests/src/Kernel/Migrate/d7/MigrateTaxonomyTermTest.php
@@ -91,6 +93,9 @@ protected function assertEntity($id, $expected_label, $expected_vid, $expected_d
+      $this->assertEqual($expected_container_flag, $entity->forum_container->value);

    This should be assertSame().

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and at Valuebound for Acro Commerce commented
Status: Needs work » Needs review
FileSize
2903007-51.patch5.94 KB
interdiff-2903007-48-51.txt801 bytes

:) Here we go!

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Issue summary: View changes
Status: Needs review » Reviewed & tested by the community

The solution suggested by phenaproxima in #19 has been implemented. The patch has been tested and it does fix the problem. Updated the IS with a link to the solution that was implementd.

So off we go to RTBC land!

Thanks rakesh.gectcr and dipakmdhrm for getting this to RTBC at Vienna,

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented

Looks totally great. +1 RTBC.

dillix’s picture

dillix CreditAttribution: dillix commented

I will try latest patch soon and write here about results.

rakesh.gectcr’s picture

rakesh.gectcr
he/him
Primary language Malayalam
Location Manchester
CreditAttribution: rakesh.gectcr as a volunteer and commented
webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
CreditAttribution: webchick at Acquia commented
Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

The OP talks about a bug that after migrating, they can no longer create forum containers. There is test coverage for the migration itself, but not for this bug. Marking back to needs work for tests, per @phenaproxima.

Separately, it feels super grody to the max and decidedly un-tubular and non-radical to shove a bunch forum logic in the taxonomy term migrations. Maybe we could also get a non-critical follow-up to make Forum a sub-class of Term so we can better encapsulate the one-off logic here.

dillix’s picture

dillix CreditAttribution: dillix commented

I tried #51 on 8.4 latest dev and nothing changes. My containers migrated as forums and I can't create new containers. When I try to create containers before upgrade it's created as containers.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

Hi dillix,

Could you please clarify what exactly do you mean that you are not able to create new containers after you have executed migrations?

Do you mean that when you try to create new containers manually after the migrations have been executed, they are created as forums?

Or something else?

Cheers,
Markus

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

Agreed with #56.

I'm wondering if the forum migration could be a secondary migration (i.e. allow taxonomy to migrate the terms, then have forum get the forum/container metadata from the source site, load the terms already migrated and save the field values etc.).

Having said that, since the term migration already does some forum logic, that might be OK as a non-critical follow-up once we've fixed the data-integrity issue here.

dillix’s picture

dillix CreditAttribution: dillix commented

@masipila, yes I have this problem:
> Do you mean that when you try to create new containers manually after the migrations have been executed, they are created as forums?

And also containers created in D7 after migration converted to forums.

@catch I think that taxonomy migration breaks creation of new containers after migration.

dipakmdhrm’s picture

dipakmdhrm CreditAttribution: dipakmdhrm as a volunteer commented
Assigned: rakesh.gectcr » dipakmdhrm
FileSize
d7-forums.png138.5 KB
d8-migrated-forums.png156.37 KB
d8-create-new-container.png145.65 KB
d8-new-container-created.png156.75 KB

@dillix:
I did some manual testing with patch from #51. And everything seems to be working well.

1. I created some forums and containers in D7:
d7-forums

2. I was able to import them as respective forums and containers in D8 (by using Migrate Drupal UI i.e. /upgrade path):
d8-migrated-forums

3. I was able to create a new forum container after migration as well:
d8-create-new-container.png


d8-new-container-created.png

Is there something wrong with what I am trying? Did you try something else?

dillix’s picture

dillix CreditAttribution: dillix commented

@dipakmdhrm on D7 site I also have advanced_forum module, but as I know it doesn't change containers functionality.

maxocub’s picture

maxocub CreditAttribution: maxocub as a volunteer commented

Thanks @dipakmdhrm for your manual testing and for confirming my results.

@dillix: Could you give us a list of all your enabled contrib modules? I just did manual testing with advanced_forum enabled and I wasn't able to reproduce your results, all containers were migrated as containers and I was able to create new containers after the migration.

I will try to add a test for the creation of forum containers after a migration.

dillix’s picture

dillix CreditAttribution: dillix commented

@maxocub modules on D7 or D8 site?

maxocub’s picture

maxocub CreditAttribution: maxocub as a volunteer commented

@dillix: Both I guess, the more info we have the easier it is to understand what's going on.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Assigned: dipakmdhrm » heddn

Assigning to myself to review this week.

dillix’s picture

dillix CreditAttribution: dillix commented

D7 modules:

 Package             Name                                                                   Version               
 Access control      ACL (acl)                                                              7.x-1.1               
 Access control      Forum Access (forum_access)                                            7.x-1.3               
 Administration      Administration menu (admin_menu)                                       7.x-3.0-rc5           
 Administration      Administration menu Toolbar style (admin_menu_toolbar)                 7.x-3.0-rc5           
 Administration      Module filter (module_filter)                                          7.x-2.1               
 Advanced Forum      Advanced Forum (advanced_forum)                                        7.x-2.6               
 Chaos tool suite    Chaos tools (ctools)                                                   7.x-1.12              
 Chaos tool suite    Page manager (page_manager)                                            7.x-1.12              
 Chaos tool suite    Views content panes (views_content)                                    7.x-1.12              
 Commerce            Cart (commerce_cart)                                                   7.x-1.13+9-dev        
 Commerce            Checkout (commerce_checkout)                                           7.x-1.13+9-dev        
 Commerce            Commerce (commerce)                                                    7.x-1.13+9-dev        
 Commerce            Commerce UI (commerce_ui)                                              7.x-1.13+9-dev        
 Commerce            Customer (commerce_customer)                                           7.x-1.13+9-dev        
 Commerce            Customer UI (commerce_customer_ui)                                     7.x-1.13+9-dev        
 Commerce            Line Item (commerce_line_item)                                         7.x-1.13+9-dev        
 Commerce            Line Item UI (commerce_line_item_ui)                                   7.x-1.13+9-dev        
 Commerce            Order (commerce_order)                                                 7.x-1.13+9-dev        
 Commerce            Order UI (commerce_order_ui)                                           7.x-1.13+9-dev        
 Commerce            Price (commerce_price)                                                 7.x-1.13+9-dev        
 Commerce            Product (commerce_product)                                             7.x-1.13+9-dev        
 Commerce            Product Pricing (commerce_product_pricing)                             7.x-1.13+9-dev        
 Commerce            Product Pricing UI (commerce_product_pricing_ui)                       7.x-1.13+9-dev        
 Commerce            Product Reference (commerce_product_reference)                         7.x-1.13+9-dev        
 Commerce            Product UI (commerce_product_ui)                                       7.x-1.13+9-dev        
 Commerce (contrib)  Commerce add to cart confirmation (commerce_add_to_cart_confirmation)  7.x-1.0-rc2+3-dev     
 Commerce (contrib)  Commerce Checkout Redirect (commerce_checkout_redirect)                7.x-2.0               
 Commerce (contrib)  Commerce Discount (commerce_discount)                                  7.x-1.0-alpha8+13-dev 
 Commerce (contrib)  Commerce Price Savings Formatter (commerce_price_savings_formatter)    7.x-1.4               
 Commerce (contrib)  Commerce VBO Views (commerce_vbo_views)                                7.x-1.2               
 Core                Block (block)                                                          7.56                  
 Core                Color (color)                                                          7.56                  
 Core                Comment (comment)                                                      7.56                  
 Core                Contextual links (contextual)                                          7.56                  
 Core                Dashboard (dashboard)                                                  7.56                  
 Core                Database logging (dblog)                                               7.56                  
 Core                Field (field)                                                          7.56                  
 Core                Field SQL storage (field_sql_storage)                                  7.56                  
 Core                Field UI (field_ui)                                                    7.56                  
 Core                File (file)                                                            7.56                  
 Core                Filter (filter)                                                        7.56                  
 Core                Forum (forum)                                                          7.56                  
 Core                Help (help)                                                            7.56                  
 Core                Image (image)                                                          7.56                  
 Core                List (list)                                                            7.56                  
 Core                Locale (locale)                                                        7.56                  
 Core                Menu (menu)                                                            7.56                  
 Core                Node (node)                                                            7.56                  
 Core                Number (number)                                                        7.56                  
 Core                Options (options)                                                      7.56                  
 Core                Overlay (overlay)                                                      7.56                  
 Core                Path (path)                                                            7.56                  
 Core                Statistics (statistics)                                                7.56                  
 Core                System (system)                                                        7.56                  
 Core                Taxonomy (taxonomy)                                                    7.56                  
 Core                Text (text)                                                            7.56                  
 Core                Update manager (update)                                                7.56                  
 Core                User (user)                                                            7.56                  
 Custom Modules      City select filter (city_select_filter)                                                      
 Custom Modules      Documents list (document)                                              7.x-1.x-dev           
 Custom Modules      Law articles (law)                                                     7.x-1.x-dev           
 Custom Modules      Multi calendar (multi_calendar)                                        7.x-1.x-dev           
 Date/Time           Date (date)                                                            7.x-2.10              
 Date/Time           Date API (date_api)                                                    7.x-2.10              
 Date/Time           Date Popup (date_popup)                                                7.x-2.10              
 Date/Time           Date Views (date_views)                                                7.x-2.10              
 Development         Devel (devel)                                                          7.x-1.5               
 Extensible BBCode   Basic Tags (xbbcode_basic)                                             7.x-1.5+0-dev         
 Extensible BBCode   Extensible BBCode (xbbcode)                                            7.x-1.5+0-dev         
 Extensible BBCode   Lists (xbbcode_list)                                                   7.x-1.5+0-dev         
 Fields              Address Field (addressfield)                                           7.x-1.0-beta5         
 Fields              Conditional Fields (conditional_fields)                                7.x-3.0-alpha2        
 Fields              Email (email)                                                          7.x-1.3               
 Fields              Entity Reference (entityreference)                                     7.x-1.4               
 Fields              Field collection (field_collection)                                    7.x-1.0-beta12        
 Fields              Field Permissions (field_permissions)                                  7.x-1.0               
 Fields              Inline Entity Form (inline_entity_form)                                7.x-1.8               
 Fields              Phone (phone)                                                          7.x-1.x-dev           
 Fields              Serial (serial)                                                        7.x-1.5               
 Fields              URL (url)                                                              7.x-1.0               
 Flags               Flag (flag)                                                            7.x-3.9               
 Flags               Flag abuse (flag_abuse)                                                7.x-2.0               
 Homebox             Homebox (homebox)                                                      7.x-2.0-rc3           
 IB                  Kabinet v2 (kabinet)                                                   7.x-1.0-dev           
 Mail                SMTP Authentication Support (smtp)                                     7.x-1.7               
 Multilingual        Localization update (l10n_update)                                      7.x-2.1               
 ODT                 Ophir ODT Importer (ophir_odt_import)                                  0.03                  
 Other               404 Navigation (navigation404)                                         7.x-1.x-dev           
 Other               Author Pane (author_pane)                                              7.x-2.0               
 Other               Better Formats (better_formats)                                        7.x-1.0-beta2         
 Other               Cabinet (cabinet)                                                      7.x-0.1-dev           
 Other               Chain Menu Access API (chain_menu_access)                              7.x-2.0+0-dev         
 Other               Colorbox (colorbox)                                                    7.x-2.13              
 Other               Comment goodness (comment_goodness)                                    7.x-1.4+3-dev         
 Other               Commerce Tuner (commerce_tuner)                                        7.x-1.x-dev           
 Other               Email Registration (email_registration)                                7.x-1.3               
 Other               Entity API (entity)                                                    7.x-1.8               
 Other               Entity tokens (entity_token)                                           7.x-1.8               
 Other               Flat comments (flatcomments)                                           7.x-2.0               
 Other               IB Local Converter (local_converter)                                   7.x-1.0-dev           
 Other               IB Product Register (product_register)                                 7.x-1.0-dev           
 Other               Libraries (libraries)                                                  7.x-2.3               
 Other               My Custom (my_custom)                                                                        
 Other               Node Convert (node_convert)                                            7.x-1.2+11-dev        
 Other               Panels bootstrap layouts builder (panels_bootstrap_layout_builder)     7.x-3.1               
 Other               Pathauto (pathauto)                                                    7.x-1.3               
 Other               Quote (quote)                                                          7.x-1.1-beta3         
 Other               Redirect (redirect)                                                    7.x-1.0-rc3           
 Other               Sphinx Search (sphinxdrupal)                                                                 
 Other               Token (token)                                                          7.x-1.7               
 Other               Transliteration (transliteration)                                      7.x-3.2               
 Other               User Stats (user_stats)                                                7.x-1.x-dev           
 Panels              Panels (panels)                                                        7.x-3.9               
 Path Breadcrumbs    Path Breadcrumbs (path_breadcrumbs)                                    7.x-3.3               
 Path Breadcrumbs    Path Breadcrumbs UI (path_breadcrumbs_ui)                              7.x-3.3               
 Permissions         Override node options (override_node_options)                          7.x-1.13              
 Placebo             Webform pop-up. (webform_popup)                                                              
 Remote Data         Remote Data (remotedata)                                               7.x-0.1               
 Rules               Inline Conditions (inline_conditions)                                  7.x-1.0-alpha7        
 Rules               Rules (rules)                                                          7.x-2.10              
 Rules               Rules UI (rules_admin)                                                 7.x-2.10              
 SEO                 Page Title (page_title)                                                7.x-2.7               
 Services            Commerce Services (commerce_services)                                  7.x-1.x-dev           
 Services            Services (services)                                                    7.x-3.20              
 Services            Services field collection (services_field_collection)                                        
 Services - servers  REST Server (rest_server)                                              7.x-3.20              
 Services - servers  XMLRPC Server (xmlrpc_server)                                          7.x-3.20              
 Spam control        CAPTCHA (captcha)                                                      7.x-1.4               
 Spam control        Honeypot (honeypot)                                                    7.x-1.22              
 Spam control        reCAPTCHA (recaptcha)                                                  7.x-2.2               
 User interface      jQuery Update (jquery_update)                                          7.x-2.7               
 User interface      Wysiwyg (wysiwyg)                                                      7.x-2.3               
 User interface      WYSIWYG: MarkItUp - BBCode (wysiwyg_markitup_bbcode)                                         
 Views               IB Cabinet (ib_cabinet)                                                                      
 Views               Views (views)                                                          7.x-3.15              
 Views               Views Bootstrap (views_bootstrap)                                      7.x-3.1+30-dev        
 Views               Views Bulk Operations (views_bulk_operations)                          7.x-3.4               
 Views               Views Data Export (views_data_export)                                  7.x-3.2               
 Views               Views UI (views_ui)                                                    7.x-3.15              
 Webform             Webform (webform)                                                      7.x-4.15

D8 Modules:

--------------------------------- --------------------------------------------------------- --------- --------------- 
  Package                           Name                                                      Status    Version        
 --------------------------------- --------------------------------------------------------- --------- --------------- 
  Field types                       Address (address)                                         Enabled   8.x-1.2        
  Administration                    Admin Toolbar (admin_toolbar)                             Enabled   8.x-1.20       
  Administration                    Admin Toolbar Extra Tools (admin_toolbar_tools)           Enabled   8.x-1.20       
  Core                              Automated Cron (automated_cron)                           Enabled   8.4.0-dev      
  Core                              Block (block)                                             Enabled   8.4.0-dev      
  Core                              Custom Block (block_content)                              Enabled   8.4.0-dev      
  Core                              Breakpoint (breakpoint)                                   Enabled   8.4.0-dev      
  Custom                            Bugtracker (bugtracker)                                   Enabled                  
  Spam control                      CAPTCHA (captcha)                                         Enabled                  
  Core                              CKEditor (ckeditor)                                       Enabled   8.4.0-dev      
  Custom                            CKEditor BBCode (ckeditor_bbcode)                         Enabled                  
  Core                              Color (color)                                             Enabled   8.4.0-dev      
  Media                             Colorbox (colorbox)                                       Enabled   8.x-1.4        
  Core                              Comment (comment)                                         Enabled   8.4.0-dev      
  Commerce                          Commerce (commerce)                                       Enabled   8.x-2.0        
  Commerce                          Commerce Cart (commerce_cart)                             Enabled   8.x-2.0        
  Commerce                          Commerce Checkout (commerce_checkout)                     Enabled   8.x-2.0        
  Commerce                          Commerce Order (commerce_order)                           Enabled   8.x-2.0        
  Commerce                          Commerce Price (commerce_price)                           Enabled   8.x-2.0        
  Commerce                          Commerce Product (commerce_product)                       Enabled   8.x-2.0        
  Commerce                          Commerce Store (commerce_store)                           Enabled   8.x-2.0        
  Core                              Configuration Manager (config)                            Enabled   8.4.0-dev      
  Core                              Contact (contact)                                         Enabled   8.4.0-dev      
  Core                              Contextual Links (contextual)                             Enabled   8.4.0-dev      
  Chaos tool suite                  Chaos tools (ctools)                                      Enabled   8.x-3.0        
  Chaos tool suite (Experimental)   Chaos tools blocks (ctools_block)                         Enabled   8.x-3.0        
  Chaos tool suite (Experimental)   Chaos tools Views (ctools_views)                          Enabled   8.x-3.0        
  Field types                       Datetime (datetime)                                       Enabled   8.4.0-dev      
  Core                              Database Logging (dblog)                                  Enabled   8.4.0-dev      
  Development                       Devel (devel)                                             Enabled                  
  Core                              Internal Dynamic Page Cache (dynamic_page_cache)          Enabled   8.4.0-dev      
  Core                              Text Editor (editor)                                      Enabled   8.4.0-dev      
  Other                             Entity (entity)                                           Enabled                  
  Field types                       Entity Reference Revisions (entity_reference_revisions)   Enabled                  
  Core                              Field (field)                                             Enabled   8.4.0-dev      
  Core                              Field UI (field_ui)                                       Enabled   8.4.0-dev      
  Field types                       File (file)                                               Enabled   8.4.0-dev      
  Core                              Filter (filter)                                           Enabled   8.4.0-dev      
  Core                              Forum (forum)                                             Enabled   8.4.0-dev      
  Custom                            FrontSlider (frontslider)                                 Enabled                  
  FZ152                             FZ152 (fz152)                                             Enabled                  
  FZ152                             FZ152 — Contact (fz152_contact)                           Enabled                  
  Core                              Help (help)                                               Enabled   8.4.0-dev      
  Core                              History (history)                                         Enabled   8.4.0-dev      
  Spam control                      Honeypot (honeypot)                                       Enabled   8.x-1.27       
  Other                             IB (ib)                                                   Enabled                  
  Field types                       Image (image)                                             Enabled   8.4.0-dev      
  Fields                            Inline Entity Form (inline_entity_form)                   Enabled   8.x-1.0-beta1  
  Multilingual                      Language (language)                                       Enabled   8.4.0-dev      
  Core                              Layout Discovery (layout_discovery)                       Enabled   8.4.0-dev      
  Field types                       Link (link)                                               Enabled   8.4.0-dev      
  Other                             IB Local Converter (local_converter)                      Enabled                  
  Multilingual                      Interface Translation (locale)                            Enabled   8.4.0-dev      
  Core                              Custom Menu Links (menu_link_content)                     Enabled   8.4.0-dev      
  Core                              Menu UI (menu_ui)                                         Enabled   8.4.0-dev      
  Core (Experimental)               Migrate (migrate)                                         Enabled   8.4.0-dev      
  Core (Experimental)               Migrate Drupal (migrate_drupal)                           Enabled   8.4.0-dev      
  Core (Experimental)               Migrate Drupal UI (migrate_drupal_ui)                     Enabled   8.4.0-dev      
  Migration                         Migrate Plus (migrate_plus)                               Enabled                  
  Migration                         Migrate Tools (migrate_tools)                             Enabled                  
  Migration                         Drupal Upgrade (migrate_upgrade)                          Enabled                  
  Custom                            My User Account (my_user_account)                         Enabled                  
  Core                              Node (node)                                               Enabled   8.4.0-dev      
  Field types                       Options (options)                                         Enabled   8.4.0-dev      
  Core                              Internal Page Cache (page_cache)                          Enabled   8.4.0-dev      
  Layout                            Page Manager (page_manager)                               Enabled                  
  Layout                            Page Manager UI (page_manager_ui)                         Enabled                  
  Panels                            Panels (panels)                                           Enabled                  
  Core                              Path (path)                                               Enabled   8.4.0-dev      
  Other                             Pathauto (pathauto)                                       Enabled                  
  Other                             Profile (profile)                                         Enabled   8.x-1.0-rc1    
  Core                              Quick Edit (quickedit)                                    Enabled   8.4.0-dev      
  Core                              RDF (rdf)                                                 Enabled   8.4.0-dev      
  Spam control                      reCAPTCHA (recaptcha)                                     Enabled   8.x-2.2        
  Web services                      RESTful Web Services (rest)                               Enabled   8.4.0-dev      
  Web services                      REST UI (restui)                                          Enabled   8.x-1.15       
  Core                              Search (search)                                           Enabled   8.4.0-dev      
  Web services                      Serialization (serialization)                             Enabled   8.4.0-dev      
  Core                              Shortcut (shortcut)                                       Enabled   8.4.0-dev      
  Other                             State Machine (state_machine)                             Enabled   8.x-1.0-beta3  
  Core                              System (system)                                           Enabled   8.4.0-dev      
  Core                              Taxonomy (taxonomy)                                       Enabled   8.4.0-dev      
  Field types                       Telephone (telephone)                                     Enabled   8.4.0-dev      
  Field types                       Text (text)                                               Enabled   8.4.0-dev      
  Custom                            Tickets (tickets)                                         Enabled                  
  Other                             Token (token)                                             Enabled   8.x-1.0        
  Core                              Toolbar (toolbar)                                         Enabled   8.4.0-dev      
  Core                              Tour (tour)                                               Enabled   8.4.0-dev      
  Core                              Update Manager (update)                                   Enabled   8.4.0-dev      
  Core                              User (user)                                               Enabled   8.4.0-dev      
  Core                              Views (views)                                             Enabled   8.4.0-dev      
  Views                             Views Bootstrap (views_bootstrap)                         Enabled                  
  Core                              Views UI (views_ui)                                       Enabled   8.4.0-dev      
  Extensible BBCode                 Extensible BBCode (xbbcode)                               Enabled
maxocub’s picture

maxocub CreditAttribution: maxocub as a volunteer commented
Title: Can't create forum container after migration from D7 » [D7] Forum containers are migrated as forums
Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs tests

Let's fix the forum container migration bug here and move the investigation for the creation of new containers after a migration to a follow-up: #2914249: Translated forum vocabulary migration creates duplicate forum vocabularies.

I also created a follow-up to move the forum logic from the taxonomy migrations to new forum migrations, as per #56 and #59: #2914251: Move forum related logic from taxonomy migrations to new forum migrations.

quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented

Great, now that there are follow up issues, all the points raised since the last RTBC have been addressed. Retesting the latest patch, #51, now.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs review » Reviewed & tested by the community

I concur with @quietone. Pre-RTBC on the assumption that #51 will pass Drupal CI.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented
Issue tags: +Needs tests

It still seems worth having some automated testing to ensure that new forum containers can be created after the migration. If the follow-up isn't reproducible, then we won't have an issue to add that test coverage. On the other hand I'm also struggling to see how the migration could interfere with this. Leaving RTBC but re-tagging for discussion at least.

maxocub’s picture

maxocub CreditAttribution: maxocub as a volunteer commented
FileSize
2903007-72.patch7.48 KB
interdiff-2903007-51-72.txt3.83 KB

Good thing we took a second look at those tests because the current test was not testing anything since the forum module was not enabled.

I added test coverage to see if we can add a new forum container after a migration, not sure if adding it programmatically is enough though, since it's not exactly like clicking on the button and filling a form, but I guess it's close enough.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Reviewed & tested by the community » Needs review

Kicking back to NR since there is a new patch.

dillix’s picture

dillix CreditAttribution: dillix commented
Status: Needs review » Needs work

Tried #72 and it does't fix my issue. I investigated a bit more my problem and found that during migration drupal creates new vocabulary Forums and imports here terms and after migration I have 2 vocabs (Forums and Форумы), so relation between forum functionality and vocabulary breaks. Also when I open taxonomy_forums field settings for forum content type there isn't any taxonomy vocab checkbox marked here (but before migration it has been set to Форумы after I enabled forum module).

PS: In my D8 site config I have only one language (Russian) and it sets as default in settings.

PPS: May be #2914251: Move forum related logic from taxonomy migrations to new forum migrations will fix my issues...

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented
Assigned: heddn » masipila

Thanks @dillix for the additional information. I'm trying to reproduce this now.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented

@dillix, I tried to reproduce this issue using Finnish language but I was not able to reproduce this.

We have two separate but related bugs here:

1. We migrated Forum containers as Forums because we were missing "is this forum taxonomy term a container" check
- This bug is being fixed under this issue

2. We seem to still have another bug which you are reporting in #74 which seems to have something to do with language setup
- maxocub created a separate issue for this in #68, see #2914249: Translated forum vocabulary migration creates duplicate forum vocabularies

Let's continue debugging the bug 2 in #2914249: Translated forum vocabulary migration creates duplicate forum vocabularies.

Cheers,
Markus

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented
Assigned: masipila » Unassigned
quietone’s picture

quietone CreditAttribution: quietone as a volunteer commented
Status: Needs work » Needs review

@masipila, thanks for trying to reproduce the issue dillix has reported and the summary.

Setting this back to NR because it does fix the problem of forum container migrating as forums and followups have been created to continue working on the problem dillix is still having and to move the code to the forum module.

masipila’s picture

masipila CreditAttribution: masipila as a volunteer commented
Status: Needs review » Reviewed & tested by the community

Patch 51 was reviewed in #52 and #53 and manually tested in #61.

Maxocub enhanced the test coverage in patch #72 and thus #71 has been addressed. There are no code changes between 51-72, the only changes are related to tests.

I did yet another manual test with patch #72 and results were as expected.

Follow-up issues have been created to a) address the separate issue that dillix is still reporting and b) to separate Forum related term migration logic away from the general term migrations.

I've read through patch #72 and as far as I can see everything is OK and all review comments have been addressed.

Setting to RTBC.

Cheers,
Markus

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

Committed/pushed to 8.5.x and cherry-picked to 8.4.x. Thanks!

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented
Status: Reviewed & tested by the community » Fixed

  • catch committed 2c7b21d on 8.5.x 
    Issue #2903007 by rakesh.gectcr, dipakmdhrm, maxocub, dillix, masipila,...

  • catch committed dd8253b on 8.4.x 
    Issue #2903007 by rakesh.gectcr, dipakmdhrm, maxocub, dillix, masipila,...
phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Reviewed & tested by the community » Fixed
Issue tags: -Needs tests

Marked Fixed by @catch, but somehow it didn't take.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.