Problem / motivation
This is a spin off from https://www.drupal.org/node/2853872#comment-12224562. When testing #2853872: Migration for forum and article comments: duplicate comment types and incorrect comment_entity_statistics, I noticed that all forum threads were closed in D8.5.x
Every node has a setting which defines if the comments are open or closed. We are talking about this field in the UI:
Open = value 2
Closed = value 1
The node migrations, for example upgrade_d7_node_forum do not have any migration process for this setting. As a result, every node falls back to the default value of the comment type in question. See root cause analysis in comment #3.
Notes about related issues:
#2887043: D7 comment field instance migration: incorrect mapping for open/closed/hidden default value is not the root cause for this bug because that issue is talking about the default value and this issue is talking about individual nodes.
Proposed resolution
All node migrations (e.g. upgrade_d7_node_forum, upgrade_d7_node_article) should have a process for migrating this node setting. Source in D7 database is comment column in {node} table.
What makes this a bit tricky is that
a) D8 Forum has its own hard coded comment type comment_forum
b) Other D7-D8 migrations create separate comment types / fields with comment_node_{node_type} pattern.
Remaining tasks
Investigate further and identify root cause
Agree approach
Patch so that the comment types are properly handled
Add tests
Review
Commit
Dance all night
Comment | File | Size | Author |
---|---|---|---|
#29 | 2902766-23-29.patch | 18.17 KB | maxocub |
#23 | interdiff-2902766-21-23.txt | 996 bytes | maxocub |
#23 | 2902766-23.patch | 18.13 KB | maxocub |
#21 | interdiff-2902766-19-21.txt | 17.06 KB | maxocub |
#21 | 2902766-21.patch | 18.83 KB | maxocub |
Comments
Comment #2
masipila CreditAttribution: masipila as a volunteer commentedComment #3
masipila CreditAttribution: masipila as a volunteer commentedFirst quick glance to figure out the root cause.
We are talking about this D8 node field in UI:
When this setting is open for an individual node, the forum node has this when the node is checked with Devel.
When the setting is closed, we the status is 1.
I looked at the migration upgrade_d7_node_forum. This migration process does not have any reference to the comment type field. As a result, the default value of the comment type is used for every node.
In plain English:
Comment #4
masipila CreditAttribution: masipila as a volunteer commentedMinor updates to issue summary.
Comment #5
masipila CreditAttribution: masipila as a volunteer commentedComment #6
masipila CreditAttribution: masipila as a volunteer commentedReading the API, it looks to me that this mapping needs to be added to D7NodeDeriver::getDerivativeDefinitions.
Assinging to myself. I'll try to write the first draft of the patch in the next couple of days.
Cheers,
Markus
Comment #7
masipila CreditAttribution: masipila as a volunteer commentedPatch attached.
Comment #8
masipila CreditAttribution: masipila as a volunteer commentedApparently the patch didn't break any existing tests but I guess we should add tests to cover the additional mapping. I'll leave that for someone who is more experienced with automated tests.
Issue summary / remaining tasks updated accoringly.
Cheers,
Markus
Comment #9
heddnThe patch in #7 doesn't necessarily account for installs using the minimal install profile, does it? At least for the article, it doesn't. I think we might be better off fixing the standard install profile and forum module to use the correct pattern. Back to NW for tests.
Comment #10
larowlanThis isn't something the standard profile does - it must be specific to migrations.
Default behaviour in standard profile is 'comment' with forum having its own predictable comment type (via its /config/install/ folder).
Forum needs its own comment type, lots of code throughout the module assumes a specific field, and that is enforced via forum/config/install.
The pattern of using comment_{node type} is specific to migrate, see \Drupal\comment\Plugin\migrate\source\d7\CommentType::prepareRow where it allows for nuances between different node types in Drupal 7 - instead of finding unique combinations, it opts to create one for each node type.
Comment #11
masipila CreditAttribution: masipila as a volunteer commentedGood morning,
The pattern of using pattern of using comment_node_{node type} is indeed coming from other migrations. Phenaproxima was trying to get rid of this back in June. See #2886839: D7 comment type migration should emulate Standard.
Summary:
1. D8 Standard installation profile uses comment comment type for Article.
2. D8 Forum must have its own comment type comment_forum. This is hard coded in Forum module just like it was mentioned also above in #10.
3. Phenaproxima was trying to get rid of the comment_node_{node_type} pattern in #2886839: D7 comment type migration should emulate Standard but that fell short. Conclusion was that we need to have separate comment types for each node types because the D7 comments might have additional fields in addition to the comment subject, body and author. These additional fields may be different for each node type in D7.
The patch in #7 assumes that 1-3 are valid. Do we have alternative approaches for this specific issue?
Cheers,
Markus
Comment #12
masipila CreditAttribution: masipila as a volunteer commentedHmm...
One thing we could consider would be as follows:
In D7NodeDeriver, we could check if the D8 node type has a comment field. If yes, then we add the mapping
foo/0/status: comment
where foo is the comment field we detected. If we find no comment fields, we don't add this mapping.Worth thinking: it is possible that D8 site has multiple comment fields. Should we apply the mapping to all of them or just the first? If first, do we mean first what happens to come first from the D8 config check? Or first based on the weight in form settings? Or first in Display settings? If first in display settings, which view mode? Note that I'm not suggesting this heavy determination logic, I'm just making a point that if we can't assume comment_node_{node_type}, we need to think this through.
Markus
Comment #13
masipila CreditAttribution: masipila as a volunteer commented@heddn,
You proposed in https://www.drupal.org/node/2853872#comment-12242655 that we should let the other migrations create comment type comment_node_article . Now if the D8 site was installed with Standard installation profile, the Article has two comment types / comment fields.
Hence: the concern of multiple comment fields per content type I raised in #12 above is valid if we let Article to have two comment fields.
Proposed solution for this issue about node comment statuses:
1. In D7NodeDeriver we check from the D8 configuration if the content type has any comment fields. The result of this check can be 0-N.
2. If we find any comment fields in D8, we map
foo/0/status: comment
for each D8 comment field.This should make your proposed approach in https://www.drupal.org/node/2853872#comment-12242655 feasible.
Issue summary updated with the proposed approach.
Cheers,
Markus
Comment #14
heddnre #13: the issue with looking into the destination from the source deriver is that the source should have zero knowledge of the destination. None at all.
If having multiple comment fields is a problem, then we can update https://www.drupal.org/docs/8/upgrade/known-issues-when-upgrading-from-d... accordingly. But we already tell folks they should start their D8 site with minimal configuration and content. If they decide to use anything beside the minimal install profile, they'll have to do some work to bring themselves back to a minimal configuration state.
Comment #15
masipila CreditAttribution: masipila as a volunteer commentedTo verify that I have not misunderstood anything:
1. Regarding Articles
We have now concluded both in this issue and in #2853872: Migration for forum and article comments: duplicate comment types and incorrect comment_entity_statistics that we should not have special handling for Articles because of D8 Standard installation profile. If the site builder is using D8 Standard, he/she needs to manually remove the comment field comment from Article node type. This will be added to the known issues documenation heddn linked above.
2. Regarding the comment fields comment_node_{node_type}
These comment types / fields are created by the other D7-D8 migrations. I'm not quite sure if comments 9-10 by heddn and larowlan were arguing against mapping to these comment fields. That is what we need to achieve if we want to preserve the node level comment statuses.
3. About the new patch
Setting this to Needs Review to get feedback on the approach. This is still not complete as we need to add tests so once you provide feedback on the general approach, please set back to Needs work for at least tests.
Issue summary is also updated.
Comment #16
masipila CreditAttribution: masipila as a volunteer commentedComment #17
dillix CreditAttribution: dillix commentedComment #18
heddn#15. Just needs tests, back to NW for that.
Comment #19
jofitz CreditAttribution: jofitz at ComputerMinds commentedHere's a few tests.
(No need for an interdiff because all the changes are in the test_only patch)
Comment #21
maxocub CreditAttribution: maxocub commented@Jo Fitzgerald: Good start! But I think we should also test nodes with closed comments, and also forum nodes with closed and open comments.
Also did some clean up of the test file.
Comment #23
maxocub CreditAttribution: maxocub commentedFixed failing tests.
Comment #25
masipila CreditAttribution: masipila as a volunteer commentedThanks for adding the test Jo Fitzgerald and maxocub!
I updated the issue summary to help reviewers. Remaining tasks at the moment are:
- Review by maintainers
- Commit
- Dance all night
Cheers,
Markus
Comment #26
heddnLooks good now. There's full test coverage and we've picked up a few more edge cases.
Comment #28
heddnComment #29
maxocub CreditAttribution: maxocub as a volunteer and commentedHere's the reroll.
Comment #30
heddnCompared the diff stats, and we don't seem to have lost anything. Back to RTBC, assuming tests come back green.
Comment #33
catchCommitted/pushed to 8.5.x and cherry-picked to 8.4.x. Thanks!