This is follow up to #2682461: Add links to documentation in all migrate process plugins.

Decide if the the documentation for the process plugins should be in the codebase itself, rather than maintaining it in the handbook, and if so, do it here.

It was agreed here to put the documentation in the codebase. But then mikeryan pointed out that the resulting patch was too much to review in one go and should be split into one issue per process plugin. This issue then became a META to manage all that work. All the child issue are now created. Each has a patch, made from the larger one here. And as of this writing they pass all tests and thus ready for review.

So, the remaining task is to complete the child issues.

The process plugins are;

Comments

quietone created an issue. See original summary.

quietone’s picture

I like this idea.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mikeryan’s picture

+1

phenaproxima’s picture

Me too!

The question is, should we do this instead of #2682461: Add links to documentation in all migrate process plugins or as a follow-up to it?

quietone’s picture

As a follow up. I prefer to help people right now. I can't think of a reason to delay that help, especially since the other issue is RTBC.

phenaproxima’s picture

Status: Active » Postponed

Okay, let's postpone this on that then.

quietone’s picture

StatusFileSize
new1.98 KB

Even though this is postponed, I wanted to learn more about documentation etc etc. I had lots of questions on this and finally decided to just post something and get some feedback.

This patch has some added documentation for the substr process plugin. I've only done one plugin because it would be sensible to get one mostly agreed upon before anyone works is done on the other plugins.

Leaving as postponed since this can wait.

quietone’s picture

Add to migration documentation meta issue.

jonhattan’s picture

+++ b/core/modules/migrate/src/Plugin/migrate/process/Substr.php
@@ -9,7 +9,39 @@
+ * @code
+ * process:
+ *   new_text_field:
+ *     plugin: substr
+ *     source: some_text_field
+ *       start: 2
+ *       length: 7
+ * @endcode
+ *
+ * If some_text_field was 'Marie Skłodowska Curie' then
+ * $destination['new_text_field'] would be 'Skłodowska'.
+ *
+ * The PHP equivalent of this is:
+ *
+ * @code
+ * $destination['new_text_field'] = substr($source['soume_text_field'], 6, 10)
+ * @endcode

Number don't add up!

+++ b/core/modules/migrate/src/Plugin/migrate/process/Substr.php
@@ -19,6 +51,8 @@ class Substr extends ProcessPluginBase {
   /**
    * {@inheritdoc}
+   *
+   * @throws \Drupal\migrate\MigrateException
    */

Doc standards states that a {@inheritdoc} docstring should not be accompanied by any other text. See https://www.drupal.org/node/2606724

xjm’s picture

Status: Postponed » Active

The other issue has been committed, so we can work on this one again now. Thanks!

quietone’s picture

Assigned: Unassigned » quietone

@jonhattan, thx for the feedback.

I've started on the other process plugins but since this will take a while I'm assigning to myself.

chx’s picture

I put such things in the handbook because the edit process at the time at least was significantly faster than editing core documentation. I am not familiar enough either with the new core development cycles in semver times or the new documentation structure, did this change enough that we want to move from handbook into core?

quietone’s picture

Status: Active » Needs review
StatusFileSize
new48.58 KB

Here is a start on documenting the process plugins code. This pretty much copies the existing handbook data to the doc blocks and @throws for the process plugins that throw exceptions. I also tried to make visible what the source value should be as well as any configuration keys.

Several of the doc blocks are incomplete, especially for the recently changed or added process plugins, like FileCopy and Route. But I refrained from working on those until there was some feedback from anyone more familiar with documentation standards. It would be better to know this is on the right track before spending more time on the details.

quietone’s picture

Assigned: quietone » Unassigned
benjy’s picture

If this is to replace the handbook pages then we should have a follow-up to delete those nodes and redirect the URL's to the pages that are hopefully auto generated from these new docs?

heddn’s picture

Status: Needs review » Needs work

Let's add a TODO for #16. And rather than bikeshed on wording, I'm inclined to get er done and RTBC and improve docs with follow-up issues (as needed).

For the incomplete docs, if you don't think they are complete enough, let's open specific TODOs to improve each of them and get this committed, warts and all.

quietone’s picture

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new49.61 KB

Needed a reroll. Updated dedupe_entity to include the migrate configuration key. And to clarify that the entity id is deduplicated, not the entity, hopefully that will help with the concerns in #18.

quietone’s picture

Follow up issue created as benjy suggested in #16, #2831626: Redirect handbook process plugin documentation to API docs.

quietone’s picture

The only process plugin that needs documentation that is not in this patch is the newish Download plugin. Based on heddn's comment in #17 to get this done and improve docs with follow ups, there is a new issue for that work. #2831630: Add documentation to download process plugin.

mikeryan’s picture

Assigned: Unassigned » mikeryan
mikeryan’s picture

Title: Add process plugin documentation to the codebase » [meta] Add process plugin documentation to the codebase
Assigned: mikeryan » Unassigned
Status: Needs review » Needs work

Well, started going through this, and it's simply too much to do all in one go, especially since there are some particular plugins (I'm looking at you, iterator, and your worst-case-scenario example) that will take some bikeshedding to get nice and clear. Let's split it out per-plugin, so the nasty ones don't delay finishing up the simpler ones, and the simpler ones can be novice tasks (both on the writing and reviewing side).

mikeryan’s picture

Issue tags: +Novice

Actually, opening up child issues would itself be a novice task...

mikeryan’s picture

One more word on the handbook - I think that will still be useful for elaborating on different use cases and more advanced examples of plugin usage, while the code docs should stick to simple straight-forward examples.

quietone’s picture

Issue summary: View changes
mikeryan’s picture

Repurposed the "De-duplication is confusing" issue as the child for documentation the dedupe plugins.

quietone’s picture

Issue summary: View changes
quietone’s picture

Issue summary: View changes
Issue tags: -Novice

Decided that the documentation needed a push. It sasn't too bad once I setup a shell script to generate all the patches. After that was just lots of clicking and (mind-numbing) editing. None of the child issues are tagged novice, which seems right to me, and I'm removing that here.

And now all the issues are made and ready for review. Yay!

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

quietone’s picture

I'm adding what I think are the results of a discussion on IRC (alexpott, mikeryan, phenaproxima, ultimike, RoloDMonkey, wonder95) about the documentation. I wasn't a participant in the discussion, I was asleep at the time.

The API Docs

  1. Are the definitive reference.
  2. Explain what the configuration keys do.
  3. Provides examples.

The Handbook pages

  1. Have a brief overview
  2. Have a link to the API documentation.
  3. Provides examples of various use cases.

What strikes me about this is what type of examples go where? Do we make a distinction based on complexity, rare situation, or even the source (Drupal vs non-Drupal).

phenaproxima’s picture

What strikes me about this is what type of examples go where?

My feeling is that you can never have too many examples, in general -- it's not so much a question of which examples should go where, but how many examples can go where. The API docs should contain two examples at most -- maybe three for more complex process plugins -- and cover the most common use cases. The handbook pages, on the other hand, can have as many examples as we want to put there, covering any range of use cases, from the most basic to the most exotic. What do we think about this?

xjm’s picture

Just a very small whitespace nitpick; these have had lists indented one level too many so please double-check that the patches under review comply with this: https://www.drupal.org/node/1354#lists

Thanks!

heddn’s picture

Status: Needs work » Fixed

There's only a single issue left here. So i'm going to close this down.

#2845485: Refactor and document the MenuLinkParent process plugin is the only outstanding process plugin docs issue left open.

phenaproxima’s picture

Status: Fixed » Active
heddn’s picture

Re-opening. I missed #2845483: Rename Iterator process plugin and add documentation. There are still two issues.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

Status: Active » Needs review

Someone double-check me, but I think we are down to only a single child issue: #2845485: Refactor and document the MenuLinkParent process plugin

If that is true, let's get some attention on it so we can close this thing down.

The last submitted patch, 8: process_doc_2776179-8.patch, failed testing. View results

heddn’s picture

Status: Needs review » Fixed

No feedback. Going to mark this meta fixed as there is only a single child issue open. Thank to all who helped with the docs.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.