Comments

quietone created an issue. See original summary.

jofitz’s picture

Status: Active » Needs review
StatusFileSize
new903 bytes

Added configuration keys. Not quite sure what else to include on this one because it is an abstract and so examples aren't really applicable.

heddn’s picture

Status: Needs review » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -15,6 +15,12 @@
  * The base class for all source plugins.

This isn't strictly true. Contrib doesn't have to use this base class. Let's remove the word "all" and I'm fine. Yes, I know, it is a super small nit.

jofitz’s picture

Status: Needs work » Needs review
StatusFileSize
new572 bytes
new914 bytes

Good point, @heddn. Change made.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

Docs seem sufficient, given this is an abstract class.

wim leers’s picture

Priority: Normal » Minor
Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,13 @@
+ * - high_water_property: Information on the property used as the high-water mark.

Violates 80 cols rule.

heddn’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new841 bytes
new910 bytes

A trivial fix of the comment length. Docs only change. Putting this back to RTBC.

gábor hojtsy’s picture

Status: Reviewed & tested by the community » Needs work

As per issue summary "make sure [...] working with translations are included. Add suitable examples too.". So looks like this is not getting to where it intended to go.

heddn’s picture

I must be missing something then, because I cannot find anything special for i18n migrations. Which means we probably need an IS update. I do see in getIds() where in I18nProfileField we are adding multiple columns to make the ids unique, but that has another issue to update its docs in #2728869: Document public function MigrateSourceInterface::getIds return value better. There doesn't seem to be anything else needed in the source base plugin for i18n migrations.

quietone’s picture

Yes, there isn't anything about translations in SourcePluginBase, so nothing to do there. All available configuration keys are listed, so that is done.

The keys are all optional, so need to have 'optional' added. And, I think the descriptions need to be more specific.

+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,13 @@
+ * - cache_counts: Whether this instance should cache the source count.
+ * - skip_count: Whether this instance should not attempt to count the source.
+ * - track_changes: Flags whether to track changes to incoming data.

These should indicate the data type of these keys and say something like, "If set xyz will happen". And can we make the language consistent, two being with 'Whether' and one with 'Flags'.

+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,13 @@
+ * - high_water_property: Information on property used as the high-water mark.

We should tell the reader that this is an array of the name of the property begin used for high water and it's table alias.

pritishkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new913 bytes
new977 bytes
heddn’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs issue summary update

this doesn't really do what was asked in #10. Specifically, it isn't more specific. Nor is does it touch the high_water_property.

pk188’s picture

Status: Needs work » Needs review
StatusFileSize
new1.08 KB
new904 bytes
heddn’s picture

Status: Needs review » Needs work
Issue tags: +Novice

This doesn't seem like a troublesome Novice issue. But before posting a patch, please review all the comments in #8 and #10. There's been a couple patches now that have very little value and haven't done really anything. Except give the possibility of commit credit. Please be careful to review the feedback and list what you are providing in the patch and what is still outstanding.

  1. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,13 @@
    + * Available configuration keys:
    

    There's no examples as requested in #8.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,13 @@
    + * - cache_counts: (optional)(Integer) If set cache the source count.
    + * - skip_count: (optional)(Integer) If set not attempt to count the source.
    + * - track_changes: (optional)(Integer) If set track changes to incoming data.
    

    There should be a space between )(

    This isn't any more specific yet either.

  3. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,13 @@
    + * - high_water_property: Array of names used for high-water mark.
    

    Missing (optional). Missing better description too.

mikeryan’s picture

+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,13 @@
+ * - high_water_property: Array of names used for high-water mark.

high_water_property is not an "array of names" - it is an array containing a single property name, an an optional table alias.

pk188’s picture

StatusFileSize
new992 bytes

Fixed #14 and #15. Working on example as mentioned in #8 and update it once completed.

pk188’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 16: 2862670-16.patch, failed testing. View results

pk188’s picture

Status: Needs work » Needs review
StatusFileSize
new990 bytes
shashikant_chauhan’s picture

StatusFileSize
new1.04 KB
new982 bytes

Fixed 80 character length issue.

pk188’s picture

@shashikant_chauhan in your patch there are more than 80 characters in a line.

shashikant_chauhan’s picture

@pk188, You don't have to count the characters in patch file instead you should count in file after applying the patch.

See https://www.drupal.org/docs/develop/standards/coding-standards#linelength for more information.

maxocub’s picture

Assigned: Unassigned » maxocub

Assigning for review

maxocub’s picture

Assigned: maxocub » Unassigned
Status: Needs review » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,14 @@
+ * - cache_counts: (optional) (Integer) If set cache the source count.
+ * - skip_count: (optional) (Integer) If set not attempt to count the source.
+ * - track_changes: (optional) (Integer) If set track changes to incoming data.
+ * - high_water_property: It is an array of single value (optional table alias)
+ *   which have the high_water_property corresponding to it.

This documentation doesn't seem very helpful to me and the formulation could be improved.

Those configuration keys are associated with the class properties, which are already documented, and should be coherent. For instance, cache_counts, skip_count & track_changes are said to be boolean on the properties, but integers here.

Also, even if this is an abstract class, we should and can provide some examples, something like:

@code
source:
  plugin: some_source_plugin_name
  cache_counts: true
  ...
@endcode

It should also be mentioned that the high_water_property cannot be used with the track_changes property, as said on line 179:
throw new MigrateException('You should either use a highwater mark or track changes not both. They are both designed to solve the same problem');
So this should absolutely be included in the documentation and we should have code examples for both possibilities, explaining their differences.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

edysmp’s picture

StatusFileSize
new1.4 KB
new1.61 KB
edysmp’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work
tacituseu’s picture

Status: Needs work » Needs review

Unrelated failure.

quietone’s picture

  1. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,36 @@
    + * - cache_counts: (optional) (boolean) If set cache the source count.
    + * - skip_count: (optional) (boolean) If set not attempt to count the source.
    + * - track_changes: (optional) (boolean) If set track changes to incoming data.
    

    Boolean isn't necessary, and we haven't added the type like this in other migrate documentation.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,36 @@
    + * - high_water_property: It is an array of single value (optional table alias)
    

    high_water_property is optional

  3. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,36 @@
    + * The highwater mark or track changes not should be set at the same time.
    

    This needs to be move to just below 'available configuration keys' and not in an example. It is vital information. And maybe change it to 'The high_water_property and track_changes are mutually exclusive.'

The examples need to explain what they do. That is, what happens when the source plugin is configured as in the example.What will happen? What will not?

maxocub’s picture

Status: Needs review » Needs work
Issue tags: -Novice

I don't think this should be tagged Novice since it requires an good understanding of the migration system.
Back to Needs work.

jofitz’s picture

Status: Needs work » Needs review
StatusFileSize
new1.84 KB
new2.34 KB

Removed instances of "(boolean)".
Marked high_water_property as optional.
Moved and edited the high_water_property sentence.

Added explanation to the examples.

heddn’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,44 @@
    + * - cache_counts: (optional) If set cache the source count.
    ...
    + * - track_changes: (optional) If set track changes to incoming data.
    

    Nit: comma between set, and the rest of the sentence.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -13,7 +13,44 @@
    + * - skip_count: (optional) If set not attempt to count the source.
    

    Nit: Add comma. And if set, /do/ not attempt to count.

  3. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -28,22 +30,28 @@
    + * Changes to incoming data are watched (because track_changes is true) which
    

    comma or period should be inserted before word, "which".

  4. +++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
    @@ -28,22 +30,28 @@
    + * In this example skip_count is true which means count() will not attempt to
    

    Comma after, "In this example". And another comma after word "true".

dhruveshdtripathi’s picture

Status: Needs work » Needs review
StatusFileSize
new1.85 KB
new1.89 KB

Changes made according to comment #33

quietone’s picture

One more thing, sorry. I find this sentence a bit clumsy at the end.

This example uses the plugin "some_source_plugin_name" and caches the count of available source records to save calculating every time count() is called.

Maybe just change "every" to "it every"?

jofitz’s picture

StatusFileSize
new1.85 KB
new878 bytes

Made change suggested by @quietone in #35.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

All feedback is now addressed.

heddn’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,44 @@
+ * - high_water_property: (optional) It is an array of single value (optional
+ *   table alias), which have the high_water_property corresponding to it.

I'm doing an implementation with high_water_property right now. And 1) it is complicated, 2) this little two line comment isn't helping.

From getHighWatermark()

The high water mark defines a timestamp stating the time the import was last run. If the mark is set, only content with a higher timestamp will be imported.

From getHighWaterField()
It looks like an array keyed by 'name' and (optionally) 'alias'.

The values of name and alias are concatenated with a period and used in SqlBase for a db query condition.

So, let's reword this:
high_water_property: (optional) It is an array of key/values for a column name and (optional) alias in the source database that defines a timestamp stating the time the import was last run. If the mark is set, only content with a higher timestamp will be imported.

And it looks like our example should add the alias part to the docs.

+++ b/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php
@@ -13,7 +13,44 @@
+ *   high_water_property:
+ *     name: something

Should be:

high_water_property:
  name: changed
  alias: n

The above will get converted into n.changed when running the database query.

heddn’s picture

Issue tags: +Vienna2017

Tagging.

rakesh.gectcr’s picture

Assigned: Unassigned » rakesh.gectcr
heddn’s picture

Status: Needs work » Needs review
StatusFileSize
new2.09 KB
new1.75 KB

Addressing my own feedback.

maxocub’s picture

Assigned: rakesh.gectcr » Unassigned
Status: Needs review » Reviewed & tested by the community

Beautiful.

jofitz’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new2.09 KB
new851 bytes

Very minor punctuation correction.

jofitz’s picture

Status: Needs review » Reviewed & tested by the community

I think this can be safely returned to RTBC!

  • Gábor Hojtsy committed 52f65aa on 8.5.x
    Issue #2862670 by Jo Fitzgerald, pk188, heddn, edysmp,...

  • Gábor Hojtsy committed d2ce6be on 8.4.x
    Issue #2862670 by Jo Fitzgerald, pk188, heddn, edysmp,...
gábor hojtsy’s picture

Version: 8.5.x-dev » 8.4.x-dev
Status: Reviewed & tested by the community » Fixed

Thanks all for making sure this looks good! Committed to 8.5.x and cherry-picked to 8.4.x.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.