A weird need to list fields twice for the XML plugin [#2835038]

Without listing fields in source plugin configuration I'm getting PHP Notice in the console:

Invalid argument supplied for foreach() DataParserPluginBase.php:209

and nothing really works. The reason is that XML plugin requires to list fields twice: one time in the source plugin configuration and another one - in the process configuration. We are not machines, doubling the same information is definitely a weird thing. This should be optional and can be used only for renaming.

If that's because of the ID key(s) then please create a config entry like key_selector or something like this.

Comments

Comment #1

13 December 2016 at 10:34

OnkelTem created an issue. See original summary.

Comment #2

OnkelTem CreditAttribution: OnkelTem at Adyax commented 13 December 2016 at 10:36

Issue summary:

View changes

Comment #3

Grayside CreditAttribution: Grayside at Phase2 for Norwegian Cruise Line commented 14 December 2016 at 02:27

The need to specify fields in the source and the process is part of the migrate architecture in Core itself.

The reason is that the source configuration to pull fields is an extraction phase to identify what you need, while the transformation phase in the process plugins might combine one or more of the source fields in mapping it to a given destination property.

It would not be at all difficult to write a PHP migration class that would auto-configure process configuration for every source field to use the get plugin or other in each case. The constructor would be a good spot in this case.

I did something like that dynamic by the actual source data in #2789699, this would be even simpler to do by field definitions.

Comment #4

OnkelTem CreditAttribution: OnkelTem at Adyax commented 14 December 2016 at 09:47

@Grayside

> The reason is that the source configuration to pull fields is an extraction phase to identify what you need,

There's already a way to identify what I need, it is item_selector.

That's why I'm saying there's room for something to facilitate this, but it should not be default.

Comment #8

OnkelTem CreditAttribution: OnkelTem at Adyax commented 14 December 2016 at 20:47

> But you might not want everything to be mapped, so do you allow configuring exclusions?

I'm not saying I need it mapped automatically, I just don't want it to look like this, where the source/fields part is just not needed at any rate:

source:
  plugin: some_feed
  data_fetcher_plugin: http
  data_parser_plugin: xml
  urls: http://someurl
  item_selector: /items/item
  fields:
     - name: id
       selector: id
     - name: title
       selector: title
     - name: created
       selector: created
     - name: updated
       selector: updated
     - name: city
       selector: city
     - name: country
       selector: country
     - name: summary
       selector: summary
  ids:
    id:
      type: string

process:
  id: id
  title: title
  created: created
  updated: updated
  city: city
  country: country
  summary: summary