EDIT: added the yml and fixed the json data (was a copy/paste error)

Problem: Not able to get the data (id or Name) from the "Sets" array. Goal is to create 4 contents out of this data.

This is the source:

{
  "Time" : "2019-10-08",
  "Centers" : [ {
    "id" : "B01502",
    "CenterId" : 1502,
    "Sets" : [ {
      "id" : "B0150200",
      "Name" : "name_valuex"
    }, {
      "id" : "B0150201",
      "Name" : "name_valueu"
    } ]
  }, {
    "id" : "B01503",
    "CenterId" : 1503,
    "Sets" : [ {
      "id" : "B0150301",
      "Name" : "name_valueb"
    }, {
      "id" : "B0150302",
      "Name" : "name_valuea"
    } ]
  } ]
}

This is my YML

id: test_data
source:
  constants:
    TITLE_PREFIX: 'title prefix - '

  plugin: url
  data_fetcher_plugin: file
  data_parser_plugin: json

  urls:
    - /test.json
    
  item_selector: 1
  fields:
    - name: id
      label: 'id'
      selector: id
  ids:
    id:
      type: string
process:
  title: constants/TITLE_PREFIX
  field_presetid: id 
destination:
  plugin: 'entity:node'
  default_bundle: bundle_name

I have tried to put different values to "item_selector:" like /Centers/Sets or "0" etc.
Modules:
Migrate: Version: 8.7.8
Migrate Plus: Version: 8.x-4.2
Migrate Tools: Version: 8.x-4.4
Drupal: 8.7.8
PHP 7.2.22

Issue fork migrate_plus-3087614

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jukka792 created an issue. See original summary.

jukka792’s picture

jukka792 CreditAttribution: jukka792 as a volunteer commented
Issue summary: View changes
jukka792’s picture

jukka792 CreditAttribution: jukka792 as a volunteer commented
Version: 8.x-5.0-rc1 » 8.x-4.2
jukka792’s picture

jukka792 CreditAttribution: jukka792 as a volunteer commented
Issue summary: View changes
heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Tag1 Consulting for Drupal Association, European Commission and European Union Institutions, Agencies and Bodies commented

Create a custom source plugin that parses the source and combines all the values into a single array. Use of a PHP generator (yield) is encouraged.

marvil07’s picture

marvil07 CreditAttribution: marvil07 at Tag1 Consulting commented
Version: 8.x-4.2 » 6.0.x-dev
Assigned: Unassigned » marvil07

marvil07 opened merge request !87

marvil07’s picture

marvil07 CreditAttribution: marvil07 at Tag1 Consulting commented
Assigned: marvil07 » Unassigned
Category: Support request » Feature request
Status: Active » Needs review

It will be nice to have a way to walk through a set of items nested in all the item selector items.

I have started one custom data parser plugin that does that.
It basically extends the json data parser plugin, and then uses a given selector to unify its values across all the elements.

E.g. the following the original example, the following source configuration bits will extract the four items.

data_parser_plugin: json_subitem
item_selector: Centers
subitem_selector: Sets

On the ideal world, jq pipelines can do the work once, but if the source endpoints cannot be changed, something like this data parser helps to produce a set that is sensible to walk.

New commits on new 3087614-json-subitems topic branch and related new MR !87.

- 3f81967 #3239180: Start a JSON subitems data parser