Issue/Requirement

Accept input of a nested array by a Feed and return values based on a given key.

This issue is based on ingesting a JSON endpoint as a source for a Drupal Feed, though it may be useful for other source formats as well. It is intended to handle a field that contains an associative array, or potentially a nested array with key=>value pairs of string or numeric values. For example:

JSON Data:

"categories":[{"title":"Category Title 0","id":1234}, {"title":"Category Title 1","id":2345}],

PHP Structure:

categories = [
  0[
    'title'=>'Category Title 0', 
    'id'=>'1234'
  ],
  1[
    'title'=>'Category Title 1',
    'id'=>'2345'
  ]
];

Proposed resolution

This involves creating a new plugin called "ArrayKeyValue" for the Tamper module, which is used by the Feeds Tamper module.
It will accept an associative array (single or multi-level) and recursively find keys that match the given key string from the plugin configuration settings. The values for these keys will be returned by the plugin as a flat indexed array.

Note: It may still be necessary to use other plugins (such as 'explode') to further modify the returned array so that it can be used by Drupal, depending on how the feed mapping is configured.

https://www.drupal.org/project/tamper/issues/3018590
https://www.drupal.org/project/feeds/issues/2453123
https://www.drupal.org/project/feeds_tamper/issues/2978476

Issue fork tamper-3230903

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

kbaringer created an issue. See original summary.

kbaringer’s picture

kbaringer commented
Issue summary: View changes

kbaringer opened merge request !5

kbaringer’s picture

kbaringer commented

Hi jamesdixon & MegaChriz, 3 files were touched as part of this fork:
MODIFIED config/schema/tamper.schema.yml
ADDED src/Plugin/Tamper/ArrayKeyValue.php
ADDED tests/src/Unit/Plugin/Tamper/ArrayKeyValueTest.php

This new plugin will accept input of an array and recursively search for items that match the "key" provided in the plugin config.
It will handle multi-dimensonal (nested) arrays.
The output will be a flat, indexed array of the values that were found matching the given key.

The three tests that I added in ArrayKeyValueTest.php cover these cases:
1) Handling a negative case, where the input is not an array
2) Input a flat associative array
3) Input a nested associative array.

All tests are now passing.

Please let me know if you have any questions. It would be great to get this merged into a stable release of Tamper as soon as possible so that I (and others) can use it on a live site.
Thank you for all of your great work on this module!

megachriz’s picture

megachriz
Primary language Dutch
commented

Thank you for your contribution! I can mention your work in the Feeds meeting tomorrow. Maybe some of us will then be able to test it soon!

A stable release of Tamper can still take a while. James Dixon has taken a break for community work and I’m focused on getting Feeds to the beta stage first.

kbaringer’s picture

kbaringer commented

Gotcha, thank you MegaChriz! If there is anything I can do to help smooth the way to getting it into a stable release please let me know, but I definitely understand. If there are any questions feel free to reach out. Thanks!

megachriz’s picture

megachriz
Primary language Dutch
commented

Thanks for your offer, @kbaringer :)

There are two Tamper plugins left to be done:
#2976171: Absolute URL Plugin
#2976175: D7 EFQ Finder to D8 Entity finder

And there are a few bugs to be fixed:
#3184186: Keyword filter: variable executed as function, security issue?
#3184133: Warning: mb_stripos(): Empty delimiter in Drupal\tamper\Plugin\Tamper\KeywordFilter->match() (line 234 of /var/www/html/modules/tamper/src/Plugin/Tamper/KeywordFilter.php)
#3185029: Rewrite plugin: use keys of source definition in replacement patterns

Especially getting the EntityFinder issue finished can be quite a challenge.

There may exist a few more Tamper issues that are worth fixing, I haven't looked much into them yet.

kbaringer’s picture

kbaringer commented

Hi @MegaChriz, if I get a chance I'll certainly see if I can take a stab at one of those open bugs. In the meantime I need to get the ArrayKeyValue plugin into my project, so I'm going to spin up a patch. I'll post it here shortly.

megachriz’s picture

megachriz
Primary language Dutch
commented

@kbaringer
You don't necessarily need to upload a patch, one gets autogenerated when using issue forks. See "plain diff" in the issue fork section underneath the issue summary.
Here's the link to the plain diff: https://git.drupalcode.org/project/tamper/-/merge_requests/5.diff

kbaringer’s picture

kbaringer commented

Oops, thanks @MegaChriz! I just saw your last message about the auto-generated patch for the merge request. I compared it with the one I rolled myself and they match perfectly except for the auto-generated one having more granular index values (see below), so I'll go with the one that you shared the link to. Thanks for clueing me in to that!

2c2
< index a6000bb..cf465b2 100644
---
> index a6000bbcdc6497d4cd5230c96cccd3a72f1ca521..cf465b2f0ee7b99c821d47d2f1f6cb491519373b 100644
starlight-sparkle’s picture

starlight-sparkle
Any
commented

Is there an update on this? It seems all the blockers referenced in the thread have been fixed already, and the remaining issue is a merge conflict on tamper.schema.yml.