Hello,

I have a kind of weird CSV file to import. The structure is like that:

ID | TITLE | TERMS
1 | Title1 | term1
1 | Title1 | term2
2 | Title2 | term3
2 | Title2 | term1
3 | Title2 | term5

I would like to get all the terms from different lines to the same node. In the end, the node with "Title 1" will have taxonomy terms of "term1" and "term2".

If I directly run the importer, I know that only last value will be taken to the term field.
Anyone have an idea?

Similar but different posts:
#1315728: Having multiple values in a XML imported into one target value
#1231522: Having multiple values in a CSV imported into one target value.
The difference is they are trying to get multiple terms from different columns of the same row. I would like to get multiple terms from different rows.

Comments

GuyPaddock’s picture

Component: Feeds Import » Code
Issue summary: View changes
sitewits’s picture

Found a quick and dirty workaround for it due to the lack of time. The main issue is the core FeedProcessor.inc erases all field values before handing it further. This means new values are added to an empty field.

I commented out the Line 730 of plugins/FeedProcessor.inc (7.x-2.0-beta1+36-dev) in the map method:

if (isset($fields[$target_name])) {
	// Empty the target for the specified language.
730:	////$target_item->{$target_name}[$mapping['language']] = array();
}
else {
	// Empty the whole target.
	$target_item->{$target_name} = NULL;
}

This now works for multiple terms on terms and nodes. I did two 80K+ imports no problem, one for terms and one for nodes.

So a solution might be to delegate emptying the field to the proper processor such as FeedsTermProcessor or FeedsNodeProcessor to handle the 3 different update settings and not wipe the original field values.

[ ] Do not update existing terms
[ ] Replace existing terms -> free to empty the field
[X] Update existing terms -> don't empty the field, append instead

I'm not too familiar with the internals of Feeds and wouldn't know if this makes sense, but this works alright for a one off import like mine.

MegaChriz’s picture

@sitewits
The reason that the target is emptied first is to keep the source value and the target value in sync. The list would also grow with each subsequent import.

For example, take the following source:

guid,title,tags
1,Item 1,cat 1|cat 2

Say that the source "tags" is mapped to a taxonomy reference field and the Feeds Tamper plugin "explode" is used to get multiple values for this target.
This will result into the following item:

guid: 1
title: Item 1
tags:
- cat 1
- cat 2

Now the source is updated, the item no longer belongs to category 2, but instead to category 3:

guid,title,tags
1,Item 1,cat 1|cat 3

When values would be appended then the imported item would look like this:

guid: 1
title: Item 1
tags:
- cat 1
- cat 2
- cat 1
- cat 3

See, "cat 3" was added, but "cat 2" wasn't removed. And "cat 1" was added again.

And when the item is imported again (because something else on the item is changed or the option "skip hash check" is on), this would be the result:

guid: 1
title: Item 1
tags:
- cat 1
- cat 2
- cat 1
- cat 3
- cat 1
- cat 3

One way of solving this problem could perhaps be to implement the hook hook_feeds_before_update() and use the function entity_load_unchanged() in there to pull in the previous values (not tested, but it may work). For an example, you could take a look at the code of the sandbox module Feeds empty. In particular, see feeds_empty_post_process_callback() in feeds_empty.feeds.inc. The module in question was created to solve the problem of not emptying the target when the source is empty. That's a different use case, but it has similarities with this request.

davidjguru’s picture

Hello, I'm working with Drupal 7, migrating nodes and I have the same problem:

- I have a multiple value field, media type for videos and images
- I'm importing from another site and I need a way to add new values to multi-value fields.

So, How to retain existing values and append new values?

Well, I tried the suggested solution in #3, but doesn't work for me:

//$target_item->{$target_name}[$mapping['language']] = array();
(Line 809 in my case, plugins/FeedProcessor.inc)
But it does not work, the new elements are still saved after emptying the field. Any other way to prevent the previous emptying of the field?

Drupal Core 7.54
Feeds Import 7.x-2.0-beta4
Media Feeds 7.x-2.0-alpha1

t_stallmann’s picture

I'm running into this as well on a recent project (still stuck on D7, unfortunately!), where we solved it by creating a custom feeds processor class extending NodeProcessor and overwriting

  protected function map(FeedsSource $source, FeedsParserResult $result, $target_item = NULL) {

from the parent FeedsProcessor class. In this case we knew the fieldnames specifically, but I imagine the checks could be expanded to be more general.

The solution which worked for us was to first implement @sitewits suggestion (in a custom class rather than editing the Feeds code directly), and then after the $value = $this->getSourceValue($source, $result, $mapping['source']); line, adding:

$fieldname = $mapping['target'];
$existing_values = array_map(function ($v) { return $v['value']; }, $target_item->$fieldname[LANGUAGE_NONE]);
if (!is_array($value)) {
      $value = [$value];
}
$value = array_unique(array_merge($value, $existing_values));
sort($value);