Thank you everyone for the feedback on my version of the patch. I will make an update this week and roll a new version.

Files: 
CommentFileSizeAuthor
#232 add_feeds_integration-1063434-232.patch7.62 KBjunaidpv
#218 field_collection_feeds-1063434-218.patch8.08 KBsherakama
#196 add_feeds_integration-1063434-196.patch5.23 KBdpfitzsi
#190 add_feeds_integration-1063434.patch4.88 KBiajay
#184 add_feeds_integration-1063434-184.patch4.49 KBNovitsh
#177 field_collection-feeds-1063434-177-7.patch4.59 KBNovitsh
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View
#173 field_collection-feeds-1063434-173-7.patch4.78 KBveronicaSeveryn
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View
#167 field_collection-feeds-1063434-167-7.patch4.32 KBveronicaSeveryn
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View
#153 field_collection-feeds-1063434-153.patch9.57 KBGaëlG
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds-1063434-153.patch. Unable to apply patch. See the log in the details link for more information. View
#152 interdiff-121-152.txt1.69 KBbbdata
#152 field_collection-feeds-1063434-152.patch6.29 KBbbdata
PASSED: [[SimpleTest]]: [MySQL] 164 pass(es). View
#143 field_collection_1063434_143.patch8.8 KBk_zoltan
PASSED: [[SimpleTest]]: [MySQL] 163 pass(es). View
#130 field_collection-feeds-1063434-130.patch8.76 KBlachezar.valchev
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#123 explode_twice.zip1.26 KBjrao
#121 field_collection-feeds-1063434-121.patch5.8 KBqueenvictoria
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#121 interdiff-1063434-119-121.txt1.87 KBqueenvictoria
#119 field_collection-feeds-1063434-119.patch5.32 KBhumansky
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#109 field_collection-feeds-1063434-109.patch5.45 KBarh1
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds-1063434-109.patch. Unable to apply patch. See the log in the details link for more information. View
#102 field_collection-feeds-1063434-102.patch5.13 KBxcf33
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#96 field_collection-feeds_integration-new.patch11.96 KBpraandy
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#93 field_collection-feeds-1063434-93.patch5.96 KBaaronbauman
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#91 field_collection_feeds_multiple.patch4.51 KBxcf33
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection_feeds_multiple.patch. Unable to apply patch. See the log in the details link for more information. View
#89 field_collection-feeds_integration-1063434-89.patch4.51 KBJohn Bickar
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#83 field_collection-feeds_integration-1063434-83.patch4.51 KBbtopro
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration-1063434-83.patch. Unable to apply patch. See the log in the details link for more information. View
#82 field_collection-feeds_integration-1063434-82.patch4.61 KBDavid_Rothstein
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#82 interdiff-16-82.txt2.3 KBDavid_Rothstein
#70 field_collection-feeds_integration-1063434-70.patch3.19 KBbtopro
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View
#69 0001-Do-not-save-empty-items-especially-useful-for-Feeds-.patch2.67 KBGaëlG
FAILED: [[SimpleTest]]: [MySQL] 11 pass(es), 9 fail(s), and 17 exception(s). View
#41 field_collection-feeds_integration-1063434-41.patch3.14 KBarvinsingla
PASSED: [[SimpleTest]]: [MySQL] 78 pass(es). View
#40 field_collection-feeds_integration-1063434-40.patch3.14 KBarvinsingla
PASSED: [[SimpleTest]]: [MySQL] 78 pass(es). View
#36 explode_twice.zip1.23 KBXearwe
#28 field_collection-feeds_integration.patch3.19 KBsir_squall
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration_0.patch. Unable to apply patch. See the log in the details link for more information. View
#16 field_collection-feeds_integration.patch3.13 KBfranz.glauber
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration.patch. Unable to apply patch. See the log in the details link for more information. View

Comments

casey’s picture

Feeds integration seems a good idea.

casey’s picture

Component: Miscellaneous » Code
Category: support » task

Let's make it a task.

bowersox’s picture

+1. We use the Feeds module and it would be great if Feeds would import data into Field Collections.

inversed’s picture

subscribe

Greg McKeen’s picture

We are using Field Collections to have Multiple values on a node of:
Question & Rating.

We provide a form via custom code, and process the submit. You could do something similar with a custom module to import the data from a CSV.

This was found somewhere in the issue queue, but I will repost how we used the code to save field_collections:

// Set up your Node
 global $user;
  $new_node = new stdClass();
  $new_node->type = 'evaluation';
  $new_node->title = '';
  $new_node->nid = '';
  $new_node->is_new = TRUE;
  $new_node->uid = $user->uid;
  $new_node->created = strtotime("now");
  $new_node->changed = strtotime("now");
  $new_node->status = 1;
  $new_node->comment = 0;
  $new_node->promote = 0;
  $new_node->moderate = 0;
  $new_node->sticky = 0;
  $new_node->language = LANGUAGE_NONE;
  // Save your new node
  node_object_prepare($new_node);
  node_save($new_node);
  unset($new_node->is_new);
  
  // $node->nid must be valid at this point
  // Questions and ratings - field_question_ratings_col
  $delta = 0;
  foreach ($input['questions'] as $qid => $rating) {
    $field_collection_item = entity_create('field_collection_item', array('field_name' => 'field_question_ratings_col'));
    $field_collection_item->setHostEntity('node', $new_node);
    $field_collection_item->field_question_ref[LANGUAGE_NONE][$delta]['nid'] = $qid;
    $field_collection_item->field_question_rating[LANGUAGE_NONE][$delta]['rating'] = $rating['rating'][0];

    $field_collection_item->save();
    $new_node->field_question_ratings_col[LANGUAGE_NONE][$delta]['value'] = $field_collection_item->item_id;
    $delta ++;
  }
  // Save the New Node again, this time with Field Collections attached.
  node_object_prepare($new_node);
  node_save($new_node);
  return $new_node;
}
fago’s picture

Title: Import large datasets » add feeds integration

I think the right approach for feeds integration is #1033202: [Meta] Generic entity processor. That generic processor should basically work for field-collection too - if it needs some adaptions we can sub-class the generic solution to add them.

hellomobe’s picture

Checking on the status. It wasn't clear to me if there was a solution in the generic entity thread. I'm using QueryPath as the parser, Node Processor. Fago, thank you for all your hard work and contributions to Drupal.

elly’s picture

(subscribin' here and on the generic entity mapper thread too)

hellomobe’s picture

http://drupal.org/node/1033202#comment-4721020 - Posting here too since it may have more to do with field-collection. The rules entity processor admin ui seems to work okay. I have two field-collections fields created.

If someone understands how the entity processor should or will work, can you give me a brief summary. Looking at it more closely (I'm not a programmer and I'm muddling through to grasp the concept as best I can), it seems that it would just create a field-collection entity, not a node that has a related field-collection. Yes, no? I think most people need it to create the node + field-collection (using this as the next phase to multi-groups). Just want to make sure I'm trying to hack the right approach.

Bevan’s picture

It would be great to get some pointers on how to move this issue forward. I may need it soon and can probably work on this, though it would take me a while to understand all the moving parts here and I would probably do it the wrong way the first time.

hellomobe’s picture

Just putting a thought out there since this is really over my head. Could we model the feed integration as a mapper like Location Mapper? Is location an "entity".

Would a sponsor pool help to get this going (some solution whatever that path takes)? I'm willing to chip in.

tim.plunkett’s picture

Status: Active » Postponed
franz’s picture

I don't get why this is so attached to the issue on #12. Using this new generic processor, I'd have to create another importer for every field collection I have in my node type.

I have a complex node type with several field collections, and I want to import at once. Would be a lot easier to have a mapper "Field Collection 1: Field 1", etc...

Did I get wrong?

hellomobe’s picture

+1 for going this direction. I also noted this in #11 referencing the location mapper. It's kind of the same concept. I agree, as I understand it, that the generic processor just does one entity at a time, versus a field-collection + node at the same time, which is the most likely use case of a field-collection entity. With the generic entity, how would the field-collection know which nid to correspond to if it is imported separately (at a different feed process) than the node?

Should we change the status to active to get help with a mapper? I've tried creating one myself, but no success.

franz.glauber’s picture

I did create one, but it have some issues, it doesn't work with field collections inside field collections, because it is impossible to determine the target field collection.

franz.glauber’s picture

FileSize
3.13 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration.patch. Unable to apply patch. See the log in the details link for more information. View

Here it goes, can be useful as a starting point.

franz’s picture

damn, wrong account =/

philipz’s picture

Patch #16 shows the field-collection fields as targets for import. I'm looking through code but can't see anything about multiple value field-collection fields. Is this supported in your patch franz? I mean let's assume csv file import and then we're having columns for each field-collection fields with somehow (semicolon) separated values for each column multi value?
I'm going to try that and maybe modify it if it doesn't work.

franz’s picture

#18 It is supposed to work, yes.

However, if you have a multiple-value field inside a multiple-value field-collection, it is impossible for feeds to determine which value belongs to which field-collection.

philipz’s picture

Ok so just to be sure let's say I have two fields in my field-collection ("fruit" and "color"). The field-collection field is unlimited.
Now to import two values into this field I have cvs file looking like this:

"fruit","color"
"apple;banana","red;yellow"

I map fruit source column to target fruit field and do the same for color.

This shuld give me two field-collection values:
1. [ apple ] [ red ]
2. [ banana ] [ yellow ]

The point is how to separate the values in single csv cell?

franz’s picture

I see now, philipz. I'm not sure how is the best way of dealing with multiple values on CSVs (I'm working with XML). Maybe you can use http://drupal.org/project/feeds_tamper to explode the values so they are passed to targets as arrays. You map the fruit to field_collection:fruit and color to field_collection:color and they should work fine.

philipz’s picture

Thanks for this idea - I'm using Feeds Tamper already in other import so I'll give it a try. If all I have to do is explode this into array then this should be easy :)
I'm working with XML files as well so I understand that you're using XML with XPath XML parser for your imports and it's working?

franz’s picture

Yes, works like a charm. =)

zorax’s picture

works like a charm with csv parser and taxonomy list. ;)
thanks

dgastudio’s picture

so, the patch is functional? on import feeds, i only have to use feeds tamper to split array?

franz’s picture

kervi, the explode is only needed if you have multiple values per field like #20

Jason Dean’s picture

#16 working great for me :)

sir_squall’s picture

FileSize
3.19 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration_0.patch. Unable to apply patch. See the log in the details link for more information. View

#19 I have tested the patch but he dident work for the "unlimited value",

A have done a new patch, for fixing that.

echataig’s picture

Tried the patch in #28.

I installed the feeds tamper module and used it to explode multiple values of a csv.

However, only the first element is inserted.

My CSV looks like :

title;body;guid;category;language;finition;ref;price_euro;price_dollar;price_sterling
Product 1;Product 1;REF11;7;en;2,3;REF1B,REF1N;10,11;20,21;30,31

On this example, the field collection {finition:2, ref:REF1B, price_euro:10, price_dollar:20, price_sterling:30} is inserted.
The field collection {finition:3, ref:REF1N, price_euro:11, price_dollar:21, price_sterling:31} was not.

[EDIT]

Tried with Feeds XPath Parser (without feeds tamper) and got the same result.
Here is the XML :

<?xml version="1.0" encoding="UTF-8"?>
<products>
    <item>
        <title>Produit import XML</title>
        <body>Produit import XML</body>
        <guid>REF11</guid>
        <categorie>7</categorie>
        <langue>fr</langue>
        <finition>
            <id_finition>2</id_finition>
            <reference>REF1B</reference>
            <prix_euro>10</prix_euro>
            <prix_dollar>20</prix_dollar>
            <prix_sterling>30</prix_sterling>
        </finition>
        <finition>
            <id_finition>3</id_finition>
            <reference>REF1F</reference>
            <prix_euro>11</prix_euro>
            <prix_dollar>21</prix_dollar>
            <prix_sterling>31</prix_sterling>
        </finition>
    </item>
</products>

Context : //item
Retrieval of id_finition : finition/id_finition

Only the first finition is inserted.
I'm not an expert in Xpath, so maybe I missed something.

If anyone has a solution, it would be greatly appreciated.

franz’s picture

Have you tried the patch on #16 too? AFAIR I used it for "unlimited" values without issues.

echataig’s picture

Thanks a lot !
Patch in #16 works perfectly, included with unlimited field collections.

Patch in #28 produces result provided in #29.

rsgracey’s picture

We applied #16, and nothing changed--still no field collection targets in the importer. When we applied the .dev version of field collection module, we got a white screen. FWIW.

franz’s picture

rsgracey, can you post versions of drupal and feeds ? Maybe that could help.

Also, try clearing caches as this is a new hook implementation.

GaëlG’s picture

#16 worked for me with Field collection 7.x-1.0-beta3 and Feeds 7.x-2.0-alpha4+40-dev.

arkjoseph’s picture

This patch is a god send. Is there any chance #16 can be applied to 6.x ? Thanks for the awesome support.

For completeness, the attached is a sample of the csv I am attempting to import with field collections.

http://drupal.org/files/Screen%20Shot%202012-04-06%20at%2012.05.15%20PM.png

Many thanks.

Xearwe’s picture

FileSize
1.23 KB

Hi,

I was in front of the same problem as many of you lately: I wanted to import large amount of data, using feeds, into nodes having a "field_collection" field. This "field_collection" field is made of two fields: format, which is a unique "Term reference" field, and "Languages", which is an infinite "Term reference" field.

My goal was to set up a database of films, each film having several "physical support", and each support having a "format" and several "languages". (for instance the movie "Avatar" can be available on two DVDs: first one will have languages English and French, and second one Egyptian and Spanish)

My CSV file has the following format:

title;support_format;support_languages
AVATAR;DVD|DVD;English,French|Egyptian,Spanish

Here is how I solved this:

  • I used
    • Feeds
    • Feeds Tamper
    • Field_collection
  • I created a new plugin to Feeds Tamper by copying plugin "Explode": I named it "Explode Twice" since its purpose is to explode twice the value of the field "support_languages" (You'll find this plugin attached, it must be added to directory sites/all/modules/feeds_tamper/plugins)
    This plugin is really a "quick and dirty made" one, please review it if you plan to use it on a production environment.
  • I used patch provided in comment #16 to make fields of my "field_collection" field appear in the "targets" list
  • I configured mapping in my Feeds importer to use "Explode Twice" for "support_languages" field, first separator being "|" and second one ","

==> Everything went well and my films are now created with correct "supports"

Thank you,
Alex

emptyvoid’s picture

I applied the patch from comment 16 and setup a multi-field entity with unlimited nodes on a parent content type.

It worked just fine right out of the box with no tweaks for fixes.

I'm not sure what the patch in 19 does to "fix" it but I found it the patch in 16 to work as advertised.

presleyd’s picture

I'm not finding the same success. I'm using a Field Collection made up of two fields: an entity reference and an integer. Both show up fine in the mapping UI but running the importer does not seem to bring either in. I wondered if the entity reference was causing the problem so I removed it from the import but to no avail. A infinite field collection made up of a single value integer didn't import either. I'm using Field Collection 7.x-1.0-beta4 and Feeds 7.x-2.0-alpha4+48-dev

Edit: My integer field had decimals in it and the entity reference doesn't work. Using a node reference from References and a text field it worked fine. In short I think this is working properly.

Shiraz Dindar’s picture

success with #16. thanks!

arvinsingla’s picture

FileSize
3.14 KB
PASSED: [[SimpleTest]]: [MySQL] 78 pass(es). View

The patch in #16 has been working well for us save one issue. We are trying to fill in field collections where the data contains 0's. Currently the code does the following:

    if (empty($v)) {
      // Avoid creation of empty field collections
      continue;
    }

This is causing any 0's from being skipped. As a trade off I have updated the patch to include the following.

    if (empty($v) && $v !== 0) {
      // Avoid creation of empty field collections
      continue;
    }
arvinsingla’s picture

FileSize
3.14 KB
PASSED: [[SimpleTest]]: [MySQL] 78 pass(es). View

Uhg... Ignore the patch above I actually had to make it $v != 0 instead of $v !== 0

johnv’s picture

@arvinsingla,
try this:

-if (empty($v)) {
+if (!isset($v)) {

see also http://zainal.wordpress.com/2006/04/25/8/

franz’s picture

timfernihough’s picture

Hi franz.glauber,

I am working with arvin.singla on the same issue and our particular implementation requires the ability for us to map to a field collection nested inside a field collection. In your patch, for field_collection_feeds_processor_targets_alter, you immediately begin by ensuring that the nested entity is not of type field collection which of course is the reason why we cannot map to a nested field collection in feeds. Is there a reason behind why you did this? We have temporarily commented out this check on our local and we can then see the ability to map to these fields but we are concerned you did this for a reason.

Can you please share why you put in that check? Thanks so much for your time.

franz’s picture

hi timfernihough,
AFAIR, this doesn't work well with multiple items (a multiple field_collection inside a multiple field_collection), beacuse Feeds cannot discern which piece of content belongs to which. If that's not your case, it should work ok too.

timfernihough’s picture

Thanks franz! After some debugging we were able to determine that is exactly what is happening.

For now, we've been able to settle by writing a hook to manipulate our incoming data feed on the fly and to avoid the need for feeds to be able to support nested data collections. If time permits, we may see if we can help by writing an update to the patch to help feeds be able to discern which content belongs to which field collection.

yurgon’s picture

#16 Dont work with term link field ;(
any ideas?

franz’s picture

@yurgon, what do you mean with don't work? Doesn't show up as a target or shows up and doesn't set a value?

GaëlG’s picture

#42 is indeed needed when empty values make sense, for example with csv files like:
organization,contactsname,contactsmail
"Drupal","John|Jack|Joe","john@drupal.test||joe@drupal.test"

emilyf’s picture

#16 is working for me, great work!

Norberto Ostallo’s picture

Status: Postponed » Needs review

#16 is working fine for me too.
Why is this issue postponed? I don't think we need to wait for #1033202: [Meta] Generic entity processor to commit this as long as we simply add field collection fields as mapping targets. I will set this to "needs review", but feel free to prove me wrong on this. We could at least provide a way to import field collection fields with this patch until a generic entity feeds importer is created.

emilyf’s picture

seems like a good point -- once i added this patch from #16 and made the modifications i mention here http://drupal.org/node/1033202#comment-6218624 it allowed me to map my entity with field collection fields.

liquidcms’s picture

@franz and @tim, re: #46, yes, this is exactly what we are trying to do as well... import into a collection which in turn has a collection.

@tim - can you share what the hook code was you used for doing this?

@franz or @tim - i'll take a look at this as well tonight; but if either of you are interested in making this support nested collections; the project i am on could likely fund this work.

@franz... thanks a ton for patch in #16; seems to be working very well. with the feeds_tamper module and a little code in a _feeds_after_parse and a _feeds_presave we have been able to import into a bilingual node with a multi collection field

liquidcms’s picture

made a few assumptions and went for broke.. and surprised i got valid data imported (but not correct).

my structure is a node with a collection field (up to 3) and in that collection i have another collection (unlimited)

i removed the line in patch in #16 disallowing nested collections and sure enough i now get my nested collection showing up in the mapping options.

then i went out on a limb as to how to add data to the cell to represent this and assume feeds_tamper could do a nested explode.

so cell has this for one of the fields in the nested collection: yes,no%%yes,no,maybe%%yes,no,never

so basically in the 3 top level collections:
yes,no
yes,no,maybe
yes,no,never

and, oddly enough.. no errors and what i get imported is:
yes
no
yes

so only 1 item added to nested collection and seems random; but nested explode must have at least worked.. :)

liquidcms’s picture

ok, so yes, no, yes result is simply because feeds_tamper doesn't do what i hoped for explode.. it simply explodes into 1 long array as opposed to making nested arrays.. so i get yes, no, yes, no, maybe, yes, no, never..

liquidcms’s picture

pretty late here so maybe not thinking all that straight.. but i think if we could get feeds_tamper to give a nested array then this should work but the field_collection_feeds_set_target() i think needs a recursive section which i don't see here.

liquidcms’s picture

wow.. and this works.. :)

so the recursion is there in the fact that it calls the callback with an array; which in turn calls the same routine using the items of the array.

so the only issue here is getting the feeds_tamper module to do a nested explode.

liquidcms’s picture

ok, long night.. the patch i added here: #1688536: support nested explodes allows feeds_tamper to create a nested array.

that combined with the patch in #16, except removing the line that prevents it from doing nested, allows importing into nested collections.

charlie-s’s picture

@franz #16 working most well with an unlimited field_collection that contains many fields, using the JSONPath parser and Feeds dev. Hoping I don't have to stumble down the nested-unlimited-field-collections with this project, but who knows. You need to supply a link to paypal you beer (or coffee/tea).

yurgon’s picture

hi,
Dont update value of reference field in field_collection
Any ideas?

Thx

liquidcms’s picture

hmm.. although my solution in #58 allows me to import into nested collections; it breaks re-importing (i.e. importing modified record to same node).

dotpex’s picture

gmclelland’s picture

The patch in #16 worked for me. Thanks everyone, huge timesaver!

seehive’s picture

Great!!! Patch in #16 works perfectly, included with unlimited field collections. just need to explode the "," for that particular field

argo27’s picture

Title: add feeds integration » patch application problem

Hi I have copied the patch in #16 renamed in fcollection.patch
Save the file to the root directory of feeds module sites/all/modules/feeds/
Then apply the patch by using git bash application
I typed

git apply -v fcollection.patch

and it returns and error

"checking patch field_collection.module...
error: field_collection.module: No such file directory"

I hve a field_collection module installed in my site.
So what I did to experiment is to copy the

field_collection.module from field_collection module folder and paste it
inside feeds module folder and run the patch again.

Then I got this error:

"error: patch_failed: field_collection.module:1142
error: field_collection.module: Patch does not apply."

How can I apply the patch successfuly? I badly needed this field-collection - feeds mapping integartion.

Im using drupal acquia in windows 7 OS

tim.plunkett’s picture

Title: patch application problem » add feeds integration
Status: Needs review » Needs work

The patch no longer applies. Someone will need to reroll it.

argo27’s picture

Thanks for the reply.
So what can I do for now?

argo27’s picture

Status: Needs work » Needs review
GaëlG’s picture

FileSize
2.67 KB
FAILED: [[SimpleTest]]: [MySQL] 11 pass(es), 9 fail(s), and 17 exception(s). View

If you applied #42 but do not want empty field collection items to be saved, here's a patch.

btopro’s picture

FileSize
3.19 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

reroll of #28 against latest checkout of dev for testing

btopro’s picture

One issue noted with the targets in this patch: It's unable to do nested field collections. For example I have a type that goes node->field_collection->field_collection. it doesn't drill down into the deeper field_collection

btopro’s picture

removing

// Thou shalt not attempt Sorcery!
  if ($entity_type == 'field_collection_item') {
    return;
  }

Enables at least 1 level of depth to sub-field_collections and works!

btopro’s picture

Here's a helpful transformation method for getting your csv file to conform to the standard needed in order to import to feeds. This takes multiple rows and globs them together based on the unique key for import (in my case 'course')

// convert csv to an associative array
function csv_to_array($input, &$header, $delimiter=',') { 
  $data = array(); 
  $csvData = str_getcsv($input, "\n"); 
  foreach($csvData as $csvLine){ 
    if(is_null($header)) $header = explode($delimiter, $csvLine); 
    else{
      $items = explode($delimiter, $csvLine); 
      for($n = 0, $m = count($header); $n < $m; $n++){ 
        $prepareData[$header[$n]] = $items[$n]; 
      } 
      $data[] = $prepareData;
    }
  }
  return $data; 
}

$header = NULL;
$rows = csv_to_array(file_get_contents('import.csv'), $header);
$record = array();
// the key we want to group everything against
$primary_key = 'course';
// loop through and reassemble based on primary key
foreach ($rows as $row) {
  if (isset($record[$row[$primary_key]])) {
    foreach ($row as $item => $value) {
      // if this is not the primary key then append its value to its location
      if ($item != $primary_key) {
        $record[$row[$primary_key]][$item].=":$value";
      }
    }
  }
  else {
    // set values for the first time they are found
    $record[$row[$primary_key]] = $row;
    unset($record[$row[$primary_key]][$primary_key]);
  }
}
// open new file
$converted = "converted.csv";
$file = fopen($converted, 'w+') or die("can't open file");
// write headers
fwrite($file, implode(',',$header) ."\n");
// iterate and write rows back to file
foreach ($record as $name => $row) {
  $output = $name .','. implode(',',$row) ."\n";
  fwrite($file, $output);
}
fclose($file);
argo27’s picture

I always get this error whenever i apply any of the patch here for fieldcollection module:

error patch failed: field_collection.module:1859
error: field_collection.module:patch does not apply

I use drupal 7 field collection v 7.x-1.0-beta4
feeds v -alpha6

Is it not compatible?

argo27’s picture

How can I re quest for a re-roll of patch in #16 for field collection 7.x-1.0-beta4?

btopro’s picture

need to apply against dev. if you have issues applying then do it manually, it's a pretty basic patch (just remove + signs and add the two functions to the end of the .module)

argo27’s picture

Thank you so much btoro.
Your manual solution recommendation works.
I am no programmer and I have no idea I can just copy and paste the code of the patch to the module file.
Thanks so much. You saved me from several days of headache :)

argo27’s picture

Can i also do the same thing to other patches and modules. What if I see a - sign than a + what does it mean? Does the patch always pasted on the .module file?

btopro’s picture

- means remove that line + means add that line. If you see a minus followed by a + that'll most likely mean replace this with that.

argo27’s picture

Thanks, it makes more sense,

sorry to ask you another question just to confirm
it this code:

diff --git a/entityreference.feeds.inc b/entityreference.feeds.inc
index 093fa5f..d90ecf9 100644
--- a/entityreference.feeds.inc
+++ b/entityreference.feeds.inc
@@ -16,14 +16,42 @@ function entityreference_feeds_processor_targets_alter

should I delete the "@@ -16,14 +16,42 @@" symbols before the function
and this means that I should paste it over the entityreference.feeds.inc file right?

Thanks

dgastudio’s picture

i'm not sure, but after apply the patch, only the array[0] is imported.

i mean, from csv file like this.

"110";"city1,city2,city,3";"1,2,3";"a,b,c";"1,2,3";"i,b,e"

after import, field collection looks like

city, 1, a

nothing more.

field collection it's set to multivalue.

feeds is set to explode columns 2,3,4,5.

column 1 is node id. unique.

field_collection = latest dev
feeds = latest dev

any advice?

David_Rothstein’s picture

FileSize
2.3 KB
4.61 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

The issue raised in #40 (where a field collection with any empty fields inside of it is being treated as an entirely empty field collection and not being saved) is still a problem.

It's possible that #42 combined with #69 is the correct fix for that, but since #69 is a much more far-reaching change and the problem with empty field collections is being specifically introduced here during the Feeds import process, here is a reroll of #16 (which seems like it was the last working patch) with code to deal with that problem during the Feeds import process as well.

With this patch applied, a field collection with empty fields inside of it can still be imported, but an entirely empty field collection will be correctly skipped. (Whether or not the empty field inside of the not-entirely-empty field collection is itself imported or not depends on the properties of the field and is a separate issue, i.e. #1107522: Framework for expected behavior when importing empty/blank values + text field fix).

btopro’s picture

FileSize
4.51 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds_integration-1063434-83.patch. Unable to apply patch. See the log in the details link for more information. View

Same patch with Sorcery added back in, you can do nested field_collections via this method.

Everett Zufelt’s picture

I haven't read through the entire thread. Does the proposed solution deal with removing (deleting) unnecessary field collection items on update?

If you have an entity with a field collection imported, and on update to that entity using fields find that a single item in that collection is no longer required, is it removed?

David_Rothstein’s picture

Just a note: I had a situation where I needed to deal with a multi-valued field inside of a multi-valued field collection, which brings up the question of how do you distinguish between a single field collection item (that contains multiple values for the given field) and multiple field collection items (each of which contains only one value for the given field)? I posted a Feeds Tamper patch at #1819986: Add a plugin for reducing an array to a single element which was helpful in getting that to work.

(This comment is unrelated to Everett's question right above mine, by the way. I don't actually know the answer to that myself.)

g089h515r806’s picture

I have create a module http://drupal.org/project/field_collection_feeds which solve this issue in another way.

yurgon’s picture

this module dont work 4 me

gboggs’s picture

With the latest dev version of Feeds and Field Collection, and the Feeds Field Collection module, I can get field collections to import, and saving user profiles with embedded field collections no longer cause blank collections to be created, but I can no longer use feeds to import new users.

"SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'field_listings_revision_id' cannot be null"

Ideas?

John Bickar’s picture

FileSize
4.51 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

The patch in #83 would not apply for me ("fatal: corrupt patch at line 125") so I added the functions to the module manually, cleared caches twice, and it worked!

I re-rolled the patch against 7.x-1.x-dev.

ashedryden’s picture

Tested patch #89, applied cleanly and seems to have no issues.

xcf33’s picture

FileSize
4.51 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection_feeds_multiple.patch. Unable to apply patch. See the log in the details link for more information. View

Created a patch based on patch #89 (please apply patch on #89 before this).

This is purely for my use case but hopefully it could work for other folks. I wrote this very quickly and it is minimally tested.

In any case my issue was:

I have a field collection field on a node that has "Unlimited value", that field collection item contains 2 fields

1. Name
2. Email

When I create my mapping for feeds I mapped the [field_collection_item:name] and [field_collection_item:email] twice hoping that the field will create multiple instances of the field collection items i.e. $field[LANGUAGE_NONE][$delta];

However, looking at the code it looks like only if the mapping is passing in an array (so have to use feeds tamper or similar tool) in order to create multiple instances of the field collection item. Otherwise $delta will always be 0 and mapping the field collection item fields multiple times will always be overwritten by the last set of mapping.

Comments and feedbacks are welcomed!

arh1’s picture

franz/.glauber: thanks for saving my Friday :)

#16 successfully adds Field Collections as targets available for mapping. Cheers!

aaronbauman’s picture

Status: Needs review » Needs work
FileSize
5.96 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

here's a new patch that includes both #89 and #91 (minus the extraneous whitespace changes)

i'm reverting back to "needs work" because after apply this patch,
i get the following Integrity constraint violation on import:
Integrity constraint violation: 1048 Column 'field_document_details_revision_id' cannot be null
This error may be related to #1810548: "Integrity constraint violation column [xyz]_revision_id cannot be null" when saving entity with attached field collection, in which case we'll have to postpone on that issue.

Regardless, though, the "multiple-values" behavior is a bit off.
For example, a multi-valued data source results in one field collection for each column of each row in the data source, instead of one field collection for each row in the data source.

aaronbauman’s picture

Another issue I've run into with the approach in the current branch of patch development: updates to field collection items are more or less ignored.
If the parent entity is getting updated, and field collections are already assigned, they do not get updated with the new feed data.

I don't think this should necessarily prevent inclusion of this patch, but I thought it worth noting for dev and testing purposes.

ledom’s picture

Tested #93 patch on last dev version, works fine but any news about handling multi-values? One field collection for each row would be better.

praandy’s picture

FileSize
11.96 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

Hi,
I have tried this approach with a complex field collection structure, but it does not work fine.
It couldn't match a nested field structure, for example a xml file:

The source

title field
(text field)
description
(text area)
links
1° item X (r) X X (a, b)
2° item X
3° item X (s) X X (c)





The imported content

title field
(text field)
description
(text area)
links
1° field collection X (r) X X (a)
2° field collection X (s) X X (b)
3° field collection X X (c)

I use json parser and xml parser (for this I have applied this patch http://drupal.org/node/1958660).

We need to transfer content "drupal-to-drupal" and we need to transform entity_reference, files and link of a field collection item in unique link field, so we create a custom "form_callback".

So I develop a patch (I have to thank Hernani for helping me).
Could you verify it, if it could be a good solution?

korn3lius’s picture

Hi,
First of all, thanks for the nice works for field_collection and feeds integration.
Patch #16 work fine for me, however, it only import first field and ignored the others.
Let me explain it clearly.
I had field collection with 2 fields => "visit" and "body weight"
and my csv files look like this:

visit | body weight
1 | 60
2 | 70
3 | 80

Patch #16 simply import the first field (visit 1 with body weight: 60) and ignored visit 2 and 3.
How can I import all the data properly. I think I must miss something here.

My environment:
Drupal: 7.21
Field collection: 7.x-1.0-beta5+0-dev
Feeds: 7.x-2.0-alpha7

Thanks you
Kornelius

vinmassaro’s picture

Status: Needs work » Needs review

Patch in #93 working for me for basic mappings. For multi-value fields, I had to process myself with hook_feeds_presave().

Shiraz Dindar’s picture

vinmassaro, can you post what you did in hook_feeds_presave()?

vinmassaro’s picture

@Shiraz Dindar: Sure. Here is what the XML source looked like that needed to be mapped into the field collection. I am using Feeds XPath Parser.

<internationalActivity>
  <title>UK Partner - South Essex Partnership NHS Foundation Trust</title>
  <location country="United Kingdom"/>
  <location country="Angola"/>
  <location country="Armenia"/>
  <location country="Azerbaijan"/>
  <description>Professional Development Program in International Healthcare Management</description>
  <startYear>2010</startYear>
</internationalActivity>

I had a field collection containing four fields:

  1. Activity title (text field)
  2. Countries (multivalue text field)
  3. Description (text field)
  4. Start year (date field)

I couldn't get the multi-value country field to populate with all the countries. What I did was map the entire internationalActivity element to a temporary target (mapped as raw XML) instead of a field. I then converted it to a PHP array inside hook_feeds_presave() and took it apart to programmatically create the field collection items. If there are items, I remove and recreate them when the feed node updates since this was easier than trying to keep track of them. I used this page as a guide to programmatically create the field collection items. Hope this is helpful!

function yale_profiles_feeds_presave(FeedsSource $source, $entity, $item) {
  if ($source->id == 'faculty_profile_importer') {
    
    // Load node object for modification.
    $node = node_load($entity->nid);
    
    // Remove field collection items to prepare for recreation during import.
    if (!empty($node->field_yp_international_activity[LANGUAGE_NONE])) {
      foreach ($node->field_yp_international_activity[LANGUAGE_NONE] as $key => $value) {
        
        // Take field collection item value and unset it so we aren't left with empty field collection entries.
        $field_collection_item_values[] = $value['value'];
        unset($node->field_yp_international_activity[LANGUAGE_NONE][$key]);
      }
      
      // Delete field collection items.
      entity_delete_multiple('field_collection_item', $field_collection_item_values);
    }
    
    // Process international activity and write it to the field collection.
    if (!empty($entity->{'Temporary target 1'})) {
      
      // If there is only one activity, it is set as a string. Cast it as an array so we can process it the same.
      if (is_string($entity->{'Temporary target 1'})) {
        $entity->{'Temporary target 1'} = (array) $entity->{'Temporary target 1'};
      }
      
      foreach ($entity->{'Temporary target 1'} as &$activity) {
        
        // Convert the raw xml to a PHP array (found from github.com/gaarf/XML-string-to-PHP-array).
        $activity = json_decode(json_encode((array) simplexml_load_string($activity)), 1);
        
        // Normalize the country array for all locations.
        if (!empty($activity['location']['@attributes'])) {
          $attributes = $activity['location']['@attributes'];
          unset($activity['location']['@attributes']);
          $activity['location'][]['@attributes'] = $attributes;
        }
        
        // Convert countries to ISO2 codes since they are saved in this format.
        foreach ($activity['location'] as &$location) {
          if (!empty($location['@attributes']['country'])) {
            $country = $location['@attributes']['country'];
            $location['@attributes']['country'] = countries_country_lookup($country)->iso2;
          }
        }
        
        // Convert year to Y-m-D H:i:s format so it can be created.
        if (!empty($activity['startYear'])) {
          $activity['startYear'] = date('Y-m-d H:i:s', strtotime($activity['startYear'] . '-01-01'));
        }
        
        // Prepare values to create a field collection item.
        $values = array(
          'field_name' => 'field_yp_international_activity',
          'field_yp_activity_description' => array(
            LANGUAGE_NONE => array(
              array('value' => (!empty($activity['title'])) ? $activity['title']: '')
            ),
          ),
          'field_yp_start_year' => array(
            LANGUAGE_NONE => array(
              array('value' => $activity['startYear'])
            ),
          ),
          'field_yp_country' => array(
            LANGUAGE_NONE => array(),
          ),
        );
        
        // Add countries to the $values array.
        foreach ($activity['location'] as &$location) {
          $values['field_yp_country'][LANGUAGE_NONE][] = array_fill_keys(array('iso2'), $location['@attributes']['country']);
        }
        
        // Begin by using the entity_create function to create a new entity of
        // type field_collection_item. The second parameter to the function
        // provides "an array of values to set, keyed by property name".
        $insert_entity = entity_create('field_collection_item', $values);
        
        // Attach the field_collection entity to the application node. This has to
        // happen separately so that it actually works -- you can't just specify
        // this via the $values array.
        $insert_entity->setHostEntity('node', $node);
        
        // Save the entity. Since it's attached to the right application node, this
        // will both create the field_collection entity and update the application
        // node to point to the new field_activity_log record.
        $insert_entity->save();
      }
    }
  }
}
xcf33’s picture

@aaronbauman re #93

I'm working on making the patch work from #93 for the following scenarios:

1. Multiple field collections, this means a node can have multiple field collection attached to it.
2. Multiple value fields in a field collection. This means create a field in the field collection that can have multiple values.

There are more complex cases such as field collection inside of field collection but I think this should work for that or at least maybe not supported at the moment.

It's a good note that field collection is not being updated when parent is updated by feeds, this is something we should build in as well.

xcf33’s picture

FileSize
5.13 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

@aaronbauman re #93

Patch based on #93 that should handle (Please patch against 7.x-1.0-beta5)
1. Multiple field collections, this means a node can have multiple field collection attached to it.
2. Multiple value fields in a field collection. This means create a field in the field collection that can have multiple values

This has been tested initially, need to also look at the update scenarios.
Did not test against field collection inside of field collections.

fehin’s picture

Just applied patch in #102 and below is the message I got

field_collection-feeds-1063434-102.patch:51: trailing whitespace.

field_collection-feeds-1063434-102.patch:54: trailing whitespace.
  $field = isset($entity->$target) ? $entity->$target : array();
field_collection-feeds-1063434-102.patch:56: trailing whitespace.

field_collection-feeds-1063434-102.patch:70: trailing whitespace.
      $sub_field_info = field_info_field($sub_field[0]);
field_collection-feeds-1063434-102.patch:74: space before tab in indent.
              $callback($source, $field_collection_item, $sub_target, $v);
Checking patch field_collection.module...
Applied patch field_collection.module cleanly.
warning: squelched 5 whitespace errors
warning: 10 lines add whitespace errors.
codesmith’s picture

#102 worked for me for a 2 field, field collection. I tried to add multiple values using a colon but that failed - not sure if it's supported or if I didn't set it up right. For example, using something similar to #20:

"fruit","color"
"apple:banana","red:yellow"

I got just one field collection entry:

fruit: apple:banana
color: red:yellow

instead of the expected:

(entry 1)
fruit: apple
color: red

(entry 2)
fruit: banana
color: yellow

Shiraz Dindar’s picture

@vinmassaro -- a belated thanks for sharing your code. I didn't end up proceeding with this but I found your code and comments illustrative nonetheless.

franz’s picture

Status: Needs review » Needs work
+++ b/field_collection.module
@@ -1862,3 +1862,127 @@ function field_collection_devel_generate($object, $field, $instance, $bundle) {
+  ¶
+  // Iterate over all values.
+  $delta = 0;
+  $field = isset($entity->$target) ? $entity->$target : array();    ¶
+  $language = (isset($entity->language)) ? $entity->language : LANGUAGE_NONE;
+    ¶
+  foreach ($value as $v) {
+	    if (isset($field[$language][$delta]['entity'])) {
+	      $field_collection_item = $field[$language][$delta]['entity'];
+	    }
+	    elseif (isset($field[$language][$delta]['value'])) {
+	      $field_collection_item = field_collection_item_load($field[$language][$delta]['value']);
+	    }
+	    if (empty($field_collection_item)) {
+	      $field_collection_item = entity_create('field_collection_item', array('field_name' => $target));
+	    }
+      // let's see if the field in the field collection has multiple cardinality
+      // in case the field is complex type such as field_url:title, field_url:link
+      $sub_field = explode(':', $sub_target);
+      $sub_field_info = field_info_field($sub_field[0]);		
+	    if (isset($_sub_targets[$sub_target]['callback']) && function_exists($_sub_targets[$sub_target]['callback'])) {
+	      $callback = $_sub_targets[$sub_target]['callback'];
+	      if($sub_field_info['cardinality'] == 1) {
+  	      $callback($source, $field_collection_item, $sub_target, $v);
+	      } ¶
+	      else {
+  	      $callback($source, $field_collection_item, $sub_target, $value);
+	      }
+	    }
+	    $field[$language][$delta]['entity'] = $field_collection_item;
+	    $field[$language][$delta]['value'] = $field_collection_item->item_id;
+	    unset($field_collection_item);
+	    if ($info['cardinality'] == 1) {
+	      break;
+	    }
+      if ($sub_field_info['cardinality'] == 1) {
+        $delta++;
+      }	    ¶

There are some ugly tabs and whitespace in this section

arh1’s picture

#102 (patched against 7.x-1.0-beta5) allows the field collections to be set as targets again, but doesn't save the values of text fields in field collections for me.

Digging a bit, I see that in text_feeds_set_target() (from feeds/mappers/text.inc) $field['und'] contains an element, whereas it's an empty array for the non-field collection fields I'm testing. This means that the foreach loop is always broken before setting the value in text_feeds_set_target() via this bit:

    if ($info['cardinality'] == $delta) {
      break;
    }

So I'm trying to understand enough of Feeds and the Field Collection integration discussed here to come up with a reasonable solution... I'm surprised no one else in this thread has hit the same problem.

arh1’s picture

Re: my previous comment...

I'm surprised no one else in this thread has hit the same problem.

Oops, now I see the notes in #102 and #94 above.

Regarding updating existing field collection items, if I clear the value of the field's target in the entity object inside of field_collection_feeds_set_target before the callback with something like this:

unset($field_collection_item->$sub_target);

..then the value of $delta inside of text_feeds_set_target in feeds/mappers/text.inc is set to 0 as desired, so that the field is iterated over once and the value saved correctly. This does not seem to break my initial import of the data, either.

This feels like a hack, though, and I'm sure it breaks some of the scenarios above, so I'm not providing a patch yet. Can anyone who's been working on this longer than I have comment on this approach?

arh1’s picture

FileSize
5.45 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds-1063434-109.patch. Unable to apply patch. See the log in the details link for more information. View

Meh, here's a patch after all, just to make it easier to discuss. Please don't apply to this to any site you care about.

I fixed the whitespace issues per #106, too.

gjones’s picture

Hi Everyone,
I was able to get Feeds 7.x-2.0-alpha8 to work with Field Collection 7.x-1.0-beta5 by utilizing the Explode feature from Feeds Tamper 7.x-1.0-beta4

Step 1: Apply the patch https://drupal.org/node/1063434#comment-7484110 onto Field collectoin 7.x-1.0-beta5
Step 2: This is my how spreadsheet is setup.

Name | Title | Phone | Email
-----------------------
Bob, Joe, Sally | Sir, Mr., Mrs. | 604-555-5555, 604-123-4567, 604-555-0011 | bob@test.com, joe@test.com, sally@test.com
John, Adam, Susan | Dr., Mr., Miss | 604-555-1111, 604-123-2222, 604-333-5511 | john@asdf.com, adam@asdf.com, susan@asdf.com

Note: Each row is an entire field collection and each column is a single field collection field. Also, I'm exporting as a TAB separated value, and having to convert it to UTF-8

Step 3: Setup your feeds mapping to the use the field collection fields.

Step 4: Setup Feeds Tampers
Using Feeds Tamper (must apply quick fix here: https://drupal.org/node/1929312#comment-7166832), I set a new Explode field tamper plugin for the each of my comma separated columns. Note: Your explode string must match your spreadsheet exactly. In other words, if your spreadsheet has a space after the comma, then your explode string needs to have this space for it to work.

Then run the feeds import.

arh1’s picture

Status: Needs work » Needs review

To reiterate quickly, #109 is based on #102 but adds support for updating existing entities (via the line in #108) and cleans up some whitespace problems (I should have provided an interdiff).

I'm taking a top-down look here and providing a pragmatic fix for my use-case, but if #102 was working for people it's worth reviewing.

Status: Needs review » Needs work

The last submitted patch, field_collection-feeds-1063434-109.patch, failed testing.

jeromec’s picture

Hello,

i would like to be able to import a node with this structure :
-nid
-title
-fieldcollection :
--myfield1 (term reference with widget set to Autocomplete term widget (tagging), 1 value)
--myfield2 text

All is working perfectly except the insert of myfield1 who is not working. When i change the widget type of Select list it's works. But i want to be able to insert new term reference, the only way to to this is use the Autocomplete term widget (tagging) type.

i have configured my feed parser to CSV, delimiter (;) , processor node, in my target configuration for myfield1 i have checked Auto create and the search taxonomy term is by Term name.

Feeds : 7.x-2.0-alpha8
Field collection : 7.x-1.0-beta5

i also have applied the patch #16

I dont know what to do.

Can someone help me please ?

azin’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, field_collection-feeds-1063434-109.patch, failed testing.

azin’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, field_collection-feeds-1063434-109.patch, failed testing.

twistor’s picture

Just a note, this should not be using feeds_alter(). A regular drupal_alter() will suffice.

humansky’s picture

FileSize
5.32 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

Patch #109 won't work for me because it is looking for the module in "sites/all/modules/field_collection/". That is probably also why it is failing testing. I've rerolled the patch with a more relative path to the modules' files

humansky’s picture

Status: Needs work » Needs review
queenvictoria’s picture

FileSize
1.87 KB
5.8 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

I've updated #119 as work so far doesn't pass in the mapping to the callbacks on the sub_targets. Thus settings such as 'autocreate' on taxonomy fields don't passed in and therefore new terms don't get created.

See http://drupal.stackexchange.com/questions/83527/how-to-import-term-refer... for more information

Interdiff attached too.

jrao’s picture

This:

+      if($sub_field_info['cardinality'] == 1) {
+        $callback($source, $field_collection_item, $sub_target, $v, $sub_mapping);
+      }
+      else {
+        $callback($source, $field_collection_item, $sub_target, $value, $sub_mapping);
+      }

and this

+    if ($sub_field_info['cardinality'] == 1) {
+      $delta++;
+    }

doesn't seem to be correct, this makes it impossible to import multiple field collection items that has a subfield with multiple values (i.e. #36), in which case this code will just put everything in the first item's multi-valued field.

The code shouldn't care about cardinality of sub fields, or try to breakup the values to fit the sub fields, that's the job of tamper.

I think this should be changed to:

+      $callback($source, $field_collection_item, $sub_target, $v, $sub_mapping);

and

+    $delta++;
jrao’s picture

FileSize
1.26 KB

A revised explode twice tamper plugin, fixed a small bug in the version from #36

ikeigenwijs’s picture

Do I understand correctly
That you need to do 2 imports with different processors

I have csv with
all node information including the collection field 1 level deep only

What i m trieing to do now, not succeeding yet is the following
Is 2 imports
1 for the node data with node processor
1 for the collection field date (https://drupal.org/project/field_collection_feeds)
both mapped to the unique field

csv file:
Title; uniq R number ; a lot of simple fields on the node ; collection field storage
"proc35";35; .a lot of fields; closet1.5.3.2,closet2.3.4.2

thx a lot for the help and directions

chhay’s picture

GaëlG’s picture

#121 with #122 edits did not work well on our multilingual website. We had to change

$language = (isset($entity->language)) ? $entity->language : LANGUAGE_NONE;

to

$language = ($info['translatable'] && isset($entity->language)) ? $entity->language : LANGUAGE_NONE;

in field_collection_feeds_set_target().

GaëlG’s picture

For anyone facing the same need, here is the code that worked for me to import an entity_translated field collection (with this patch: https://drupal.org/comment/8016755#comment-8016755) with parent field set untranslatable and some subfields set as translatable.

/**
 * Implements hook_feeds_processor_targets_alter()
 */
function field_collection_feeds_processor_targets_alter(&$targets, $entity_type, $bundle_name) {
  foreach (field_info_instances($entity_type, $bundle_name) as $name => $instance) {
    $info = field_info_field($name);
    if ($info['type'] == 'field_collection') {
      $new_targets = array();
      feeds_alter('feeds_processor_targets', $new_targets, 'field_collection_item', $info['field_name']);
      foreach ($new_targets as $sub_name => $target) {
        $new_name = $info['field_name'] . ':' . $sub_name;
        $targets[$new_name] = $target;
        if (isset($target['name'])) {
          $targets[$new_name]['name'] = $instance['label'] . ': ' . $target['name'];
        }

        // We override callback for now and retrieve original later.
        $targets[$new_name]['callback'] = 'field_collection_feeds_set_target';
        $targets[$new_name]['form_callback'] = 'field_collection_feeds_mapper_form_language';
        $targets[$new_name]['summary_callback'] = 'field_collection_feeds_mapper_summary_language';
      }
    }
  }
}

/**
 * Process Field Collection items
 */
function field_collection_feeds_set_target($source, $entity, $target, $value, $main_mapping) {
  static $sub_targets = array();

  $args = explode(':', $target);
  $target = array_shift($args);
  $sub_target = implode(':', $args);

  // Retrieve the submapping so we can pass it to the callback
  $sub_mapping = array();
  $config = $source->importer()->getConfig();
  if (!empty($config['processor']['config']['mappings'])) {
    foreach ( $config['processor']['config']['mappings'] as $mapping ) {
      if ( $mapping['target'] == $target . ':' . $sub_target && $main_mapping['language'] == $mapping['language']) {
        $sub_mapping = $mapping;
        $sub_mapping['target'] = $sub_target;
        break;
      }
    }
  }
  // Now we retrieve old callbacks and keep then on a static cache
  if (!isset($sub_targets[$target])) {
    feeds_alter('feeds_processor_targets', $sub_targets[$target], 'field_collection_item', $target, $sub_mapping);
  }
  $_sub_targets = $sub_targets[$target];

  $value = is_array($value) ? $value : array($value);
  $info = field_info_field($target);

  // Iterate over all values.
  $delta = 0;
  $field = isset($entity->$target) ? $entity->$target : array();
  $language = ($info['translatable'] && isset($entity->language)) ? $entity->language : LANGUAGE_NONE;

  foreach ($value as $v) {
    if (isset($field[$language][$delta]['entity'])) {
      $field_collection_item = $field[$language][$delta]['entity'];
    }
    elseif (isset($field[$language][$delta]['value'])) {
      $field_collection_item = field_collection_item_load($field[$language][$delta]['value']);
    }
    if (empty($field_collection_item)) {
      $field_collection_item = entity_create('field_collection_item', array('field_name' => $target));
    }

    // Hack: feeds mapping callbacks (at least text_feeds_set_target), seem to
    // want an empty array for a Field Collection item target, to be sure the
    // item is iterated over at least once.
    //unset($field_collection_item->$sub_target);// Commented out for language handling. Might cause problems?

    // let's see if the field in the field collection has multiple cardinality
    // in case the field is complex type such as field_url:title, field_url:link
    $sub_field = explode(':', $sub_target);
    $sub_field_info = field_info_field($sub_field[0]);
    if (isset($_sub_targets[$sub_target]['callback']) && function_exists($_sub_targets[$sub_target]['callback'])) {
      $callback = $_sub_targets[$sub_target]['callback'];
        $callback($source, $field_collection_item, $sub_target, $v, $sub_mapping);
    }
    $field[$language][$delta]['entity'] = $field_collection_item;
    $field[$language][$delta]['value'] = $field_collection_item->item_id;
    unset($field_collection_item);
    if ($info['cardinality'] == 1) {
      break;
    }

      $delta++;

  }
  $entity->{$target} = $field;
}

/**
 * Implements hook_feeds_presave().
 */
function field_collection_feeds_presave($source, $entity) {
  // Do not save any empty field collection items that may have been created
  // during the mapping process. Since the mapping is done field by field in
  // field_collection_feeds_set_target(), we have to wait until this hook (when
  // the field collection item is fully built up) before we can check if it's
  // empty.
  $config = $source->importer()->getConfig();
  if (!empty($config['processor']['config']['mappings'])) {
    // Find all field collection mappings.
    foreach ($config['processor']['config']['mappings'] as $mapping) {
      if (isset($mapping['target'])) {
        $args = explode(':', $mapping['target']);
        $field_name = array_shift($args);
        if (($field = field_info_field($field_name)) && $field['type'] == 'field_collection') {
          // If the field collection item is empty, do not save it.
          if (!empty($entity->{$field_name})) {
            foreach ($entity->{$field_name} as $langcode => &$items) {
              foreach ($items as $delta => $item) {
                if (isset($item['entity']) && field_collection_item_is_empty($item['entity'])) {
                  unset($items[$delta]);
                }
              }
              // Clean up the final array.
              if (empty($items)) {
                unset($entity->{$field_name}[$langcode]);
              }
              else {
                $items = array_values($items);
              }
            }
          }
        }
      }
    }
  }
}


/**
 * Returns the language summary text for use in a mapper target summary_callback
 *
 * @param array $mapping
 *   Associative array of the mapping settings.
 * @param array $target
 *   Array of target settings, as defined by the processor or
 *   hook_feeds_processor_targets_alter().
 * @param array $form
 *   The whole mapping form.
 * @param array $form_state
 *   The form state of the mapping form.
 *
 * @return string
 *   Returns, as a string that may contain HTML, the summary to display while
 *   the full form isn't visible.
 *   If the return value is empty, no summary and no option to view the form
 *   will be displayed.
 */
function field_collection_feeds_mapper_summary_language($mapping, $target, $form, $form_state) {
  if (module_exists('locale')) {
    list($field_name, $subfield_name) = explode(':', $mapping['target']);
    $info = field_info_field($subfield_name);
    if ($info['translatable']) {
      $language_options = array(LANGUAGE_NONE => t('All languages')) + locale_language_list('name');
      if (empty($mapping['language'])) {
        return t('Language: @search', array('@search' => $language_options[LANGUAGE_NONE]));
      }
      return t('Language: @search', array('@search' => $language_options[$mapping['language']]));
    }
  }
  return FALSE;
}

/**
 * Returns tha language form for use in a mapper target form_callback
 *
 * @param array $mapping
 *   Associative array of the mapping settings.
 * @param array $target
 *   Array of target settings, as defined by the processor or
 *   hook_feeds_processor_targets_alter().
 * @param array $form
 *   The whole mapping form.
 * @param array $form_state
 *   The form state of the mapping form.
 *
 * @return string
 *   Returns, as a string that may contain HTML, the summary to display while
 *   the full form isn't visible.
 *   If the return value is empty, no summary and no option to view the form
 *   will be displayed.
 */
function field_collection_feeds_mapper_form_language($mapping, $target, $form, $form_state) {
  $form = array();
  if (module_exists('locale')) {
    list($field_name, $subfield_name) = explode(':', $mapping['target']);
    $info = field_info_field($subfield_name);
    if ($info['translatable']) {
      $language_options = array(LANGUAGE_NONE => t('All languages')) + locale_language_list('name');
      $form['language'] = array(
        '#type' => 'select',
        '#title' => t('Language'),
        '#options' => $language_options,
        '#default_value' => !empty($mapping['language']) ? $mapping['language'] : LANGUAGE_NONE,
      );
    }
  }
  return $form;
}

vladimir-m’s picture

Hi GaëlG,
Thank you very much for this solution. Works fine for me.

colan’s picture

Issue summary: View changes
Status: Needs review » Needs work

Please provide the above code in patch format.

lachezar.valchev’s picture

Status: Needs work » Needs review
FileSize
8.76 KB
PASSED: [[SimpleTest]]: [MySQL] 132 pass(es). View

Hi all,

I took the initiative to put GaëlG's code from #127 into a patch.

All credits go to GaëlG.

Regards,
Lachezar

pebosi’s picture

Patch and importing of field_collection fields works fine for me. Thanks

snowfly’s picture

Hi which version of feeds does the patch in #130 works with
thanks

pebosi’s picture

I used latest dev.

ikeigenwijs’s picture

A short follow up,

I managed to do the import for field collections in field collections structure,

I added a custom uid (base on different fields) of the parent in all the children and used this field to map all the children to the correct parrent while the csv contains 1 row for each parent or child.

HarlsHicks’s picture

I'm having issues with these patches. I've tried a few of the working ones, even #130. I'm coming up with the same issue, the callback $v is supposed to be an array, but for some reason it's not getting sent as one. I'm using the latest dev of both field collections and Feeds. Here's the error I get with patch #130:

ResponseText: Recoverable fatal error: Argument 4 passed to file_feeds_set_target() must be an array, string given, called in /modules/field_collection/field_collection.module on line 2018 and defined in file_feeds_set_target() (line 51 of /modules/feeds/mappers/file.inc).

Anybody have any ideas?

jason.blanda’s picture

#135 All of the above patches will not work with Feeds 7.x-2.x-dev use Feeds 7.x-2.0-alpha8 #130 worked perfectly.

caw67’s picture

ptach #16 works great! can you implement it in the module?

colan’s picture

Status: Needs review » Needs work

Needs a re-roll based on #136. All patches must apply cleanly to the dev branch.

AtlasX’s picture

Working without errors using patch #121 with a multi-value field collection containing a node reference field and a text field.

Tried patch #130 but gave me this error:

Notice: Undefined index: language in field_collection_feeds_set_target() (line 1976 of /home/ramjet/public_html/testsite/modules/field_collection/field_collection.module).

Still looking to update existing fields which I've been unable to do. It still won't remove fields that are no longer in an updated XML file or make changes to existing ones. It sometimes causes duplicates as well.

I've been using Feeds and importing XML for a while now without this module and it always works great. In that case, my new import seems to overwrite all existing fields and repopulates them from the updated XML, which, again, works great for what I need it for.

dwadson’s picture

#135 - I added a line before 2018 to force $v to an array:

$v = is_array($v) ? $v : array($v);
rcodina’s picture

Patch #16 work for me on Feeds 7.x-2.0-alpha8 However, with Feeds 7.x-2.0-alpha8 I'm not able to import images and files from their URI and I really need this feature. Could anyone do a patch which works with Feeds 7.x-2.x-dev?

johnv’s picture

Title: add feeds integration » Add Feeds integration to FieldCollection

Better title for my dashboard.

k_zoltan’s picture

FileSize
8.8 KB
PASSED: [[SimpleTest]]: [MySQL] 163 pass(es). View

Re-rolled patch from comment 130 #1063434-130: Add Feeds integration to FieldCollection so that it clearly applies and work with the most recent version of Feeds and Field Collection.
This patch also includes the fix suggested in comment 140 #1063434-140: Add Feeds integration to FieldCollection

Please note that the most recent dev vesion of Feeds includes the patch for the general Entity Processor #1033202: [Meta] Generic entity processor https://www.drupal.org/node/1033202

k_zoltan’s picture

Status: Needs work » Needs review

Lets see if this passes the tests ... if yes than I think this is ready to be reviewed.

joshparkinson’s picture

Applied patch #89 and it works perfectly so thanks alot for that.

I have a problem however, let me lay it out.

I have a field collection 'FAQ TAB'
Inside this i have the fields 'Tab Title', 'Tab Content'
'Tab Content' is a field collection containing 'Title', 'Body'
'Tab Content' can have unlimited values, so an unlimited number of title and body fields linked together.

How can i add the multiple title and body fields through the feeds importer?

I know i can use feed tamper and explode it etc, but this doesnt seem to work for unlimited value field collections.

Where do i start in trying to solve this?
Apologies for the lack of effort on my part, only been working within drupal for about a month and situations similar to mine seem not to exists on the internet.

Any help would be massively appreciated

hauruck’s picture

Greetings

I have just patched with the #143 and it works pretty well. My only issue remaining is how I use this feed integration with fields that have multiple values. Here is my example:

source CVS -> field collection field in drupal

Author 1: First name -> Author: First name
Author 1: Last name -> Author: Last name

then I map the next author to the same fields hoping with would create anothe entry:

Author 2: First name -> Author: First name
Author 2: Last name -> Author: Last name

unfortunatly only one value gets imported, the Author 1.
Any suggestion on how to import multiple value fields?

priyanka12chavan’s picture

Recently we faced similar issue where we need to add unlimited specifications for a each product. We created field collection with fields Category, Label, Value1, Value2 with unlimited fields. We came across Field collection feeds module. It's docs https://www.drupal.org/node/1831004 provides proper steps to setup importer.

imperator_99’s picture

I've successfully applied the patch in #143 to no avail - my field collection fields don't show up as targets in feeds. I'm using field_collection 7.x-1.0-beta7 and feeds 7.x-2.0-alpha8. There's no errors in the logs, not really sure what to do from here...any advice appreciated!

hauruck’s picture

I found a solution to the multi value problems I was having. Use the tamper module and follow this guide:

http://thejibe.com/blog/12/09/feeds-importing-multiple-values-one-multi-...

bbdata’s picture

Version: 7.x-1.x-dev » 7.x-1.0-beta7

For anyone using 7.x-1.0-beta7 and doesn't need the functionality from #127/#130/#143. Use the patch from #121.

Otherwise you will lose the ability to configure the target of taxonomy term references.

bbdata’s picture

Version: 7.x-1.0-beta7 » 7.x-1.x-dev

Sorry, changing version back.

bbdata’s picture

FileSize
6.29 KB
PASSED: [[SimpleTest]]: [MySQL] 164 pass(es). View
1.69 KB

I would like to also add that there is an unexpected behavior found with a node reference field in a field collection item.

When the feeds importer is set to update a node with a field collection field that contains a node reference field, the node reference field is not updated. This is because the references_feeds_set_target() of the references module would append the source value to the existing array of values. This ends up getting ignored when the node reference field only allows one single value.

I am not 100% sure if this is a bug in the references module or in patch #121. A test was conducted by mapping directly to a node reference field that was part of the node, and it correctly updated with the new value. However, this will not happen when the reference field is inside a field collection item.

The following patch to #121 is rather specific and corrects this by emptying the reference field if it only supports one value before it is passed to references_feeds_set_target(). What happens now is that the source value gets correctly updated.

Perhaps someone with a better understanding of both modules can chime in where this fix will better fit.

GaëlG’s picture

FileSize
9.57 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch field_collection-feeds-1063434-153.patch. Unable to apply patch. See the log in the details link for more information. View

I could not have it work to delete field collections removed from the source. Besides, according to #1107522: Framework for expected behavior when importing empty/blank values + text field fix, the new default behavior is to erase field values for blank fields in source. So I made a new patch to handle this all.

It includes #127.

I removed the lines about $subfield because these variables are never used.

#130 is not included because it added

    if (empty($v)) {
      // Avoid creation of empty field collections
      continue;
    }

without explaining why and not even notifying it was added. It looks wrong to me because a field inside a field collection could be emptied in the source and this should empty the fied in Drupal on update. Avoiding empty collections creation is a task handled by field_collection_feeds_presave().

I did not keep #140 either because I don't see why this should be handled by the Field Collection implementation of feeds_set_target(). If there is a bug because $v is a string, the bug is either caused earlier (the function should be called with $value as an array of arrays) or more probably later in the subtarget mapping callback (the callback should be able to handle string values, it's something that text_feeds_set_target() does for example). I might be wrong on this but at least it is an argued point of view.

jaysonjaynes’s picture

@GaëlG, that one worked for me! I was having the issue of importing in a "0" for a decimal value on an exploded field, and it was ignoring it and shifting the next value into that position. Thank you!

Status: Needs review » Needs work

The last submitted patch, 153: field_collection-feeds-1063434-153.patch, failed testing.

Novitsh’s picture

Applied patch from #153 and added the field collection field in my Feeds importer.
Getting this error back:

An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /batch?id=167&op=do StatusText: OK ResponseText: Fatal error: Class name must be a valid object or a string in /drupal/sites/all/modules/contrib/entity_translation/includes/translation.handler_factory.inc on line 93

EDIT: disabling feeds_et did the trick there.
I've also figured out that for me it's working by only making the FC host translatable, not the child fields, and patching this line after #152:

-  $language = ($info['translatable'] && isset($entity->language)) ? $entity->language : LANGUAGE_NONE;
+  $language = ($info['translatable'] && isset($mapping['language'])) ? $mapping['language'] : LANGUAGE_NONE;
SocialNicheGuru’s picture

I could not get 153 to work.

mkhamash’s picture

Patch #153 did work in my case but I think it needs more work to cover more use cases.

troybthompson’s picture

I applied #153 to Field collection 7.x-1.0-beta8+11-dev (2015-Mar-26) and I'm using Feeds 7.x-2.0-beta1

Only some fields in the collections I have showed as available to be mapped. In my case, Entity Reference fields showed, but Text, Decimal, and Date did not. Thoughts?

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 153: field_collection-feeds-1063434-153.patch, failed testing.

firoz2456’s picture

@troybthompson Can you elaborate more, what type of fields you are trying to map?

troybthompson’s picture

I don't know how much more details I have. Without the patch, none of the fields within my field collections show up for mapping.

I have multiple field collections in the content node type I'm trying to import to.

After the patch, only entity reference fields in my field collections show up to map. Within the same field collection that the entity reference field shows, there is also a text field that's not showing up to map. There are two of those field collections that are acting the same way. I also have a field collection that only contains a date field that is not showing up.

All fields were showing up with an older version of feeds.

Does that help? If there's any other details that would help, please let me know. I'm stuck with some feeds importers needing the new version of feeds and some needing the old method.

ScorpionGhost’s picture

troybthompson, patch does not work with Field collection 7.x-1.0-beta8 and Feeds 7.x-2.0-beta1. Help!

troybthompson’s picture

Has anyone been able to map to core fields inside field collections with the latest versions of things?

veronicaSeveryn’s picture

FileSize
4.32 KB
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View

I had a need to implement field_collection field through Feeds, and also discovered that the existing patch isn't longer working with the most recent code.. So I tried to re-work it a little. Attaching a patch (I am currently using it for my needs on one of the sites and it works for me). Not sure if it will need any more adjustments for more complex fields inside field_collection structure, but looks like it does the job at this moment.

The patch is based off of 7.x-1.0-beta8+11-dev field_collection code (March 26th, 2015 1:43:07 PM)

veronicaSeveryn’s picture

Status: Needs work » Needs review

The last submitted patch, 16: field_collection-feeds_integration.patch, failed testing.

The last submitted patch, 28: field_collection-feeds_integration.patch, failed testing.

veronicaSeveryn’s picture

FileSize
4.78 KB
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View

Uploading revised patch from comment #168 - I think I found a way to save complex fields within Field Collection structure. Tested with saving dates, text, numbers, lists.

Zatox’s picture

Working for text fields with Feeds 7.x-2.0-beta1 and Field Collection 7.x-1.0-beta8
Thanks for updating the patch @veronicaSeveryn !
I think we'd need more testing to update the issue to RTBC. Especially on the complex fields... @troybthompson maybe?

peleo’s picture

Issue summary: View changes

Sorry but it did not work for me with Feeds 7.x-2.0-beta1. Doesn´t let me mapping field collections...

troybthompson’s picture

Thanks! This is working for me in all of my cases so far.

Feeds 7.x-2.0-beta1+6-dev (2015-Aug-05)
Field collection 7.x-1.0-beta8+11-dev (2015-Mar-26)

Now they just have to get Address Field feeds support working again and I'll be back in business. =^)

Novitsh’s picture

FileSize
4.59 KB
PASSED: [[SimpleTest]]: [MySQL] 166 pass(es). View

Many thanks first to veronicaSeveryn. I've updated the patch from #173 a bit to comply with coding standards. Also removed some unused variables and declarations that are directly overriden.
More testing on this patch is still required.

SocialNicheGuru’s picture

will this let me do the following

my field_collection might have fields within it called field_my_name and field_my_birthday

the import data has a field name and birthday

will the patch allow me to map name to field_my_name within the field collection and birthday to field_my_birthday within the field collection?

veronicaSeveryn’s picture

I had a Field Collection with the following field types inside: date, textfield, integer, decimal, addressfield, textarea, and I think URL. I was able to import them all successfully with the patch I had provided.

For the Feeds I am utilizing 7.x-2.0-dev version. And like I said, Field Collection patch was based off 7.x-1.x-dev branch but I was utilizing 7.x-1.0-beta8 in my site and it applied and worked with it without a problem for the fields I had to import.

UPDATING: I apologize, I typed the wrong version number previously: it might not work with FC 1.0-Beta1... So your best bet would be at least BETA 8, which worked for sure.

mark.bennett’s picture

I am currently using #177 successfully in order to recognize the first level of field collections in a content type. However, I have field collections inside of field collections and anything under that first level of field collections is not available to map.

For instance, I have a Profile content type. I have a variety of fields in Profiles (email, text, date, link, integer, etc) as well as numerous field collections; Address, Course, Presentation, etc. Under the Presentation field collection though, I have a few standard fields as well as its own "sub" field collection called Review. As a step farther, under the Review field collection, there is another field collection called Reviewer. So the hierarchy looks like this:

Profile
--> Presentation
------> Review
----------> Reviewer

With #177, I was finally able to recognize and map all field collections at the Presentation-level. However, I still cannot recognize any field collections or fields under that level (nothing in Review or below). Would it be possible to tweak the patch to recurse through this hierarchy?

pcambra’s picture

I've found an issue when importing a file field inside a field collection, the mapper for file.inc does this:

$v->setAllowedExtensions($instance_info['settings']['file_extensions']);

Which checks the field collection settings and not the file inside the field collection, wondering if the best way around is to provide a custom mapper or to add a set targets alter.

pcambra’s picture

In case someone else finds #181, I've added a patch in feeds #2619788: Get instance info from the entity instead of from the source for file mappings, not sure if that's the way to go, but also this gist that handles file fields inside field collections https://gist.github.com/pcambra/9de9997ba8780e1f2a2e

pavlosdan’s picture

I had a similar problem as Mark in #180 and tried to play around with the patch a bit.

After commenting out the return statement at:

  // Thou shalt not attempt Sorcery!
  if ($entity_type == 'field_collection_item') {
    //return;
  }

The items in the nested field collections started showing up and some test imports showed it worked fine with some simple textfields (against latest Dev versions of Feeds and Field Collection modules).

I am not sure though if this introduces more issues.

Was the taxonomy issue at #121 resolved? I can't get the feed to create taxonomy terms even when I check the auto-create checkbox. I am not sure if this is because the taxonomy term field is within one of the nested field collections. If that's the issue, is there any way to make this work? I've been trying to make sense of the code to see why it wouldn't work but so far no luck.

Any help is appreciated.

Thanks for all the work put into this already!

EDIT: the imported fields create one item each for the top level field collection, not multivalued fields of the nested collection items. More research and testing needed.

Novitsh’s picture

Changed the patch based on the feedback of #183.
I'm wondering why that line was introduced in the first place?

With the patch taxonomy should be able to be created. Could it be a permission issue in your case @pavlosdan? Feeds can create items as anonymous or as a user. In that last case you need to make sure it has the correct permission to create taxonomy.

iajay’s picture

I found one bug in the above patch(s)
When the feed is updated and the rows are deleted from the xml feed then the field collection values are not deleted and causes duplicates
I think I can explain this better with an example
I have a field collection - status_action - with fields action_date and action_text
This is the structure of the xml

<status_actions>
			<status_action>
				<action_date>2014-12-01</action_date>
				<action_text>Proposed Rules</action_text>
			</status_action>
			<status_action>
				<action_date>2015-02-11</action_date>
				<action_text>Final Regulations</action_text>
			</status_action>
		</status_actions>

When the xml is imported first it is working correctly and I get
2014-12-01 - Proposed Rules
2015-02-11 - Final Regulations

Now suppose the xml is updated to

<status_actions>
			<status_action>
				<action_date>2015-02-11</action_date>
				<action_text>Final Regulations</action_text>
			</status_action>
		</status_actions>

So basically a value is deleted and the feed is again imported via cron
the new values that I get are

2015-02-11 - Final Regulations
2015-02-11 - Final Regulations

I hope I am able to explain the problem correctly
Is there a way around this?
Can we simply delete the existing values and then import the updated xml or can you guys think of some better option?
Awaiting reply
Regards
Ajjaykummar

nico.knaepen’s picture

Status: Needs review » Needs work
monsterplant’s picture

I'm quite confused as to how this patch is meant to work. I have a csv with the following setup:

node title | ID (field collection field 1) | Quantity (field collection field 2) |
test | 1 | 1 |
test | 2 | 2 |

It imports fine for the first row, but for the second one it does not create a new field collection set in the same node. Can someone please enlighten me on how it is meant to be setup?

iajay’s picture

I created a patch, now the code deletes the field_collection before saving.. This is my first patch.. so excuse me if I made some mistake
The patch is based on #184 above
regards

dpfitzsi’s picture

Status: Needs work » Needs review

Switching to Needs Review so that Simpletest checks latest patch.

iajay’s picture

One more problem which I can explain better with an example
consider this xml

<status_action>
		<action_date>2015-01-30</action_date>
		<action_text>Action 1</action_text>
		</status_action>
	    <status_action>
		<action_date>2015-03-19</action_date>
		<action_text>Action 2</action_text>
		<effective_date>2015-04-23</effective_date>
	    </status_action>
	    <status_action>
		<action_date>2015-03-19</action_date>
		<action_text>Action 3</action_text>
		<effective_date>2015-05-23</effective_date>
	    </status_action>

Considering the above the effective date value is assigned to the wrong position After important it looks like
Action Date:2015-03-19
Action Date: Action 1
Effective Date : 2015-04-23

Action Date:2015-03-19
Action Date: Action 2
Effective Date : 2015-05-23

Action Date:2015-03-19
Action Date: Action 3
Effective Date :

So is there a solution for this?
Regards

iajay’s picture

I will try to simplify this further

<status_action>
  <effective_date>2015-03-23</effective_date>
</status_action>
<status_action>
  <action_text>action 2</action_text>
  <effective_date>2015-04-23</effective_date>
</status_action>
<status_action>
 <action_text>action 3</action_text>
 <effective_date>2015-05-23</effective_date>
</status_action>

In this case the action 2 gets assigned to first record and the action_text is not present in the first record
this is how it shows
Action text: action 1
effective date:2015-03-23

Action text: action 2
effective date:2015-03-23

Action text:
effective date:2015-05-23

Now even if a blank tag is added like then also it gives the same result

djcj’s picture

Is there not any batch processing?

This does not work for large data sets (13,000 rows) (16MB xml file). It will error out with SQL timeout. Increasing/removing the timeout isn't a good approach, because a user will not wait that long to import.

Fatal error: Maximum execution time of 30 seconds exceeded in /home/djcjne5/public_html/music/drupal/includes/database/query.inc on line 1020

djcj’s picture

Working Alternative: Field Collection Feeds
http://drupal.org/project/field_collection_feeds

I ended up using this module instead, and it can handle BIG data (13,000+ collection items, 3 fields per collection item). Used XML as an input file and XPATH to grab the fields.

Versions I used:
Field collection feeds - 7.x-1.0-alpha3
Field collection - 7.x-1.0-beta11

EDIT:
Import is not actually working into field collections still. It takes too long and times out when running in front end and background job modes. The XML file is 400,000 lines. Will update this post if I make progress.

dpfitzsi’s picture

Testing out the last patch against a field collection that has multivalue taxonomy checklists inside them appears to fail. Only the first term gets added. This also doesn't allow you explode terms using Feed Tamper (only one will get added).

It appears that in the previous few patches the sub targets carnality check was removed and this appears to be causing the problem. I have added a check back in and set it to use the single value version in the case that there is only one value (like a simple textfield) and to use a multivalue array when there are multiple terms (in the case of taxonomy).

hi.chawla’s picture

Version: 7.x-1.x-dev » 7.x-1.0-beta11
Issue tags: +Multiple Values are not getting saved

dpfitzsi patch is also not working for multiple entries, only first field is getting saved.
Our csv file contains following data

Node Id (Unique)|Field 1|Field 2
876|Ted;Mark|Red;Yellow

only Ted and Red and getting saved for node 876

NWOM’s picture

Version: 7.x-1.0-beta11 » 7.x-1.x-dev
Issue tags: -Multiple Values are not getting saved

Changing back to dev version, since patches are always applied against the dev. Also, removing the tag according to Adding tags to issues and using tags.

arborems’s picture

I also had problems with dpfitzsi patch #196, but was able to get it to work after changing the line "$sub_field_info = field_info_field($sub_target[0]);" to "$sub_field_info = field_info_field($args[0]);" I suspect this is what dpfitzsi originally intended. $sub_target is a string so $sub_target[0] just gives the first letter of the field name.

In my case $sub_target was either 'field_datetime:start' or 'field_datetime:end', so going back to the $args array was necessary to get only the first part, 'field_datetime'.

dpfitzsi’s picture

Issue summary: View changes

Thank you everyone for the feedback on my version of the patch. I will make an update this week and roll a new version.

djdevin’s picture

Using the patches above I can import to field collections however there is one case that is not working.

When importing to a boolean checkbox field (off/on checkbox) you cannot import to the opposite of the default. That is, if the checkbox is not checked, you cannot import it to checked, and vice versa.

I think what's happening is that the field collection is being created, and the default is being saved. Then when the new value is imported, instead of the value being overridden with what is being imported, it is being appended. The field ends up having a value of 0 and 1.

I made this change to fix the issue, it unsets the field before trying to set values on it. The callback is "list_feeds_set_boolean_target" and if you look at that function, it appends to whatever is there. All other fields seemed to still work.

@@ -100,6 +100,7 @@ function field_collection_feeds_set_target($source, $entity, $target, $value, $m
 
       if (isset($_sub_targets[$sub_target]['callback']) && function_exists($_sub_targets[$sub_target]['callback'])) {
         $callback = $_sub_targets[$sub_target]['callback'];
+        unset($field_collection_item->{$sub_target});
         $callback($source, $field_collection_item, $sub_target, array($v), $sub_mapping);
       }
edvanleeuwen’s picture

I am not able to get the patch working using the latest dev (2016-Apr-29). I noticed that in hook_feeds_processor_targets_alter() the info type returned is entityreference, not field_collection. Am I missing something?

djcj’s picture

#201 solved my issue: check boxes were not updating on import

The import still throws errors, but at least it appears to work.

Notice: Undefined index: value in field_collection_feeds_set_target() (line 1582 of /var/www/html/sites/all/modules/field_collection/field_collection.module).
Warning: array_flip(): Can only flip STRING and INTEGER values! in EntityAPIController->load() (line 219 of /var/www/html/sites/all/modules/entity/includes/entity.controller.inc).
SocialNicheGuru’s picture

patch doesn't apply well for me. It stops at

 } catch (Exception $e) {
    drupal_set_message($e->getMessage(), 'error');
$patch -p1 < add_feeds_integration-1063434-196.patch
(Stripping trailing CRs from patch; use --binary to disable.)
patching file field_collection.module
Hunk #1 succeeded at 1874 with fuzz 1 (offset 369 lines).
$ tail field_collection.module

      unset($field_collection_item);

      if ($info['cardinality'] == 1) {
        break;
      }
      $delta++;
    }
  } catch (Exception $e) {
    drupal_set_message($e->getMessage(), 'error');

so I get a WSOD:
PHP Parse error: syntax error, unexpected end of file in field_collection/field_collection.module on line 2004

NWOM’s picture

Version: 7.x-1.x-dev » 7.x-1.0-beta11

That was most likely my fault. I changed the branch to dev, assuming that a patch would be created for it. Since usually only patches that are based on the dev version will get committed. This most likely only applies correctly on 7.x-1.0-beta11. I'll set it back to 7.x-1.0-beta11, just to be safe.

Could anyone else confirm whether this works on the dev release? I'm not currently in the office to double-check the version that I applied this on.

NWOM’s picture

Version: 7.x-1.0-beta11 » 7.x-1.x-dev

Please disregard the last post. It appears that the patch was indeed created for the dev release. Switching back to dev. It may need a re-roll.

sherakama’s picture

[edit]
Aaahhh,
I totally started from the wrong patch. I will update accordingly.
sorry for the spam.
[edit]

sherakama’s picture

sherakama’s picture

With the patch in #196 I was only getting one field collection.

I have made a few more adjustments based on that work:

1. Use &drupal_static convention instead of raw static variables
2. Move the feed mappers loading out of a loop to ensure only loads once, even with static var. Minor performance improvement.
3. Remove the field_collection save in each field loop. Even though it was not saving the parent node it was still causing an enormous amount of database writes and had huge performance implications. Just save the field collections along with the parent entity for a write only once boost.
4. Only stop looping at the correct cardinality of a field. It is possible that a field could be set to 2, 3, 10, etc
5. Check for multiple values on callback save incase feeds_tamper or something explodes and provides an array.

NWOM’s picture

#209 works perfectly for me. Thank you!

elpino’s picture

Status: Needs review » Needs work

Patch in #209 doesn't work for mapped fields on field collection if they are multivalue.

There's an error in delta handling and cardinality.

If source contains multiple values (and these are exploded into an array via Feeds Tamper), the code cycles through them but instead of assigning multiple values for a single field, the code creates new field collections for each value and stops at the field collection's cardinality instead of the individual field's cardinality. So on save, only the last value is saved.

sherakama’s picture

FileSize
5.83 KB

elpino was right. The patch in 209 did not account nor handle multiple value fields.

New patch attached.

sherakama’s picture

Nope, that last patch did not work either.

sherakama’s picture

Ok,

New patch with (hopefully) the correct handling of multiple value field collection items.

moonray’s picture

The importer seems to work well for me, aside from a single notice:

Notice: Undefined offset: 1 in field_collection_feeds_set_target() (line 1952 of /sites/all/modules/contrib/field_collection/field_collection.module).

This was tested on a multi value field in a field collection.

lamp5’s picture

In my case not work:
Fatal error: Call to undefined function stanford_feeds_helper_field_collection_file_feeds_set_target() in sites/all/modules/field_collection/field_collection.module on line 1987
I have got multi value field collection with textfield, textarea and imagefield

moonray’s picture

During a debug session for the following issue, I ran across this particular line, which looks very much out of place:

<?php
        // HiJack the file callback so we can make it work.
        if ($callback == "file_feeds_set_target") {
          $callback = "stanford_feeds_helper_field_collection_file_feeds_set_target";
        }
?>

The problem I'm trying to debug is that when you have multiple sources mapped to the same target, it overwrites the field values rather than setting it as an additional value for the multi-value field.

sherakama’s picture

FileSize
8.08 KB

@moonray,

You are 100% correct. There was an issue with file fields and I totally forgot to translate that function to the patch. I've wrapped all of the patch functionality in to a tidy little module until it gets added to the project here: https://github.com/SU-SWS/stanford_feeds_helper.

Updated patch attached to include the correct function and name.

Thanks for the testing!

moonray’s picture

Why didn't you instead of this

+      $sub_mapping = array();
+      $config = $source->importer()->getConfig();
+      if (!empty($config['processor']['config']['mappings'])) {
+        foreach ($config['processor']['config']['mappings'] as $mapping) {
+          if ($mapping['target'] == $target . ':' . $sub_target) {
+            $sub_mapping = $mapping;
+            $sub_mapping['target'] = $sub_target;
+            // Needs language or feeds mappers shout php notices.
+            $sub_mapping['language'] = $config['processor']['config']['language'];
+            break;
+          }
+        }
+      }

Do this?

+      $sub_mapping = $main_mapping;
+      $sub_mapping['target'] = $sub_target;

Also, I noticed a stray dpm() in your patch.

Additionally, you might want to change
while ($value[$delta]) {
to
while (isset($value[$delta])) {
to prevent a notice mentioned in above comments.

guptahemant’s picture

hi @sherakama
while using patch #218 i am getting the following notice while trying to import the image files and hence the image is not importing.
The file top_banner_2.jpg has an invalid extension.
I have already checked the field for allowed extension and it allows for the uploading of .jpg files
please help

hyside’s picture

@guptahemant

It may be unrelated to your problem, but is the extension of the image files you are importing in uppercase? If so, it might be worth checking out this issue I ran into some time ago...

https://www.drupal.org/node/2650600

guptahemant’s picture

Hi @hyside
Thanks for your reply but that wasn't the case.

dnmurray’s picture

Patch 196 worked for me. I had the same problem as in #204 when using the current -dev; I fixed it by hand. (I'm not sure what I'm going to do to fix my drush make, though.) The 218 patch re-introduced the issue if trying to import lists using feeds_tamper. Only the first value is getting saved.

Jigar.addweb’s picture

Issue tags: +Field collection
yfreeman’s picture

Can the patch be updated as follows, as $value[$delta] will fail if == ""

diff --git a/field_collection.module b/field_collection.module
index 578704d..d7df3cd 100644
--- a/field_collection.module
+++ b/field_collection.module
@@ -1991,7 +1991,7 @@ function field_collection_feeds_set_target($source, $entity, $target, $value, $m
   $field = isset($entity->$target) ? $entity->$target : array();
   try {
 
-    while ($value[$delta]) {
+    while (isset($value[$delta])) {
 
       // Zero out.
       $field_collection_item = null;
moonray’s picture

That should be + while (!empty($value[$delta])) { if you want to keep the equation the same.

elpino’s picture

Patch #218 worked great for me, amended with moonray's comment on #266.

ScorpionGhost’s picture

It does not update existing records.

My modules:
Feeds 7.x-2.0-beta2
Field collection 7.x-1.0-beta11
Commerce Feeds 7.x-1.4

Drupal 7.50

Patch field_collection_feeds-1063434-218.patch

It does not update the value of field collection. Any other well-field updates

Help me please. What should be changed?

Feed informs Updated Content 1. But the value of field Collection - old

ScorpionGhost’s picture

Please include the version FEEDS, FIELD COLLECTION and the date of issue. On which the patch has been tested. I'm sure - it will be convenient for everyone to find bugs. I'm not running the update has added value fields field collection perhaps neither one version of the module. Although I tested on a beta 11 and DIV. Picture image added, too, only the first. Patch field_collection_feeds-1063434-218.patch

SocialNicheGuru’s picture

the patch includes a rogue dpm call https://www.drupal.org/node/1063434#comment-11476479

ScorpionGhost’s picture

New update field_collection 7.x-1.0-beta12

junaidpv’s picture

I came across this thread while trying to upgrade my field_collection to latest version.

Here is the re-rolled patch from comment#218 and updated as per comment#226.