Support for Feeds

Thanks, @pikku-h

I don't have much experience with Feeds, but I've created a patch with your work for the benefit of others and to get the process started.

We'll want to add some comments/documentation should this ever be committed.

Gus

Works like a charm, added the patch and was able to import all Youtube videos. Please commit to the next release of Youtube. Nice job.

I finally got around to testing the patch myself. This is definitely useful! Great work, pikku-h.

I only ran into one problem. When I tested a feed of Apple's YouTube channel, http://gdata.youtube.com/feeds/base/users/Apple/uploads?alt=rss&v=2&orde..., the videos had a parameter at the end, such as '&feature=youtube_gdata'. Feeds was attempting to save this as a part of the video_id, but was throwing an error because it is so long (and it doesn't belong, of course). I altered a bit of code in youtube_set_target to ensure this doesn't happen and was having success after doing so.

I also added comments to the two functions, a majority of which came straight from hook_feeds_processor_targets_alter() documentation.

Leaving this as 'needs review' so someone can test my minor change. After that I don't see why we can't get this committed.

Thanks again,
Gus

There seems to be some issue importing empty values. If I map a value to the youtube field and run import, I will get an empty Youtube video visible on each item. The weird thing is that if I edit the item, there is nothing saved on the youtube field, because my source files don't have the ID values, only the empty elements.

I don't have time to test it now, but I think the $value has to be checked if it's empty or not.

@mErilainen,

Have you found any more time to test this and perhaps create a patch?

If not, could you provide clearer instructions on how to reproduce this problem? What is the case when you are getting empty values?

Thanks,
Gus

No, I haven't needed this feature yet, but I might have to do something about the issue soon.

I'm getting these empty video every time when I have an empty xml element () where I'd like to get the Youtube ID from. Furthermore I need to do rewriting only if the value exists, so I can't use a simple Feeds Tamper rewrite plugin because then I would get the empty prefix (http://youtu.be/ or http://www.youtube.com/watch?v=) to every video.

This is a typical problem with Feeds when I want to skip only the empty value, I don't want to skip the whole feeds item.

It worked for me perfectly.

I applied patch #3 only (do not try to apply both #1 and #3 they don't apply).
I have a video content type where the video URL is required field so i did not face any problems like "mErilainen" did.

For feed parser i used this module: Feeds Youtube.

@mErilainen

In the attached patch I've wrapped everything in youtube_set_target() in a check for !empty($value), as suggested in #4.

Could you test it to see if it solves the problem you've been experiencing? Or have you come up with something else that does?

This seems to have passed testing in all other situations thus far.

Thanks,
Gus

There is some issue with the field language for me. It seems that you have gone further to solve the issue which Feeds has when importing multilingual values. I don't have any language assigned for that field, or any fields, which I'm importing, but in your code the entity language is used when storing the Youtube ID:

<?php
$entity->{$target}[$entity->language][0] = array(
      'input' => $value,
      'video_id' => $video_id
    );
?>

In my case all the other fields are stored under 'und' but the Youtube ID is under 'fi' which is the default language for the content. Probably you just have to stick to 'und' like Feeds does it, or find a way to see if the field has language set.

Here is a related issue #1183440: Multilingual Feeds - Make field import language-aware.

EDIT:
This works in my case:

<?php
$entity->{$target}['und'][0] = array(
      'input' => $value,
      'video_id' => $video_id
    );
?>

Another issue with the patch is that it assumes that the Youtube share URL is in the longer format. The short format (eg. http://youtu.be/_iSzgi2VhDw) doesn't have "=" so the $video_id is empty after import. Then I get empty black player, but saving the node seems to fix it.

@mErilainen

Thank you for testing this so thoroughly!

As for #9: You are right, we shouldn't be assigning the field's value using the entity's language. I swapped $entity->language for LANGUAGE_NONE.

And for #10: I realized that this was the third place that the field's input was being checked/validated, each time happening with different code. It didn't make sense to create new techniques for these patches to grab the video_id when another technique was already being used in the module. I went ahead and separated out the existing technique into youtube_get_video_id(). Previous validation and this new feeds functionality now all use that function to grab the video_id, meaning this patch will now accept the youtu.be format when importing a feed.

Please apply this patch to the latest 7.x-1.x branch code. I'd appreciate it if you could retest all issues you've surfaced so far: empty values, different languages, and various URL formats.

Thanks again,
Gus

I've re-rolled this patch so it applies to the 7.x-1.0 release. Please use this when testing.

It should work in all standard situations and in the special cases discussed above.

Hello,

I have applied your patch successfully and now I can see the Youtube fields in Feeds mapping. So I mapped them and run my import CSV file with the YouTube URL in the cell (it's in UTF-8). All the data is imported okay, except the youtube fields. The fields are not hidden...

In the mapping I'm using my header's name Book_trailer (not the field_book_trailer from the field).

I tried doing it only with the GUID and the Youtube field, but it didn't work. I'm puzzled.

Oh, if I edit the node manually after and paste the URL it displays normally. It's like Feeds is not adding the URL from my CSV in the database, but it adds all the rest of the data. Not sure if it is relevant, but not all the nodes have book trailer, but it should stop the import of the cells with data right?

Ok, I have managed to make one video work for each node, but not 2 even though I have two fields one for each URL and the data show in both it will not display on the node. (no it's not hidden)

Thanks for the help,
Mario

Mario,

It sounds like each item on your feed has multiple video URLs. So when you are setting up your mapping, you need more flexible options under 'Source' to map them to the appropriate fields under 'Target'. I believe this will need to be done with the help of another module that extends Feeds, allowing you to specify exactly what parts of your feed should be mapped to these fields.

A quick look found Feeds XPath Parser and Feeds QueryPath Parser. An example of one that parses feeds from youtube.com is Feeds: YouTube parser.

Hope this helps,
Gus

This patch was a lifesaver for importing youtube videos.

It seems the kinks have been worked out and the patch is being used successfully.

I've committed Feeds support to the 7.x-1.x branch.

Thanks to pikku-h, mErilainen, and everyone else involved.

I've change the function youtube_set_target because my source sends an array of youtube video for my multivalue field and the target wasn't adapted.

I'm trying to import videos via a CSV in feeds, and all of my other field mappings work except for the youtube field. I was optimistic when I saw feeds support had been added some time ago, but unfortunately all I get when attempting to import the video url is a blank field.

I am using the stable release version of this module (7.x-1.3) and the dev version of Feeds (7.x-2.x-dev).

Can someone else please attempt to do a basic csv node import using a youtube field and see if they are successful?

I troubleshot this issue for a long time, only to find out (not surprisingly) that the solution was very simple. Basically, the $value pulled in by youtube_set_target was an array, and in order for youtube_get_video_id to process the video url properly, you have to pass $value[0] to it, not just $value.

I've attached a patch for this. If it gets committed to the project, it'll be my first official patch contributed on Drupal.org!

I can confirm that the feeds import support once worked, but it somehow doesn't seem to cooperate anymore since using the latest developer version of feeds (in my case: 7.x-2.0-alpha8+56-dev).

@JakeWilung / #21: Your patch almost works, as it replaces
$video_id = youtube_get_video_id($value);
with
$video_id = youtube_get_video_id($value[0]);

However, a few lines further down,
'input' => $value,
should also be replaced by
'input' => $value[0],
as the import would otherwise fail to import the array as a value for the edit field, while at the same time taking the (hopefully) correctly identified video ID.

Long story short: There's a better way to approach this, as the original, rudimentary implementation is flawed.

It should take care of the fact that the $value could be an array. It should process all entries, and then feed them into a multiple value field (cardinality > 1) – or stop after the first (in case cardinality == 1).

I created this via copy & paste from (a) the Email field module (which gave me the basic technique) and (b) pikku-h's original implementation. It looks like this:

function youtube_set_target($source, $entity, $target, $value, $mapping) {
  $value = is_array($value) ? $value : array($value);

  $info = field_info_field($target);

  $i = 0;
  $field = isset($entity->$target) ? $entity->$target : array();
  foreach ($value as $v) {
    $video_id = youtube_get_video_id($v);
    if ($video_id) {
      $field[LANGUAGE_NONE][$i] = array(
        'input' => $v,
        'video_id' => $video_id,
      );
    }
    if ($info['cardinality'] == 1) {
      break;
    }
    $i++;
  }

  $entity->{$target} = $field;
}

... Works for me. I've attached a patch.

Umm ... I think Calystod's patch in #18 looks even better than mine.

#18 works for one field (didn't try for two or more).

It used to work for one field, yes. But I think starting with the latest dev version of feeds, even just one value will come as an array consisting of one element.

Thanks to everyone who tested this and supplied a patch! I've committed Calystod's patch from #18 to the 7.x-1.x branch. It will be available in the next development release and whenever a 7.x-1.4 release is created.

I then made a minor authorship goof when committing it (still getting used to drupal.org generating commit messages). I accidentally gave the actual commit's authorship to GerZah and tagged it with an incorrect issue number. I then fixed the issue number, but hadn't realized the author was wrong. So then I fixed the author. Sorry about that! I need to step away from the computer. :)

Thanks again!

Thank you for your hard work! – So I now have a commit that isn't actually mine? :-O ;-)

GerZah, Yup. A holiday bonus. :) Thanks for your help!