xpath settings to extract URL from data source
field mappings to convert remote URL to local file attachment
animated demo of the fetcher in action

Experimental project

This is a sandbox project, which contains experimental code for developer use only.

  • Given a feed that contains an URL or filepath - fetch that file into the CMS and register it as a file attachment field
  • Given a feed that contains an URL or filepath - fetch that files text contents and save the content in a textfield

This is about processing the content data of a feed and resolving a reference (an URL) into content, not about fetching the feed source itself. Hence it's primarily a tamper process, and only a fetcher after that.

Feeds_tamper module is not a strict requirement unless you need to do rewriting on the fly. Out of the box, this module allows you to map to file fields, and will quietly download the file in the background without any further configuration.

Import a normal article RSS feed that supplies its images as enclosures

<link rel="enclosure" type="image/jpeg" href="https://farm6.staticflickr.com/5559/30102441272_3cf8a1f6de_b.jpg" />

and get the image to my local site, attached as a normal file, and rendered in my local teasers using image styles etc.
This happens immediately as soon as you start mapping an URL into a file field 'url' using the Feeds processor mapper.

Animated demo of feeds file fetcher

OR

Import a media rss style feed :

<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/" xmlns:atom="http://www.w3.org/2005/Atom">
...
<item>
<title>RE.8 Rear cockpit diagram</title>
<link>http://thevintageaviator.co.nz/node/2800</link>
<media:content url="http://thevintageaviator.co.nz/files/images/re-8-gallery/screensize/re8-202-re-8-rear-cockpit-diagram.screensize.jpg"/>
</item>

And use Feeds to import each of create a https://drupal.org/project/media_gallery">Media Gallery local copy of all that data.

Usage - simple

If you can extract the URL of an image or file directly during the 'parser' and 'mapping' phases of building a feed (xpath parser is perfect here) - you don't even need to add a separate 'tamper' process, it'll be done for you automatically.

An example of this is provided as a demo you can enable.
The feeds_tamper_file_fetcher_demo feed provided in this project can be enabled - it can be pointed at Flickr, and will fetch each of the images in a Flickr steam as a local node with file attachment.

Media entities - One use-case

To do this in a more abstract way, we may want to import each file as a media entity. This can use

  • Feeds XPath Parser, Feeds mediaRSS (or equivalent)
  • feeds entity field mapping (Feeds-2.x +)
  • feeds tampering to download and register the remote file (This module - different from feeds file fetcher which acts on text content)
  • data mapping to re-attach each new media/file/image item to a media_gallery (under investigation)

See also

Other semi-related modules for comparison & contrast.

  • feeds_mediarss
  • feeds file entity processor Unrelated - This allowed importing into a file entity as the target entity. Now in Feeds core, so deprecated.
  • Feeds Field Fetcher Unrelated - that's just about setting a feed data source.
  • Feeds URL Fetcher Unknown (missing documentation) looks like it also is just about fetching the initial datasource, not the field contents.
  • Project information

    • caution Minimally maintained
      Maintainers monitor issues, but fast responses are not guaranteed.
    • Module categories: Import and Export
    • Created by dman on , updated