Creating a custom Feeds workflow

Overview

The Feeds process is broken up into a few parts:

• Fetcher
• Parser
• Item
• Processor(TODO)
• Add hooks for imported items

Fetcher

Overview

During the "Fetcher" stage of the Feeds process, the goal is to grab the data for the parser stage. This is where you'd want to make your API call, grab a queue item, or wherever the data is and pass it in as "light" of a format you can to the parser.

This file should be located at:

modules/custom/your_custom_module/src/Feeds/Fetcher/YourFetcher.php

Code Example

<?php

namespace Drupal\your_custom_module\Feeds\Fetcher;

use Drupal\Core\Config\ImmutableConfig;
use Drupal\feeds\FeedInterface;
use Drupal\feeds\Plugin\Type\Fetcher\FetcherInterface;
use Drupal\feeds\Plugin\Type\PluginBase;
use Drupal\feeds\Result\RawFetcherResult;
use Drupal\feeds\StateInterface;

/**
 * Your Class Description.
 *
 * @FeedsFetcher(
 *   id = "your_custom_id",
 *   title = @Translation("Your Custom Title"),
 *   description = @Translation("Your Custom Description"),
 * )
 */
class YourFetcher extends PluginBase implements FetcherInterface {

  /**
   * {@inheritdoc}
   */
  public function fetch(FeedInterface $feed, StateInterface $state) {
    $config = \Drupal::configFactory()->get('your_custom_module.settings');
    $result = $this->yourHelperFunction($config);
    if ($result !== FALSE) {
      return new RawFetcherResult($result);
    }
    else {
      return new RawFetcherResult('');
    }
  }

  /**
   * Make the API queries to get the data the parser needs.
   *
   * @param ImmutableConfig $config
   *   Drupal Config object.
   *
   * @return string
   *   Returns an JSON encoded array of stdClass objects.
   */
  public function yourHelperFunction(ImmutableConfig $config) {
    /*
     * For my use case I did the following here.
     *
     * 1) Construct and make an API call.
     * 2) Process some data through a queue.
     * 3) Return a JSON encoded array for the parser to process.
     *
     * The fetch() function is expecting a string back.
     */
    return 'I return a JSON encoded array for my use case';
  }

}

Item

Overview

Defining a Feeds Item gives you the ability to map your custom data to fields for your content type/entity/whatever in the user interface. The mappings in here will also need to be duplicated in Parser.

The Feeds Item is also required at the Parser Stage since the goal of that stage is to return an array of Feeds Item objects for the Processor to handle.

This file should be located at:

modules/custom/your_custom_module/src/Feeds/Item/YourItem.php

Code Example

<?php

namespace Drupal\your_custom_module\Feeds\Item;

use Drupal\feeds\Feeds\Item\BaseItem;

/**
 * Class YourItem.
 */
class YourItem extends BaseItem {
  protected $guid;
  protected $title;
  protected $field1;
  protected $field2;

}

Parser

Overview

During the Parser stage, the goal is to take the data given to you via the fetcher and set the data to the fields defined in your Feeds Item and finally return that in a ParserResult object.

In the below example; if from the Fetcher stage you return enough data to populate more than one item you can loop in the parse() function and continue to call addItem() to pass more items to the processor at once. 

Code Example

<?php

namespace Drupal\your_custom_module\Feeds\Parser;

use Drupal\feeds\FeedInterface;
use Drupal\feeds\Plugin\Type\Parser\ParserInterface;
use Drupal\feeds\Plugin\Type\PluginBase;
use Drupal\feeds\Result\FetcherResultInterface;
use Drupal\feeds\StateInterface;
use Drupal\your_custom_module\Feeds\Item\YourItem;
use Drupal\feeds\Result\ParserResult;

/**
 * Your Class Description.
 *
 * @FeedsParser(
 *   id = "your_custom_id",
 *   title = @Translation("Your Parser Title"),
 *   description = @Translation("Your Parser Description")
 * )
 */
class YourParser extends PluginBase implements ParserInterface {

  /**
   * {@inheritdoc}
   */
  public function parse(FeedInterface $feed, FetcherResultInterface $fetcher_result, StateInterface $state) {
    $result = new ParserResult();
    $raw = $fetcher_result->getRaw();
    $items = json_decode($raw, TRUE);

    foreach ($items as $item) {
      if ($item !== NULL) {
        $yourItem = new YourItem();
        // Process out the $item into an easily usable data set.
        $yourItem->set('guid', 'your field values');
        $yourItem->set('feeds_key', 'value');
      }
      $result->addItem($yourItem);
    }
    return $result;
  }

  /**
   * {@inheritdoc}
   */
  public function getMappingSources() {
    return [
      'guid' => [
        'label' => $this->t('GUID'),
        'description' => $this->t('Unique ID for Feeds.'),
      ],
      'field1' => [
        'label' => $this->t('Field 1'),
        'description' => $this->t('Field 1 Description.'),
      ],
      'field2' => [
        'label' => $this->t('Field 2'),
        'description' => $this->t('Field 2 Description'),
      ],
    ];
  }


  /**
   * {@inheritdoc}
   */
  public function getSupportedCustomSourcePlugins(): array {
    return ['csv'];
  }

}

Processor

Overview

TODO.

Code Example

TODO.

Add hooks for imported items

Overview

If you want to act on imported items, you can do so by using normal hook_ENTITY_TYPE_insert() and detect if an entity is being imported with the parameter $entity->feeds_item->imported.

Code Example

Here is a snippet of code from MegaChriz comment in #3047520-3: Option to send welcome email on user import to send a welcome email when a user is imported :

/**
 * Implements hook_ENTITY_TYPE_insert() for 'user'.
 */
function mymodule_user_insert($entity) {
  if ($entity->feeds_item->imported) {
    _user_mail_notify('register_admin_created', $entity);
  }
}