Migrate Process S3

When integrating with a third-party API, you may have file uploads stored on Amazon S3. While you could link to files in the bucket, this may not be ideal for reasons such as:

• the bucket may not be public and cannot be made public
• the bucket content must be mirrored locally for legal or privacy reasons
• you want to leverage Drupal-side file processing (like image styles)
• you don’t want to use s3fs or a whole S3 file system replacement

This useful module provides a process plugin to download objects from an S3 bucket as files to your Drupal site.

See the README for more details.

Requirements

• Drupal 8

This Module is Part of a Migration Framework

This module is part of a migration framework created for a client project in which we migrated information from Digital Measures to a Drupal site. However, the specific problems each module solves are common to migration processes in general. With the exception of Digitalmeasures Migrate, these modules are fairly generic and can be used for a variety of other migrations.

For more information on how this framework was used for a Digital Measure Migration, read:

• We Give Back: Drupal Module Framework for Migrations
• Case Study: University of Minnesota’s Provost Office

Here are the modules in the migration framework:

• Digitalmeasures Migrate
  Provides a method to access Digital Measures API through Drupal.
• Migrate Process XML
  Reads XML and allows you to extract particular key sections using XPath.
• Migrate Process S3
  Allows you to download objects from an S3 bucket as files to your Drupal site.
• Migrate Process URL
  Allows you to manipulate URL values that are provided within the data.
• Migrate Process Regex
  Provides a way to use Regular Expressions in a Drupal migration.
• Migrate Process Vardump
  Often used for debugging, this module takes any data given to it and dumps it to the terminal output and then passes it on.
• Migrate Process Skip
  Provides a few different mechanisms to define what is “empty” and should be skipped.
• Migrate Process Trim
  This module provides a quick and simple means to remove leading or trailing characters (such as spaces) in a Drupal migration.
• Migrate Process Array
  Provides utilities for interacting with arrays during migration.