I am looking for a way to import large numbers of images into Drupal as file entities from an Amazon S3 bucket. The bucket is already setup and working as the file system for the Drupal install.
Ideally I'm looking to get Drupal to recognise these files, without having to actually process them, really I just need to tell drupal they exist so they are picked up by the media module. I can generate a csv of the files with basic details (from an existing database) and have looked at using various approaches with Feeds to import them directly as entities, but Drupal spends a looong time processing each of these. Primarily because S3 is very slow and there are potentially hundreds of thousands of files involved.
Any thoughts would be greatly appreciated.
Comments
I ended up using S3FS (https:
I ended up using S3FS (https://drupal.org/project/s3fs) over the Amazon S3 module. It's much faster at saving/reading to S3 in my experience.