In many cases, existing sites add the S3 module at a later stage and an option to bulk upload the previously uploaded files would be very handy. I would suggest a drush script to quickly achieve the same.

Since I cannot find anything useful in this regard, I've raised a new feature request.

I am currently working on a drush script and shall update the same after some testing & improvements, to ease your job :)

CommentFileSizeAuthor
#1 amazons3-drush-bulkupload-1985962-1.patch3.79 KBkvanniarajan
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

kvanniarajan’s picture

kvanniarajan CreditAttribution: kvanniarajan commented
FileSize
amazons3-drush-bulkupload-1985962-1.patch3.79 KB

Patch for latest dev version (repository checkout) attached, which would add a handy option to drush to achieve the bulk upload functionality.

Usage is as follows:
drush help amazons3-bulk-upload-scheme
Moves all managed drupal files in the given scheme to S3

Examples:
drush amazons3-bulk-upload-scheme public Upload all managed files in public scheme to S3.
drush s3up private Upload all managed files in private scheme to S3. This will remove the access restrictions on private files. So, make sure you understand what you are
doing.
Arguments:
scheme A valid drupal file scheme (without :// suffix). All managed files in this scheme will be uploaded to S3 and saved under s3 scheme.

Aliases: s3up

Let me know if this helps. I would be glad to make any further improvements to the patch, in order to get featured into the module.

kvanniarajan’s picture

kvanniarajan CreditAttribution: kvanniarajan commented
Status: Active » Needs review
justafish’s picture

justafish
she/her
Primary language English
Location London, UK
CreditAttribution: justafish commented
Issue summary: View changes
Status: Needs review » Needs work

Very cool! You should make it a separate module/command that's generic for any stream wrapper though, as I don't see why this wouldn't work for others.

Let me know and I'd be happy to list it on the module page!

nicxvan’s picture

nicxvan CreditAttribution: nicxvan commented

This looks like what I need to move an exising files directory over, however I cannot get the drush command to run, any insight? Website is on pantheon.

nicxvan’s picture

nicxvan CreditAttribution: nicxvan commented

I was able to get it running, it was because the Pantheon project didn't have a settings.php file. Adding that allowed it to run fine.

deviantintegral’s picture

deviantintegral CreditAttribution: deviantintegral at Lullabot for NBCUniversal commented
Status: Needs work » Closed (won't fix)

I like how simple this is, especially for sites with a small number of files. I'd be hesitant to support this for the more typical S3 use case where there's thousands of files to move. In those cases, an independent rsync plus a database update hook makes more sense to me.

Anyways, I also agree this isn't really S3 specific, and would be trivial to make work with any destination scheme. I'm closing this as wontfix, but if you do create a separate project for this please drop the link in this issue.