How to exclude ignore git folders on a Drush adb archive dump backup?

currently when i do a drush adb, it tar's up everything in the folders even the git folders, which make it alot bigger than it should be.

Comments

moshe weitzman’s picture

Title: How to exclude ignore git folders on a Drush adb archive dump backup? » How to exclude ignore git folders on a Drush archive dump?
Category: support » feature

not currently supported.

dman’s picture

I'm getting the same. I'm trying to learn from the archive-dump approach (and similar current discussions on interoperability of site dumps).
I've got .svn stuff that chews up the processor with all the little files for *every single folder*

On top of that, There are a few other places I'd like to exclude from the packager, just to be practical.
files/backup_migrate for one.
files/imagecache for two (the transferred site can rebuild them, thankyou)
files/tmp.

and surely more. (some of the folk in teams that came before me dropped sql dumps in unlikely places within the site)

So, in a similar way to how we easily wedged in ssh-options a while ago, #686384: Allow the site-aliases configs to define extra options when invoking ssh
I think we will be able to wedge in a bunch of exclusions in ... um ... --tar-options ?
It looks that the current code uses tar (-r = --append) a lot. I had a similar handmade script that copied everything into a temp folder then tarred at the end, so my use of those flags is not identical with the way I'm trying to follow by using archive-dump. Still, looks very do-able. And may cut the processing/tarring/zipping time that I'm seeing down to a quarter.

dman’s picture

I can do a patch for this pretty easy, if I can get a "sure, why not?" from the team.

patoshi’s picture

a patch would be great! is it a modification when you tar recursively?

dman’s picture

Assigned: Unassigned » dman

I'll stick this in my todo next time I'm working with dumps and custom drush stuff. Won't be too long.

The options will be just the same as normally get passed to tar, they will go through verbatim, so something like
drush archive-dump --tar-options="--exclude=.git --exclude=.svn" @sitename
would be normal.
Then you get to put that option into drushrc.php under the command-options as needed.

janusman’s picture

Status: Active » Needs review
StatusFileSize
new3.37 KB

How about this?

This implements --tar-options just as mentioned in #5.

greg.1.anderson’s picture

Looks pretty good to me.

dman’s picture

Bravo - nice and small and tidy.

moshe weitzman’s picture

That looks good. It would be even cooler if one could use tokens like %files (see example.aliases.drushrc.php for more about tokens). Is this easily done, Greg?

greg.1.anderson’s picture

StatusFileSize
new4.5 KB

Can't drive this to completion right now, but here's a patch that should mostly work. Does not support %modulename, but %files and %modules and %themes are supported.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

I think this is good enough to commit. We should add an Example which shows how to use %files.

moshe weitzman’s picture

Status: Reviewed & tested by the community » Needs work

Not applying cleanly for me.

jamesharv’s picture

Status: Needs work » Needs review
StatusFileSize
new4.2 KB

This is going to be really useful for me on a current project, so I've updated the patch so that it applies cleanly against current master. Hope that helps.

skwashd’s picture

Assigned: dman » Unassigned
Status: Needs review » Reviewed & tested by the community

I've played with the patch from #13 against a fresh git clone and it works as expected.

greg.1.anderson’s picture

Trivial point, but this patch is still missing the example requested in #11. Other than that, looks okay to me.

jonhattan’s picture

Note the existance of ‘--exclude-vcs’ in recent versions of tar. http://www.gnu.org/software/tar/manual/html_section/exclude.html

skwashd’s picture

Status: Reviewed & tested by the community » Needs work

@jonhattan CentOS/RHEL 5 doesn't support and right now one of my big clients is stuck with that for their environment. We also want to exclude css, js and some other paths from files/ in our archive. I think @JamesHarv's patch covers both of those.

Marking as needs work based on #15.

Disclaimer: I work with @JamesHarv.

jonhattan’s picture

Mine was just a note in the aim to just document its existance. It would be fine to add it as a command example but I can live without that. Beyond that I'm fine with current patch.

blame centos btw :p

jamesharv’s picture

Status: Needs work » Needs review
StatusFileSize
new4.79 KB

Here's a patch which includes an example of using the %files placeholder in --tar-options (Addressing #15).

skwashd’s picture

Status: Needs review » Reviewed & tested by the community

#19 looks good to me and has the extra example.

greg.1.anderson’s picture

Status: Reviewed & tested by the community » Fixed

Committed; thanks.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.