unable to extract tarball with tar

Crazy!

I have no problems tar zxvf'ing the feature and here is my info:

yh@ubuntu:~/downloads$ tar --version
tar (GNU tar) 1.20

yh@ubuntu:~/downloads$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.10
DISTRIB_CODENAME=intrepid
DISTRIB_DESCRIPTION="Ubuntu 8.10"

Thinking about what the next debugging steps are...

So this is crazy but... I applied the patch and gave it a shot -- this is what I get:

$ tar zxf imageboard.tgz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors

tar xf works however. : |

Patch from #2 works fine for me. Tested in Ubuntu 9.04 with the graphical tool (File Roller) and from command line (tar xzf).

Patch #2 also fixes the issue for me on Windows XP using WinRAR 3.62.

Please forgive my ignorance, but could someone please tell me how to install/apply this "patch"? I'm getting this error when I try to extract a file in cPanel (ex. - og) - Thank You!

Patch #2 doesn't work for me using V 1.1.2.33.

Without the #2 patch on OSX 10.5.6 (tar version 1.14)

Macintosh-9:modules $ tar -zxvf user_public_profile.tgz 

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors

With the #2 patch, removing the gzencode (size now 38915 bytes instead of the previous 3567 bytes) step i get:

Macintosh-9:modules dayre$ tar -zxvf user_public_profile.tgz 

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors

I get the same error as @dayre, both before and after the patch.

RE #6:

If you look at the code it's clearly applied two times.

where? I only see one gzencode() function call (which you have removed in the patch)..

Strange .. this patch and the patch in the other issue http://drupal.org/node/592242 dont seem to do it for me. This patch makes it even worse as the main tgz is no longer able to be opened whereas before I could open that but not the inner tar .. the files however do open on my linux box in the command line (before any patches) just not on winblows desktop with 7zip.

I think the custom function which creates the tar file is not perfect. Some applications are not able to untar it..

On windows you may try Total commander, the built in packer works with the current .tgz file..

With the help of some other folks we were able to determine (somewhat tentatively) that the problem lies further down the stack than the tar/gz implementation in Features. In particular, writing the tar/gz directly to disk seems to work, while downloading output from the browser does not work (for some people). This makes me think this issue may be related to

- Web server settings or
- Browser settings

I'll be looking into this issue further but if you plan on doing more work with this issue it would be great to confirm the assumptions above before moving forward.

Just wanted to mention that I am bumping up against a probably related problem. I'm running 6.x-1.0-beta3, and when I download a feature it can't be opened properly if saved using the default filename.

However, if I change the filename from the suggested featurename.tgz.gz to featurename.tar.gz it opens right up with no problem.

The outer tgz does open but I can confirm the inner tar does not after downloading from any of my browsers and trying to open from within my desktop environment.
FireFox 3.53
Chrome 3.0.195.24
Safari 4.0.3
IE 7

However if I move the tgz to my ubuntu server I can untar it there and everything works as expected.

Im on windows vista .. and my test sever is Ubuntu Server 8.10 (inside vmware workstation)
tar (GNU tar) 1.20

How would I trigger the export to tgz directly to disk from the features export? ... ive tried the drush commands but nothing seems to do that
Here is what my results of drush features commands were...

[aegir@hive][~/drupal-6.14]
$ drush -l mesite.hive features export blog_feature
 /usr/bin/drush: line 45:  8529 Segmentation fault      /usr/bin/env php $SCRIPT_PATH "$@"

[aegir@hive][~/drupal-6.14]
$ drush -l mesite.hive features update blog_feature
Module appears to already exist in sites/all/modules/metech/blog_feature
Do you really want to continue? (y/n): y
Created module: blog_feature in sites/all/modules/metek/blog_feature

But its not tgz'd ... how did you get it to export a tar'd feature directly to disk as u say? .
The export command fails 4 me as u see above, Im running drush head from a few weeks ago.

Cheers!
Josh

Ahah!

The problem lies in output buffering. Note how Drupal's own file_transfer() kills the output buffer completely before transferring a file. Modifying Features to do the same fixed our issues with it.

A patch against CVS HEAD is attached.

(For those who are curious, a little XDebug sleuthing showed that a single space was in my output buffer and that was blowing this all up. It's probably the result of some contrib module with a space before the opening <?php tag.)

couldnt get your patch to work, did you mean to patch features.export.inc instead? The code you referenced is "almost" there in the latest dev and not at all in head... are you using an outdated head?

I did try adding your if statement to the similar function and it does not seem to have fixed the bug for me.
I also tried to test head which resulted in none of my features showin up over @ admin/features, so I didnt get too far with that one.

Oh well .. not quite there yet! .. anyways it not a debilitating bug for me since I can untar the inner tar from my linux command line just not in windows, which isnt the worst thing really. Basically i just need to untar then tar it again (which is required) for adding these features to my feature server .. otherwise when drush downloads the updates they wont untar.

Cheers for taking a stab at it.

While the problem is being fixed, a tip for Windows users.
I use 7-zip and had problems opening the TAR inside the features TGZ. I first assumed it was a problem of 7-zip so I used Winzip (11) instead. Since I have no licence for Winzip, I added Peazip to my PortableApps and that one also works for me.
While looking for info on features I accidentally fond this post and realized that it might not be a 7-Zip problem.

Rino ... so you were able to get them to open in Winzip & Peazip but not 7zip?

I use 7zip on my windows desktop... but I dont think its the unZipper programs. Because when I was testing If I left the tgz as output from the features module and added it as a release in my features server when running an update on the feature with drush .. it fails saying its not a proper archive or someasmuch ...and thats on the same system that tar'd it to begin with. Like I was saying above the only thing that worked for me was to untar and then retar from the linux command line, then all things were good and happy.

After reading all the posts, I have the idea that the problem has to do with packer routines. How strict are the rules for packing unpacking in the different tools and how do they deal with unexpected content? How do they react on a missing or an extra (control)character / space?

After all, Linux does not pack the feature, The feature server is doing that, using the webserver.

I am checking out OpenAtrium and I can not use Linux for the intended environment so I did not test. If Atrium is not working without linux, it will not be used. I worked with Winzip and Peazip and had no problems with unpacking. The features are recognized and can be used.

I've committed Steven Merrill's patch here: http://drupal.org/cvs?commit=274786

I would love to know whether this addresses people's issues or if there are still remaining problems.

That appears to have partially fixed the problem.

Using tar xzf [filename] I can properly extract the files.

Using 7-zip I can now extract the tar from the tgz file. However, I cannot extract the contents of the tarball.

I'm going to close this for now as there have not been any additional complaints since this patch went in. Please reopen if there are remaining issues and you have some actual clues as to how to start addressing them : )

I added following line into "features.export.inc", line 432. This seems to fix the problem with 7-Zip:

function features_tar_create($files) {
  $tar = '';
  foreach ($files as $name => $contents) {
    $binary_data_first = pack("a100a8a8a8a12A12",
      $name,
      '100644 ', // File permissions
      '   765 ', // UID,
      '   765 ', // GID,
      sprintf("%11s ", decoct(strlen($contents))), // Filesize,
      sprintf("%11s", decoct(time())) // Creation time
    );
    $binary_data_last = pack("a1a100a6a2a32a32a8a8a155a12", '', '', '', '', '', '', '', '', '', '');

    $checksum = 0;
    for ($i = 0; $i < 148; $i++) {
      $checksum += ord(substr($binary_data_first, $i, 1));
    }
    for ($i = 148; $i < 156; $i++) {
      $checksum += ord(' ');
    }
    for ($i = 156, $j = 0; $i < 512; $i++, $j++) {
      $checksum += ord(substr($binary_data_last, $j, 1));
    }

    $tar .= $binary_data_first;
    $tar .= pack("a8", sprintf("%6s ", decoct($checksum)));
    $tar .= $binary_data_last;

    $buffer = str_split($contents, 512);
    foreach ($buffer as $item) {
      $tar .= pack("a512", $item);
    }
  }
  $tar .= pack("a1024","");  //adding this line
  
  if (function_exists('gzencode')) {
    $tar = gzencode($tar);
  }
  return $tar;
} 

I've got the idea from http://www.php.net/manual/de/function.gzwrite.php#88746

The same problem with beta6. Fix #25 doesn't work for me.

kenorb@NS8210H:~$ tar zxvf file.tgz
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Error exit delayed from previous errors
kenorb@NS8210H:~$ tar --version
tar (GNU tar) 1.20
$ php -i | grep zlib
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

on Ubuntu 9.03

Workaround:

zcat file.tgz | tar xvzf -

Proper patch.
See: http://www.mimetype.org/

#25 & #28 do not fix the issue with 7-zip.

#29: In my opinion there are two different problems, one on Linux caused by wrong headers, and second probably because of some bad compression inside compressed file.
Problem that I had and it's first reported is caused by double compression (on Linux) because of the wrong headers. Please test the patch in #28 by someone who experienced problem on Linux.

Patch in #28 allows me to extract the archive where before I could not, so that is great. Though I still get a corrupt CRC error on extraction.

This worked for me. I opened the file with ZipGenius, copied all the files and pasted into a new folder named like the feature created.

Patch in #28 fixed things for me

Patch #25 fixed things for me, for 7-zip, on Windows, patch #28 didn't.

#35 David:
There are too separate issues, on Linux with double compression and on Windows with 7-zip.
Let's fix tar.gz issue first which was originally posted.

David: please test the patch #28 by how was originally reported by uncompressing it via command: tar xvzf file.tar.gz .
If you have problem with 7-zip, create separate issue with some description what happened.

Using Firefox and 7-zip also. Patch #28 did not fix the problem for me. Suggestion #25 DID work for me. Thanks Skorpjon.

#25 does fix it for 7zip! Odd that it didn't work the first time I tried... oh well. It's working :)

I've committed both of these changes here: http://drupal.org/cvs?commit=370146 and they seem to work fine for me.

I would really appreciate it if everyone could give this a whirl and just give a thumbs up or down if you are no longer running into problems.

the patch in #39 fixed this issue for me on Windows. Before adding the patch, the .tar inside the .tgz from features would not open in 7-zip. After patching, it works fine.

Setting to fixed. Please reopen if more problems are found.

Same here using 1.0

The downloaded export is not a tar.gz but simply a tar.

I don't know if that's normal.

tar -xvvf default_features-6.x-1.0.tar
tar -xvvf default_features-6.x-1.0.tar
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Error exit delayed from previous errors

The funny thing is I'm using features on 2 sites on the same codebase on a multisite, and one site exports fine but the other does this thing.

For anyone having trouble with the tar ball.
I write a small module that write the feature directly to the file system.
It can save a lot of time if you working on local sever and don't need to download the feature.

http://drupal.org/project/features_direct_save.

Good life.
yuval.

Same problem... features 7.x-1.0-beta3 and 7.x-1.x-dev

jn@UL80VT:~$ tar --version
tar (GNU tar) 1.25

jn@UL80VT:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=11.04
DISTRIB_CODENAME=natty
DISTRIB_DESCRIPTION="Ubuntu 11.04"

reopening this, happening with latest -dev release.

# tar zxvf feature-7.x-1.0.tar 

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

# tar --version                                                                                                                              
tar (GNU tar) 1.26
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.

the error message says it: don't use the "z" flag because the tar is not zipped.

Seems to be active on latest 7.x-2.x

dgtlmoon 18:02 ~/Downloads  $ ls -al chia_recipes-7.x-1.12.tar 
-rw-rw-r-- 1 dgtlmoon dgtlmoon 89090 Jun 13 18:01 chia_recipes-7.x-1.12.tar

dgtlmoon 18:02 ~/Downloads  $ tar xvf chia_recipes-7.x-1.12.tar 
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors

So looks like a real file but something wrong in the way it's written

dgtlmoon 18:03 ~/Downloads  $ file chia_recipes-7.x-1.12.tar 
chia_recipes-7.x-1.12.tar: data

On my system, a real working tar file will say

dgtlmoon 18:04 ~/Downloads $ file out.tar 
out.tar: POSIX tar archive (GNU)

And gzipped

dgtlmoon 18:04 ~/Downloads  $ file out.tar.gz 
out.tar.gz: gzip compressed data, was "out.tar", from Unix, last modified: Thu Jun 13 18:04:29 2013

So it seems a problem creating the archive, in my case the problem is that there are two whitespaces at the start of the tar

0000000 2020 6863 6169 725f 6365 7069 7365 632f
0000010 6968 5f61 6572 6963 6570 2e73 6f63 746e
0000020 7865 2e74 6e69 0063 0000 0000 0000 0000
0000030 0000 0000 0000 0000 0000 0000 0000 0000

(0x20, 0x20) if compared to a normal tar archive, these should not exist

When I delete those two char's, everything seems to work OK again

dgtlmoon 18:07 ~/Downloads  $ file chia_recipes-7.x-1.12.tar 
chia_recipes-7.x-1.12.tar: POSIX tar archive (GNU)

Can anyone else having faulty feature archive's being generated hexdump the first few lines too?

I've got the same thing happening to me on 7.x-2.0-rc3 on a linux environment. When deleting the whitespace the file is fixed. When using my local windows environment the file doesn't contain the whitespace.

Just throwing in a guess -- see if your site regular pages have extra white space?

my site did have extra whitespace on regular pages for a while, not sure if this was the same time I was experiencing this bug or after (was a few months ago, so I forget).

Definitely not seeing this on any sites here and looking at the code it's pretty obvious that it's not writing spaces to the tar file in the Features code. So it must have been something specific on your site.

Re-open this if you find a way to reproduce it.

I'd suggest debugging the output via xdebug to see where these white spaces are coming from.
Sometimes some Apache mis-configuration could cause that or wrong encoding.

I'm getting the same issue on one of our servers but not on any others. I'll try to compare apache config on the server where the issue occurs and figure out how it differs from other working servers.

Hey all - As this issue seems stalled and specific to individual users environments and not Features code, I'm closing it as cannot reproduce. If someone comes up with a way to reproduce it that is related to actual Features code, please feel free to reopen it.

Thanks!

Many thanks to others who pointed out that the extra space at the beginning of the TAR is most likely caused by an extra space before a file's <?php opening tag. For me, settings.php (of all places!) was the culprit. After removing the space, problem solved. I'd like to share my grep command that helped me find this, in case it helps someone else. I ran it from Drupal root. Of course, you can modify as needed:
grep -r --include="*.php" --include="*.inc" --exclude="*.tpl.php" "^\s* <?php" .