Re: tar: compress fast - extract very slow



On 2007-10-22, shedir@xxxxxxxxx <shedir@xxxxxxxxx> wrote:
Hi folks,


I'm having this behavior on different machines and do not know why:
* I create a tar.gz-file VERY fast (about 20minutes for 120GB => 4GB
(=not much real data in it - mostly empty))
* I extract the same tar.gz-file BUT VERY SLOWLY (about 10 hours!!!)

At compressing the CPU-load is about 100% => OK
At extracting the CPU-load is about 25% => boring

Is this a problem of the filesystem or a problem of tar?
I have tried this on vmfs, ext3 and riserfs - always the same result!

How can I fix this - as I need a fast restore of the uncompressed file
in a disaster-event.

You list the compression size ratio as 120GB/4GB = 30. The
time ratio is 10hr/20min = 30. Hmmm...

when doing a full uncompress operation, it's writing the
120GB of data to disk. The writing of that much data would
appear to be disk-I/O-bound, which is why the CPU is only
25% busy. Being as you've tried different filesystem types
with no improvement, I would guess it's the physical disk
drive that's the limiting factor.

Have you tried simple ext2 rather than ext3? There's a
chance that _might_ help. Do you have DMA enabled on the
disks(s)? If not, that should help, provided it works.

--
Robert Riches
spamtrap42@xxxxxxxxxxx
(Yes, that is one of my email addresses.)
.



Relevant Pages

  • Re: Typesetting email automatically (macro package?)
    ... I wound up writing a Perl script to extract the ...
    (comp.text.tex)
  • Re: Parsing a data matrix.
    ... information before starting writing useless code. ... Let say I will like to extract the similarity values of item C ... > This list isn't a free script writing service, so if you don't have any ...
    (perl.beginners)
  • Re: Problem with zipfile and newlines
    ... I think I've worked it out after reading the 'Binary mode for files' ... The upshot of this is that writing out the zipfile-extracted files ... I've got a zipped archive that I can extract files from with my ... files in python via the zipfile module. ...
    (comp.lang.python)
  • Re: Connes & Marcolli paper on renormalization
    ... > If you follow my recipe of writing the time-ordered exponential ... > in the momentum representation, you can extract a formula for how to ...
    (sci.physics.research)