Re: Zbip2 Size limitatations ?

On Friday 30 December 2005 23:56, Franck Y wrote:
>Hi everyone,
>
>I make several backup with bzip2, it seems that he has the better
> compression.
>
>I do my backup like this for several folders.
>
>tar cvf - /home/data/ | bzip2 -9 > /backup/data.tar.bz2
>
>When i test the archive with bzip2 -t archive.tar.bz2, everything is
>correct when the archive is under 1 Go. But i get CRC error when it's
>upper ....
>
>
>Has anyone experience this ?

No, other than I've also heard that bzip2 has problems with source
streams above 2GB, which would explain the effect your are seeing.

As an amanda user myself, it seems rather foolish to make a single
backup file that big considering what you have lost if its
unrecoverable, and bz2 stuff can be entirely too sensitive to the
medium its on. Its demonstrated that to me on many occasions.

"gzip best", OTOH, is considerably more robust in my experience, and
faster in the compression cycle than bz2. And, when you have a good
backup manager, the individual files can be made fairly smallish so
that each file in the archive remains a manageable size. My amanda
disklist has 44 entries scattered over 2 machines because I've
purposely made each entry only one, under a gigabyte of raw data,
directory tree. Having smallish files means amanda can more easily
manage the storage to make maximum use of the tape it is allowed to
use each night, and I've often filled the tape to 98% capacity without
ever hitting an EOT from the drive. With a mixture of entries that
compress, and those that aren't (because they already are, like rpms &
tar.gz's etc) gzip can routinely reduce 16GB of raw data into 6GB on
the tape, so I don't consider the better compression performance of
bz2 to be an advantage other than in the eye of the beholder. Storage
today is cheap, use what works more dependably rather than that which
may work slightly better, but only 99% of the time. For backups, a 1%
failure rate is unacceptable to me.

>--
>Franck

--
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules. I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

--
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Relevant Pages

  • Re: Backup Software?
    ... If the hardware itself support compression then NT Backup will turn it ... read through the file system and tables of the running operating system. ... or save a physical image instead). ...
    (microsoft.public.windowsxp.general)
  • Re: Compressing backup file
    ... I am looking at creating a backup file, ... let it run for 2 hours before giving up on a test saveset). ... do the compression there, but wouldn't be able to automate ... Unofficial OpenVMS Hobbyist Support Page: ...
    (comp.os.vms)
  • Re: DDS4 - hardware compression not working
    ... I'm prompted for a 2nd tape. ... I'm using NTBackup, the built-in backup ... I've tried it with and without software compression ... I've watched the data stream, ...
    (comp.periphs.scsi)
  • RE: SBS 2003 SP2 / DLT compression?
    ... The NTbackup supports hardware compression, and it is based on the backup ... The two most common tape drives are DDS3, ... Hardware compression is turned on by default if you use SBS Backup ...
    (microsoft.public.windows.server.sbs)
  • Re: SBS2003 Backup Probs - Still
    ... Compression should be done by the hardware, ... Les Connor [SBS Community Member] ... try a small manual backup on each>> tape first. ...
    (microsoft.public.windows.server.sbs)