Re: Rsync and Compression

From: Tom 'Needs A Hat' Mitchell (mitch48_at_sbcglobal.net)
Date: 04/21/04

  • Next message: Michael Mansour: "Re: Clustering on FC1" 
    Date: Tue, 20 Apr 2004 23:38:50 -0700
To: For users of Fedora Core releases <fedora-list@redhat.com>

    On Wed, Apr 21, 2004 at 06:59:37AM +0100, WipeOut wrote:
    >
    > I have created a script that will backup some files from my web server
    > to my local server via the internet.. I will be using Rsync over SSH..
    >
    > I am unsure of the best way to impliment compression since bothe Rsync
    > and SSH have the ability to compress data..

    On a local link I turn compression off. The CPU effort and latency to
    compress then uncompress does not justify the time saved in transfer
    time.

    Also most digital content (images, rpms) do not compress enough to
    justify the cycles. Some increase in size....

    Since this depends on your content and your local system capabilities
    I can only advise you to list all the possible environment knobs in
    your script and then benchmark by turning them on and off.

    Of interest if compression proves to be an advantage for backups then
    you should check into compression on the httpd server side. It is
    possible to present compressed content to an aware client that is then
    expanded locally by the browser.

    If you think about it a distant proxy server could do this and make
    the link look faster. Some services are apparently doing this and
    charging extra for it. As a content provider you should do this to
    save both you and your customers bandwidth. It might be interesting
    to make sure that precompressed content is not expanded to make a link
    look slow. I seriously doubt that any service would like to be caught
    doing this ... but ... ya never know.

    The backup impact of this is that the pages on the web site are
    compressed already and will not compress any more. So why bother. 

    -- 
	T o m  M i t c h e l l 
	/dev/null the ultimate in secure storage.
-- 
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list
  • Next message: Michael Mansour: "Re: Clustering on FC1"

    Relevant Pages

    • Re: Server backup times very slow (mB/min etc) Ntbackup and BackupExec; These times typical?
      ... Gigabit eithernet to do the backups to a central backup server, ... off compression, but the speed increase was minimal.. ... the d drive is Raid 5 with 3 drives and the E drive is a single SATA ...
      (microsoft.public.windows.server.general)
    • ISA 2006 EE Web publishing crashing
      ... one web server for a month or so, then today I finally switched my 5 web ... ISA Server was unable to compress a response body from ... for completing the compression process. ... Compression Failure (Allocated Memory Exhausted) ...
      (microsoft.public.isa)
    • RE: OWA Calendar loading error
      ... On a server that is running Microsoft Windows Small Business Server 2003 ... In the drop-down menu next to Compression, ... IIS 6.0 compression corruption causes access violations ... Open the IIS Temporary Compressed Files folder. ...
      (microsoft.public.windows.server.sbs)
    • Re: Rsync and Compression
      ... >>I am unsure of the best way to impliment compression since bothe Rsync ... Yes but my backup will be going via the internet so I would rather ... >your script and then benchmark by turning them on and off. ... Already being used on my web server.. ...
      (Fedora)
    • Re: Compressing file system, NFS
      ... > for decompression and could even do compression (which I don't need at the ... > moment) without slowing down the clients. ... an encrypting userland NFS server. ...
      (comp.unix.bsd.freebsd.misc)