Re: File backup directly to remote?



On Jun 7, 4:26 pm, Roby <r...@xxxxxxxxxxxxxx> wrote:
mechphi...@xxxxxxxxx wrote:
On Jun 6, 4:35 pm, Roby <r...@xxxxxxxxxxxxxx> wrote:
mechphi...@xxxxxxxxx wrote:
I have a server (we'll call "Alpha") that has 88GB of /home and only
20GB of free space.
I need to make backups (or rsync) the folder to a remote server
("Beta"). I need to do it in such a way as to copy directly over, or
TAR it and copy but without creating the tar on Alpha first.

I did some looking, and I found two possibilities I'd like feedback
on, if I may.
Rsync (over ssh) seems reasonably convenient. But if I go that route,
I'll still need to make an initial tar of /home to send over to Beta,
and so I'm still stuck with too little free space on Alpha to tar and
then copy.

So, I did some more looking and found a possible way to tar directly
to the remote:

# tar zcvf - /home | ssh <beta's ip> "cat > alpha.tgz"
I guess I can edit that to something like:
# tar zcvf - /home | ssh userback@<beta's ip> "cat > alpha.tgz"

What about using scp? Wouldn't that be better? Anyway...
Will this really send the contents of /home directly to a tar on the
remote server without taking up additional space on its own drives?

If I want to automate a backup, I suppose I can use a password-less
copying ssh keys over so I can avoid password prompting.
I found this script I can use for automated incremental backups which
is PERFECT!
Except for the space thing...
http://www.faqs.org/docs/securing/chap29sec306.html

Can I alter the tar command lines to do the over ssh thing?

Or, any better recommendations?
Thanks!
Liam

rsync isn't just for updating an existing set of files on the
destination machine.  It will be happy to create the initial
file set, so no need to pre-install files from a tarball.

Incidently, if the two machines are on the same lan (i.e.,
don't have the internet in between), then you can skip the
ssh and go a bit faster.

Roby

Well, that initial rsync of 88GB is going to take a LONG time without
compressing....
I'll look more into rsync.
Unfortunately they're nowhere near the same LAN. =) I've had to open
up and forward ports in the receiving firewall in order to communicate
with each other.

Playing with ssh options I've gone ahead and shared the public keys to
try to do an automated passwordless transfer. Except, I keep getting
permission errors like:
bash: /media/Teraback/test.tgz: Permission denied
in Beta even though the account I'm using has recursive write access.
Although, and this could be a problem later, /home is obviously not
owned by the account I'm using with the shared keys so reading
contents may be an issue....
=/

Thanks for any advice.

rsync's 'z' switch turns on compression; i.e., rsync -avz ...
but I don't know how much elbow-room it needs at the source end.

Between the compression option, the encryption option (-e), and the
feature of only copying changed/new files, this is going to end up
perfect. :) -knocks on wood-
As far as I can tell, rsync uses no extra drive space on the server
end, and just a touch of system resources (thank goodness for quad-
core CPU's and fast, high buffered drive RAIDs. :)
This initial rsync is going to take a looong time....)

Thanks for the advice!
-Liam
.



Relevant Pages

  • Re: File backup directly to remote?
    ... 20GB of free space. ... I need to make backups the folder to a remote server ... TAR it and copy but without creating the tar on Alpha first. ... Rsync seems reasonably convenient. ...
    (comp.os.linux.misc)
  • Re: File backup directly to remote?
    ... 20GB of free space. ... I need to make backups the folder to a remote server ... TAR it and copy but without creating the tar on Alpha first. ... Rsync seems reasonably convenient. ...
    (comp.os.linux.misc)
  • Re: File backup directly to remote?
    ... 20GB of free space. ... I need to make backups the folder to a remote server ... TAR it and copy but without creating the tar on Alpha first. ... Rsync seems reasonably convenient. ...
    (comp.os.linux.misc)
  • Re: Rsync backup to tar or gzip archive
    ... >>only" is because I'm going be be doing nightly backups to a remote site ... >>Additionally, as the file systems I'm going to be backing up to are SMB, ... > FWIW, tar can back up changes only, by keeping a list of files it has ... I'd recommend adding a drive to the box where you run the rsync ...
    (Fedora)
  • Re: Question about Backing up with tar
    ... I suggest rsync instead of tar. ... Rsync has a highly efficient algorithm that copies only changed data, ...
    (Fedora)