Re: Looking for advice on rsync

From: Bill Unruh (unruh_at_string.physics.ubc.ca)
Date: 10/16/04 

Date: 16 Oct 2004 21:03:43 GMT

"Tony Lawrence" <pcunix@gmail.com> writes:

]Yesterday I was at a client site where they explained that they wanted
]to keep a stand-by server up to date and ready to take over in case of
]main system failure. Fine, lots of people do that, and currently they
]are doing it by restoring backups every morning. What they were asking
]about was using rsync or some other mechanism to keep the machines
]more current.

]My first reaction was to question them about their app: it's
]apparently a mess of Basic programs that work on hundreds of different
]data files 24 x 7. I asked if they shut down the users for backup.
]They don't. I explained that the backup can't really be guaranteed
]consistent if users are writing data to files because obviously files
]are going to be backed up while they are being written to. As the
]files are related to each other (A/R header and detail files, indexes,
]etc.) you can have inconsistent versions on the backup media.
]. Somebody told them (obviously incorrectly) that rsync could prevent
]this. There are databases that will let you replicate data while
]you run, but that's application level, and this app has no such
]ability. Rsync can't do any better than a backup for that.

]I talked about snapshots, and asked if they could shut down users for
]the very brief time it takes to do that. Nope. Can't ever stop the
]flow of data. I explained that rsync can't answer that problem any
]better than a tape backup can: files and indexes may be inconsistent
]with each other.

]I also wonder if rsync's rolling checksum might even make things
]better or worse. On the one hand we get less data transferred as
]opposed to just a rcp or whatever, but open files that may be getting
]writes during the checksum make things even more confusing.

]They can't change the app. They can't shut down users. Their bank says
]they have to have better disaster recovery. These things seem
]impossible to reconcile. My feeling is that if the banks demands have
]to be met, then the user community HAS to put up with periods where
]they can't use the app. If they use snapshots, that period can be
]brief, otherwise its going to be fairly long (13 GB of data to
]transfer).

They need some sort of file locking mechanism within their program so that
if the lock file is present, the app suspends its use of the file until it
is freed. That way rsync. or whatever could lock the file, do the update
and then unlock it. The users would simply notice a brief hesitation in
service-- something they already probably experience 10 times a day anyway.

Relevant Pages

  • Re: Looking for advice on rsync
    ... >etc.) you can have inconsistent versions on the backup media. ... Somebody told them that rsync could prevent ... >you run, but that's application level, and this app has no such ... If they use snapshots, that period can be ...
    (comp.os.linux.misc)
  • Looking for advice on rsync
    ... My first reaction was to question them about their app: ... I asked if they shut down the users for backup. ... Somebody told them that rsync could prevent ... Their bank says ...
    (comp.os.linux.misc)
  • Re: Looking for advice on rsync
    ... >]etc.) you can have inconsistent versions on the backup media. ... Rsync can't do any better than a backup for that. ... >]They can't change the app. ... >]they have to have better disaster recovery. ...
    (comp.os.linux.misc)
  • Re: In the distant past
    ... I view it in the app intended for it. ... Backup really critical data in at least one other ... Corrupt fonts are possible I suppose, but only on the G5 where I ... RAM in my G5, and 2GB in both the iMac and the macbook pro, I ...
    (comp.sys.mac.advocacy)
  • Re: [opensuse] Suggestions for backup software?
    ... be something capable of bridging DVDs for a total backup, and probably CDs for incremental backup. ... At home I have an old file server running 10.0 and all you need is rsync that is already installed on your system. ... The way I set up rsync is to set up a primary script that calls rsync which then reads a second file to get a list of directories to back up. ...
    (SuSE)