Re: [SLE] Trying to make a near Perfect Backup System
From: Greg Freemyer (greg.freemyer_at_gmail.com)
Date: 07/14/05
- Previous message: Alexandr Malusek: "Re: [SLE] 64-bit Matlab on SUSE Linux 9.3, Opteron CPU"
- In reply to: Dale L Roblin: "[SLE] Trying to make a near Perfect Backup System"
- Next in thread: François Pinard: "Re: [SLE] Trying to make a near Perfect Backup System"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Thu, 14 Jul 2005 09:10:18 -0400 To: suse-linux-e@suse.com
A long post deserves a long answer:
On 7/14/05, Dale L Roblin wrote:
> Hello all,
>
> I would like some advice on what would be considered as a "bullet-proof"
> backup system. The system I am currently using is as follows:
>
> some P4
> 5X 200GB HDD configured as an LVM using ReiserFS (2X SATA, 3X IDE)
> 1X 20GB HDD holding the OS: SuSE 9.2
> Backup Software: Rsync-based shell script run from cron every 24 hours
> to backup the entire division.
> Most of the rsync script was gathered from:
> http://www.mikerubel.org/computers/rsync_snapshots/
>
> I only backup our division, not the entire organisation - that is the IT
> department's problem.
>
> Currently we have no large-capacity tape drive for off-site storage, so
> that is an obvious MUST-HAVE. I am considering the HP Ultrium 100-200GB
> tape device for this, should I be going in another direction with tapes?
>
> The above PC works very well. However, it's greatest weakness is
> disk-failure.
> We recently had the backup system die due to disk failure (there were no
> spare 200GB disks available in our IT section at the time too :( ), and
> at the same time one of our most important data analysis pc's also had a
> disk failure. I could not recover data from the backup LVM, so we had
> effectively lost all data. Fortunately we were able to recover data from
> the analysis PC disk via a data recovery specialist.
>
> Since we have 40+ pc's in the division, the likelihood of both the
> backup system and a crucial PC failing at the same time is remote, but
> it happened. Not surprisingly I was immediately in the "hot-seat" to
> answer why it happened and why there was no plan for this etc...
>
>
> So, my questions on how do I make a "bullet-proof" backup system are
> these:
>
Are you prepared to spend time and money on this project? I would
guess that you will need to spend between $10K (US) and $50K (US) and
a lot of time too. The more "bullet-proof" the more expensive and
time-consuming. Just the software for a project like this could cost
over $20K if you go with the market leader (Veritas Net Backup).
> 1. I need a tape device for off-site storage, are the LTO Ultrium
> devices good enough?
>
I just saw this year plus old write-up on tape drives. Probably worth
your time to read:
http://www.clipper.com/research/TCG2004040.pdf
Anyway, personally I like the LTO, but you talk about the LTO 100/200.
I assume you are talking about a LTO-1 drive with 100 GB native, 200
GB compressed? FYI: The amount of compressibility varies greatly
based on the data type.
Above you show having 1 TB of disk on your backup server.
Is that big enough to hold a full backup? If not, with LTO-1 you
will need a small autoloader or tape library. Small ones are not
outrageously expensive and a lot of backup software know how to manage
them.
Alternatively, there are tape drives out there that will hold almost a
TB (compressed) and I like the idea of getting the full backup on a
single tape.
In particular, the LTO-3 has been out about 6 months and is likely to
be supported for many years. (400native/800compressed). (LTO had 75%
of the market in 2004, so the product line is very unlikely to go
away.)
The largest capacity drive I know of is the Sony SAIT-2 (1 TB Native,
2.6 TB Compressed). (I have no first hand experience with this
drive). It was supposed to be released in 2005, but I don't know if
they are shipping or not. Also I expect they will be very, very
expensive. (ie. $10-15K US each)
> 2. Should have a 'twin' of the above PC so I have a backup of the
> backup, since I can't fit more disks into the box?
Assuming you are going to use a disk to disk to tape backup scheme it
would be very unusual for a department level backup system to have
fully redundant systems.
OTOH, if you have the resources to duplicate everything, that would be
great. I would use SLES 9 with Linux-HA and DRDB to setup a HA
cluster of the two machines and DRDB mirroring the data between the
computers live.
>
> 3. Would a RAID system be better (I have not used RAID yet though) -
> would that need to be hardware RAID?
Absolutely you need RAID. It sounds like you have a five disk based
solution currently. I suspect that if any one drive fails, you lose
it all. If true the odds of your failure are 5 times the odds of a
single disk failure. Very Bad.
Linux has great software RAID, but I prefer hardware RAID and it
really is not that expensive.
When building a backup server like this, read speed is the most
critical aspect. (ie. Tape drives eat lots of data very quickly, your
disk system has to be fast to keep up.)
I would use RAID 5. For this kind of load it should be faster than
RAID 1, and might be faster than RAID 10.
>
> 4. I have read a little on SUN Storedge systems, can these be used with
> Linux, and are they better than anything else?
What do you want to do with that? Just build your own solution.
> 5. So far I have only been using shell scripts for backup, should I be
> considering commercial packages, if so which are "in-favour" at present?
>
If you decide to look at commercial solutions, I would look at HPs
Data Protector. Especially for a solution like this, it is pretty
cost effective. (ie. HP charges by the number tape drives you are
using, not by the number of computers being backed up.)
DP also supports the disk-disk-tape concept, but I must admit to not
having set that up yet myself.
>
> Sorry for the epic posting, but the aforementioned disk failures have
> prompted the investigation into a "bullet-proof" backup system (and
> quite rightly so).
>
Before your done you will be writing several epics. Companies spend
lots of money on backup systems. I was talking to friend that works
at a very large company. He said they produce 30-40,000 tapes per
week!!!!!
>
> Many many thanks for any assistance.
>
> -dale
>
> --
> Dale Roblin
Greg
-- Greg Freemyer The Norcross Group Forensics for the 21st Century -- Check the headers for your unsubscription address For additional commands send e-mail to suse-linux-e-help@suse.com Also check the archives at http://lists.suse.com Please read the FAQs: suse-linux-e-faq@suse.com
- Previous message: Alexandr Malusek: "Re: [SLE] 64-bit Matlab on SUSE Linux 9.3, Opteron CPU"
- In reply to: Dale L Roblin: "[SLE] Trying to make a near Perfect Backup System"
- Next in thread: François Pinard: "Re: [SLE] Trying to make a near Perfect Backup System"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
