remove/replace a harddisk in a unsynced raid1?

From: peter pilsl (pilsl_at_goldfisch.at)
Date: 09/22/04 

Date: Wed, 22 Sep 2004 16:28:09 +0200

One disk on a server is broken. This disk is part of several raids and
while the server and the raids start up well after about 20minutes the
disk starts to behave bad and finally the server crashes.
For I didnt have the time to exchange the disk and rebuild the raids
today I just wanted to remove the harddisk from all raids.
And I didnt make it.

My first idea was:

run raidhotremove and mark the disk as "failed-disk" in raidtab.
This did not work, cause raidhotremove told something like "disk is busy"
I tried to unmount the filesystem and tried again: it did not work.
Then I stopped and restartet the raid : the disk marked as "failed disk"
was taken into the raid again.
I could not believe it.

Finally I ended up with just leaving the raids down and telling the
people in my neck that they cant access the data on this filesystems for
now.

This is not what I have a raid for. Why was this so? My suspect is that
  after the hardreset and the boot all raids were in process of resync
and the raids in question were all "delayed" cause other raids were
resynced before. Maybe one cannot hotremove a disk of a raid that is out
of sync?? I couldnt wait until the sync was finished cause the server
crashed again before.

Additional questions:
* what would have happened if I would just have unplugged the disk and
rebooted. I didnt want to try cause about 20people were standing behind
me and pushing to access their emails and it would have taken a bit
longer cause the broken disk is unfortunately hda, so I would have had
to rebuild the bootsector on a different disk and this is nothing to be
done in 10minutes when you have a lot of stress.

* what is the best way to replace the disk tommorow? My standard way is
to replace the disk, boot with a live-cd, partitionize the new disk.
create a raidtab where the new disk is "raid-disk" and the old disk is
"failed-disk". Then I fire up the new raid, mount the old disk and copy
all files to the new raid. After that I mark the old disk as "raid-disk"
as well and reboot and wait until syncronizing is done.
I would appretiate a way where I dont need to copy, but just put in the
new disk and have sync doing its way. I tried that once with the result
that somehow the new empty disk overwrote the old datadisk which
resulted in painfull dataloss.

thnx alot,
peter 

-- 
http://www2.goldfisch.at/know_list
http://leblogsportif.sportnation.at

Relevant Pages

  • Re: remove/replace a harddisk in a unsynced raid1?
    ... server has the broken harddisk replaced, is up, running, all raids ... I tried to remove the bad disk (that was working fine the ... disk, started the server with knoppix-cd, started the /-raid (which is ... Now how I can I make the spare disk also an active mirror? ...
    (comp.os.linux.misc)
  • vinum status in daily log
    ... Got a couple of server with vinum mirrored volumes. ... disk or the other tends to get out of sync and is labeled as stale. ... the daily report which is mailed to me. ...
    (freebsd-questions)
  • Re: Strategies for keeping 2 system disks in sync
    ... If you want to keep both in sync down to ... Look at the UAF on the server, ... The workstation has a slow motion IDE disk drive. ...
    (comp.os.vms)
  • Re: Creating Soft-RAID of existing installation
    ... > second harddisk and created partitions of the same size as my running ... form your old to your new disk. ... the now empty partitions to the raids. ...
    (Fedora)
  • Re: [OT] backup using tapes or IDE disks
    ... currently have somewhere between 45 and 50Tb of disk, ... Home directories are saved to tape (around 2Tb for a full ... raids are saved to other raids. ...
    (alt.games.mame)