Re: Software RAID 5 SATA array crashed

From: Adar Dembo (adembo_at_gmail.com)
Date: 09/18/05

  • Next message: Seeker5528: "Re: Opinions -- When will Sid be usable again?"
    Date: Sun, 18 Sep 2005 03:30:31 -0700
    To: Alvin Oga <aoga@mail.Linux-Consulting.com>
    
    

    Alvin Oga wrote:

    >hi ya
    >
    >the only major piece of info that is missing:
    > a) cat /proc/mdstat
    >
    >
    adar@adar:~$ cat /proc/mdstat
    Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6]
    [raid10] [faulty]
    md0 : active raid5 sdb[0] sde[4] sdf[3] sdd[2] sdc[1]
          1172229120 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
    unused devices: <none>

    > b) what exactly was the commands you type vs the english
    > explainations
    >
    Here is the pertinent command history. I didn't know how to use mdadm,
    so much of it is probably meaningless. On another terminal, at some
    point, I issued "mdrun", which finally brought up the array and began
    the resync process. Unfortunately, I'm not sure when within this command
    history I used mdrun.

       89 sudo mdadm
       90 sudo mdadm -q
       91 sudo mdadm --help
       92 sudo mdadm --misc -Q
       93 sudo mdadm --misc -Q /dev/md0
       94 sudo mdadm --misc -Q /dev/sdb
       95 sudo mdadm --misc -Q /dev/sdb1
       96 sudo mdadm --misc -Q /dev/md0
       97 sudo mdadm --assemble -Q /dev/md0
       98 sudo mdadm --assemble /dev/md0
       99 sudo mdadm --assemble --scan /dev/md0
      100 cat /proc/mdstat
      101 sudo mdadm --misc -Q /dev/md0
      102 sudo mdadm --detail --misc -Q /dev/md0
      103 sudo mdadm --detail -Q /dev/md0
      104 sudo mdadm -Q --detail /dev/md0
      105 sudo mdadm -Q --detail /dev/md0 | less
      106 mount
      107 sudo mount -a
      108 cat /proc/mdstat
      109 sudo mdadm -Q --detail /dev/md0 | less
      110 fdisk /dev/sdc
      111 /sbin/fdisk /dev/sdc
      112 ls /dev/sd*
      113 sudo /sbin/fdisk /dev/sdc
      114 mdadm --help
      115 sudo mdadm -Q --assemble --scan -E /dev/md0 | less
      116 sudo mdadm --assemble --scan -E /dev/md0 | less
      117 sudo mdadm --misc --scan -E /dev/md0 | less
      118 sudo mdadm --misc --scan -E /dev/md0
      119 sudo mdadm --misc -S /dev/md0
      120 sudo mdadm --misc --scan -E /dev/md0
      121 sudo mdadm --misc --scan -E -D /dev/md0
      122 sudo mdadm --misc -E /dev/md0
      123 sudo mdadm -E /dev/md0
      124 sudo mdadm /dev/md0 -E
      125 sudo mdadm --misc /dev/md0 -E
      126 sudo mdadm -v --misc /dev/md0 -E
      127 sudo mdadm -v --misc /dev/md0
      128 sudo mdadm -v --misc --scan /dev/md0
      129 sudo mdadm /dev/md0
      130 sudo mdadm --detail /dev/md0
      131 sudo mdadm --detail /dev/md0 | less
      132 sudo mdadm --manage /dev/md0\
      133 sudo mdadm --manage /dev/md0
      134 sudo mdadm --manage /dev/md0 -r /dev/sdc1
      135 sudo mdadm --manage /dev/md0 -r /dev/sdc
      136 sudo mdadm --manage /dev/md0 -r /dev/sdc1
      137 sudo mdadm --manage /dev/md0 -f /dev/sdc1
      138 cat /proc/mdstat
      139 sudo mdadm --misc /dev/md0 -R
      140 cat /proc/mdstat
      141 dmesg | less
      142 sudo less /var/log/syslog
      143 sudo mdadm --misc /dev/md0 -q /dev/sdc
      144 sudo mdadm --misc /dev/md0 -Q /dev/sdc
      145 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine
      146 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine | less
      147 sudo mdadm --misc /dev/md0 -Q /dev/sda --examine | less
      148 sudo mdadm --misc /dev/md0 -Q /dev/sdb --examine | less
      149 sudo mdadm --misc /dev/md0 -Q /dev/sdc --examine | less
      150 sudo mdadm --misc /dev/md0 -Q /dev/sdd --examine | less
      151 sudo mdadm --misc /dev/md0 -Q /dev/sde --examine | less
      152 sudo mdadm --misc /dev/md0 -Q /dev/md0 --examine | less
      153 sudo mdadm --misc /dev/md0 -Q /dev/md0 --examine
      154 sudo mdadm --misc /dev/md0 --detail
      155 sudo mdadm --misc /dev/md0 --detail --scan
      156 less /etc/mdadm/mdadm.conf
      157 man mdadm.conf
      158 sudo mdadm --examine --scan
      159 sudo mdadm --examine\
      160 sudo mdadm --examine
      161 sudo mdadm --scan
      162 sudo mdadm --misc --scan
      163 sudo mdadm --misc /dev/md0 --scan
      164 sudo mdadm --misc /dev/md0 --scan --detail
      165 sudo mdadm --query /dev/md0
      166 sudo mdadm --query /dev/md0 --detail
      167 less /proc/partitions
      168 man badblocks
      169 shutdown -h now
      170 sudo shutdown -h now
      171 /bin/dmesg
      172 /bin/dmesg |less
      173 psaux
      174 dme
      175 top
      176 dmesg | less
      177 less /proc/mdstat
      178 less /dev/md0
      179 mount
      180 mdadm
      181 sudo mdadm
      182 sudo mdadm --help
      183 man mdadm
      184 sudo mdadm
      185 man mdadm
      186 man mdadm
      187 man mdadm
      188 ps aux
      189 dmesg | less
      190 cat /proc/mdstat
      191 sudo mdadm --misc /dev/md0
      192 sudo mdadm --misc /dev/md0 --query
      193 sudo mdadm --misc /dev/md0 --examine
      194 sudo mdadm --misc /dev/md0 --run
      195 cat /proc/mdstat
      196 sudo mdadm --assemble /dev/md0 --update=summaries
      197 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
      198 sudo mdadm --misc /dev/md0 -s
      199 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
      200 sudo mdadm --misc /dev/md0 -S
      201 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
      202 sudo mdadm --stop
      203 sudo mdadm --stop /dev/md0
      204 sudo mdadm --assemble /dev/md0 -v --update=summaries | less
      205 sudo mdadm --assemble /dev/md0 -v --update=summaries 2>&1 | less
      206 sudo mdadm --stop /dev/md0
      207 sudo mdadm --assemble /dev/md0 -v --update=summaries 2>&1 | less
      208 man mdadm
      209 man mdadm
      210 man mdadm
      211 man mdadm
      212 cat /proc/mdstat
      213 xfsinfo
      214 man xfs_repair
      215 xfs_repair -n
      216 sudo xfs_repair -n
      217 sudo xfs_repair -n /dev/md0
      218 man xfs_repair
      219 man xfs_checkl
      220 man xfs_check
      221 man xfs
      222 sudo xfs_check
      223 sudo xfs_check /dev/md0
      224 less /dev/md0
      225 less -f /dev/md0
      226 sudo less -f /dev/md0
      227 cat /proc/mdstat
      228 fdisk /dev/md0
      229 sudo fdisk /dev/md0
      230 sudo mdadm --start
      231 sudo mdadm --assembl /dev/md0 --run
      232 sudo mdadm --assemble /dev/md0 --run
      233 sudo mdadm --stop /dev/md0
      234 ls /dev/md0
      235 sudo less -f /dev/md0
      236 sudo mdadm --assemble /dev/md0 --run
      237 sudo mdadm --stop /dev/md0
      238 sudo mdadm --assemble /dev/md0 --verbose --run
      239 sudo mdadm --assemble /dev/md0 --verbose --verbose --run
      240 sudo mdadm --stop /dev/md0
      241 sudo mdadm --assemble /dev/md0 --verbose --verbose --run
      242 sudo mdadm --stop /dev/md0
      243 dmesg
      244 sudo shutdown -h now
      245 ps aux
      246 dmesg
      247 cat /proc/dmesg
      248 cat /proc/mdstat
      249 cat /proc/mdstat
      250 sudo fdisk /dev/md0
      251 y
      252 exit
      253 ps aux
      254 man xfs
      255 man xfs_admin
      256 man xfs_info
      257 man xfs_repair
      258 ps aux
      259 ls /usr/sbin/xfs_*
      260 man xfs_io
      261 man xfs_ncheck
      262 man xfs_rtcp
      263 man xfs_quota
      264 man xfs_mkfile
      265 man xfs_freeze
      266 man xfs_d
      267 man xfs_db
      268 man xfs_db
      269 xfs_db
      270 sudo xfs_db
      271 sudo xfs_db /dev/md0
      272 man xfs_check
      273 sudo xfs_check /dev/md0
      274 sudo xfs_check -v /dev/md0
      275 sudo xfs_check -vvv /dev/md0
      276 man xfs_check
      277 sudo xfs_check -b 1 /dev/md0
      278 sudo xfs_check -b 10 /dev/md0
      279 sudo xfs_check -i 10 /dev/md0
      280 apt-cache search xfs | less
      281 apt-cache show mondo
      282 apt-cache search xfs | less
      283 apt-cache show xfsdump
      284 apt-cache show xfsprogs
      285 man uuid
      286 apt-cache show libuuid1
      287 apt-cache search fs tools
      288 apt-cache search fs recovery
      289 apt-cache show testdisk
      290 ps aux
      291 dmesg
      292 exit
      293 dd
      294 man dd
      295 man dd
      296 dd if=/dev/md0 bs=512 count=1
      297 sudo dd if=/dev/md0 bs=512 count=1
      298 sudo dd if=/dev/md0 bs=512 count=1 | less
      299 man less
      300 sudo dd if=/dev/md0 bs=2048 count=1 | less
      301 sudo dd if=/dev/md0 bs=65536 count=1 | less
      302 sudo dd if=/dev/md0 | less
      303 sudo dd if=/dev/md0 | strings | less
      304 man od
      305 sudo xfs_repair -n -o /dev/md0
      306 sudo xfs_repair -n -o assume_xfs /dev/md0
      307 cat /proc/mdstat
      308 which xfs_repaid
      309 /sbin/xfs_repair -v
      310 /sbin/xfs_repair --version
      311 man xfs_repaid
      312 man xfs_repair
      313 dd if=/dev/md0 of=/dev/stdout |less
      314 sudo dd if=/dev/md0 of=/dev/stdout |less
      315 exit
      316 ps aux
      317 mail
      318 rm /home/adar/mbox
      319 ps aux
      320 cat /proc/mdstat
      321 ls
      322 sudo mount -a
      323 sudo xfs_check /dev/md0
      324 sudo xfs_repair -n /dev/md0

    >On Sat, 17 Sep 2005, Adar Dembo wrote:
    >
    >
    >
    >>Sep 11 23:46:57 localhost kernel: md: kicking non-fresh sdc1 from array!
    >>Sep 11 23:46:57 localhost kernel: md: unbind<sdc1>
    >>
    >>
    >
    >obviously bad
    >
    >
    >
    >>Note the message about sdc being non-fresh. Also note that the array is
    >>both DIRTY and DEGRADED.
    >>
    >>
    >
    >degraded means it works as if it was one giant disk, and if any of the
    >remaining disks dies, you are most likely to lose all data on all disks
    >unless you happen to know how to recover tdata when 2 of the 5 disks
    >are considered bad ... ( raid allows for 1 of 5 to go bad in your config )
    >
    >
    >
    >>I played around with mdadm but I could never get the array to start. All
    >>
    >>
    >
    >the exact commands you used and in what order makes all the difference
    >
    >the output of /proc/mdstat is what you should be looking at
    >
    >it'd be 100x simpler to reformat and restore from backups
    >unless you know why you did which commands in which order
    >
    > say based on some software-raid-repair-howto for which each
    > failure situation is probably different
    >
    >c ya
    >alvin
    >
    >
    >
    >

    -- 
    To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org 
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
    

  • Next message: Seeker5528: "Re: Opinions -- When will Sid be usable again?"

    Relevant Pages

    • [Full-disclosure] Undisclosed Sudo Vulnerability ?
      ... our proprietary LIDS detected some suspicious shell ... activity on an internal .mil machine i am in charged of. ... undisclosed bug in latest sudo. ... $ cat>blaat.uue<<'EH' ...
      (Full-Disclosure)
    • Undisclosed Sudo Vulnerability ?
      ... our proprietary LIDS detected some suspicious shell ... activity on an internal .mil machine i am in charged of. ... undisclosed bug in latest sudo. ... $ cat>blaat.uue<<'EH' ...
      (Bugtraq)
    • Re: dapper sudo
      ... So do 'sudo -s' to open a shell as root, and type directly cat ...
      (Ubuntu)