Re: readahead logic and I/O errors


On Wed, 15 Feb 2006, Michael Tokarev wrote:

The thing is: I just fired a cdrom drive in my PC.
It was a good device, and now it's dead.
And the reason is the readahead logic, plus an unreadable
(damaged, scratched) CD-rom.

By default, linux enables read-ahead for all block devices
(I can't be certain about "all", but it's true for hard
disks, cd-roms and floppies at least). And when a read
request comes to the device, according to readahead logic,
linux tries to get more data from the drive than requested.

Now, one of blocks requested to be read can't be read due
to whatever condition. It happens on removable media, it's
a normal (sort of, anyway) condition.

Looks like I had many unreadable blocks on that CD. And,
instead of obvious (in my view anyway) thing to do, namely,
to STOP readahead operation and return only whatever data
which was requested by an application (it was just `cp')
after FIRST I/O error, linux continued trying reading-ahead,
discovering more and more failed blocks, as dmesg said.

And finally, after about 100 blocks failed in a row, the
drive fired.

I wasn't able to stop the process - `cp' was in D state
sitting in one read() syscall thus unkillable. Umount
didn't work (obviously). Only one option left was to
reboot/poweroff the system, which I was finally forced
to do.. but too late.

I don't understand the logic in ll_rw_blk.c (the file
is quite large), but my guessing is that initially,
the read request was one for several blocks, but when
it failed, linux continued trying to get next block,
next-to-next block and so on - hence all consequtive
failed block numbers in dmesg, ie, it split original
large request into several smaller chunks, failing each
in turn.

Even if I'm wrong here and it originally made "short"
reads, the question remains:

Why linux tries to continue readahead operation after
I/O errors, instead of just dropping the reads, and
repeat them WHEN ACTUALLY NEEDED? `cp' will stop after
first failed read(), so only one bad sector will be tried,
instead of numerous, finally firing the drive?

When setting readahead to 0, the same CD can be read up
to the first error, and will fail correctly.

The same happens with floppy as well - exactly the same
thing. I especially damaged a floppy disk (I still have
several here), just to verify - after ~15 minutes of waiting
while it will finish it's readahead nonsense, I become bored
and finally rebooted the system.

So, to sum: current linux behaviour (similar since 2.4 -- at
least I remember seeing something like that, repeated attempts
to read something from a bad drive, -- up to current 2.6.15)
is that it's trivial to kill the system just by inserting a
cd-rom or floppy with some unreadable sectors. Worse, it's
possible to FIRE the system this way (with certain drives
anyway), as just happened here (I understand, maybe, after
some more waiting, when it will try all 128 or 256 more sectors,
it WILL finally do something useful again and stop retrying,
but eg on this my drive, each read attempt took about 30 sec,
retrying several times by its own, so total time is umm..
quite large, and there's nothing one can do with it but
to poweroff the PC).

From my point of view, it's a serious bug (even if not counting
my dead CD drive).

Thanks.

/mjt

Aside from the obvious read-ahead bug you discovered, have you
tried your CD drive after the reboot? It is not possible to
kill those things with software, even if attempting to write
with too much infrared LED drive. There is nothing except for
the removable disc to get hurt. Even the "head" isn't in contact.
It's just some infrared light, focused with a voice-coil, that
moves on a fixed platform.

Of course you can make many cow-flops from discs, but can you
describe how you think the drive got "fired" as you say?

Cheers,
*** Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.66 BogoMips).
Warning : 98.36% of all statistics are fiction.
_


****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@xxxxxxxxxxxx - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/