Re: [1/1] Block device throttling [Re: Distributed storage.]

On Tuesday 28 August 2007 02:35, Evgeniy Polyakov wrote:
On Mon, Aug 27, 2007 at 02:57:37PM -0700, Daniel Phillips
(phillips@xxxxxxxxx) wrote:
Say Evgeniy, something I was curious about but forgot to ask you
earlier...

On Wednesday 08 August 2007 03:17, Evgeniy Polyakov wrote:
...All oerations are not atomic, since we do not care about
precise number of bios, but a fact, that we are close or close
enough to the limit.
... in bio->endio
+ q->bio_queued--;

In your proposed patch, what prevents the race:

cpu1 cpu2

read q->bio_queued
q->bio_queued--
write q->bio_queued - 1
Whoops! We leaked a throttle count.

We do not care about one cpu being able to increase its counter
higher than the limit, such inaccuracy (maximum bios in flight thus
can be more than limit, difference is equal to the number of CPUs -
1) is a price for removing atomic operation. I thought I pointed it
in the original description, but might forget, that if it will be an
issue, that atomic operations can be introduced there. Any
uber-precise measurements in the case when we are close to the edge
will not give us any benefit at all, since were are already in the
grey area.

This is not just inaccurate, it is suicide. Keep leaking throttle
counts and eventually all of them will be gone. No more IO
on that block device!

Another possibility is to create a queue/device pointer in the bio
structure to hold original device and then in its backing dev
structure add a callback to recalculate the limit, but it increases
the size of the bio. Do we need this?

Different issue. Yes, I think we need a nice simple approach like
that, and prove it is stable before worrying about the size cost.

Regards,

Daniel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages