Re: Bandwidth Allocations under CFQ I/O Scheduler

On Wed, Oct 18 2006, Jakob Oestergaard wrote:
On Wed, Oct 18, 2006 at 11:40:56AM +0200, Arjan van de Ven wrote:
...
Hi,

I can see that that makes it simple, but.. what would it MEAN? Eg what
would a system administrator use it for?

For example, I could allocate "at least 100 iops/sec" for my database.
The VMWare can take whatever is left.

I have no idea how much bandwidth my database needs... But I have a
rough idea about how many I/O operations it does for a given operation.
And if I don't, strace can tell me pretty quick :)

That's crazy. So you want a user of this to strace and write a script
parsing strace output to tell you possibly how many iops/sec you need?

It then no longer means "my mp3
player is guaranteed to get the streaming mp3 from the disk at this
bitrate" or something like that...

In a sense you are right.

You cannot be certain that the mp3 player will get a specific bandwidth.
The mp3 player will be accessing the underlying storage through a
filesystem, which again means that accessing a file sequentially *will*
cause non-sequential I/O on the underlying device(s).

If you wanted to guarantee any specific bandwidth, you would somehow
assume that you had an infinite (or at least very very high) number of
seeks at your disposal. Or that seeks were free... In any other
scenario, the total "capacity" of your underlying storage, the maximum
amount of bandwidth (including non-free seeks) available, would vary
depending on how it is currently used (how many seeks are issued) by all
the clients.

So, what I'm arguing is; you will not want to specify a fixed sequential
bandwidth for your mp3 player.

What you want to do is this: Allocate 5 iops/sec for your mp3 player
because either a quick calculation - or - experience has shown that this
is enough for it to keep its buffer from depleting at all times.

But that is the only number that makes sense. To give some sort of soft
QOS for bandwidth, you need the file given so the kernel can bring in
the meta data (to avoid those seeks) and see how the file is laid out.
For the mp3 case, you should not even need to ask the user anything. The
player app knows exactly how much bandwidth it needs and what kind of
latency, if can tell from the bitrate of the media. What you are arguing
for is doing trial and error with a magic iops/sec metric that is both
hard to understand and impossible to quantify.

Describing iops/sec for your mp3 player is at least as simple as
sequential bitrate. The difference is, that you can implement iops/sec
allocation whereas you cannot implement bitrate allocation (in a
meaningful way at least) :)


so my question to you is: can you
describe what it'd bring the admin to put such an allocation in place?

Limiting on iops/sec rather than bandwidth, is simply accepting that
bandwidth does not make sense (because you cannot know how much of it
you have and therefore you cannot slice up your total capacity), and,
realizing that bandwidth in the scenarios where limiting is interesting
is in reality bound by seeks rather than sequential on-disk throughput.

I don't understand your arguments, to be honest. If you can tell the
iops/sec rate for a given workload, you can certainly see the bandwidth
as well. Both iops/sec and bandwidth will vary wildly depending on the
workload(s) on the disk.

If we find that it can be a good approach.. but if not, I'm less certain
this'll be used..

I can only see a problem with specifying iops/sec in the one scenario
where you have multiple sequential readers or writers, and you want to
distribute bandwidth between them.

If you only have one app doing io, you don't need QOS. The thing is, you
always have competing apps. Even with only one user space app running,
the kernel may still generate io for you.

In all other scenarios, I believe iops/sec is by far a superios way of
describing the ressource allocation. For two reasons:
1) It describes what the hardware provides
2) By describing a concept based on the real world it may actually be
possible to implement so that it works as intended

Same arguments. You can't universally state that this disk gives you
80MiB/sec, and you can't universally state that this disk gives you 1000
iops/sec. You need to also define the conditions for when it can provide
this performance. So if you instead say this disk does 80MiB/sec if read
with at least 8KiB blocks from lba 0 to 50000 sequentially. Or you can
state the same with iops/sec.

--
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: Bandwidth Allocations under CFQ I/O Scheduler
    ... I could allocate "at least 100 iops/sec" for my database. ... You cannot be certain that the mp3 player will get a specific bandwidth. ... realizing that bandwidth in the scenarios where limiting is interesting ...
    (Linux-Kernel)
  • Re: Bandwidth Allocations under CFQ I/O Scheduler
    ... bandwidth. ... The iops/sec metric doesn't help you. ... It's an entirely diffent thing from the mp3 player. ... least foo amount of the disk, but don't let it suck everything. ...
    (Linux-Kernel)
  • Re: Bandwidth Allocations under CFQ I/O Scheduler
    ... So you want a user of this to strace and write a script ... bandwidth for your mp3 player. ... What you want to do is this: Allocate 5 iops/sec for your mp3 player ... available disk performance... ...
    (Linux-Kernel)
  • Re: [RFC][PATCH 0/8] Use EDF to throttle RT task groups
    ... to handle generic periods, then extend the push-pull logic (I think you ... are referring to it with load-balancing) to fully support it, ... or how the allocation would ... since the current interface specifies bandwidth in a single-cpu ...
    (Linux-Kernel)
  • Re: Audio quality degradation over FM transmission
    ... >>>the bandwidth is a bit lower, as the FM signals bandwidth is about 15Khz, ... the MP3 player doesn't have that quality. ... > he was talking about the MP3 player, not the transmitter. ...
    (sci.electronics.design)
Loading