Re: [accounting regression since rc1] scheduler updates

On Tue, 2007-08-21 at 13:36 +0200, Ingo Molnar wrote:
* Martin Schwidefsky <schwidefsky@xxxxxxxxxx> wrote:

Wouldnt it be more consistent if a virtual box would not show any
dependency on external load? (i.e. it would slow down all of its
internal functionality transparently, without exposing it via /proc.
The only way to observe that would be the TOD interfaces:
gettimeofday and real-time clock driven POSIX timers. Even
timer_list could be driven via virtual time - although that would
probably break user expectations, right?) Or would
accounting-in-virtual-time break user expectations too? (most of the
other hypervisors let guests account in virtual time.

No, imho it is less consistent if the virtual box shows virtual time.
If you look at the top output as a user and it shows that some process
used x% of cpu what does it tell you? [...]

it tells me something really important: that the virtual box's scheduler
gave this task as much CPU time as it could.

So? As far as accounting is concerned the user doesn't care one bit what
the scheduler decided. The user cares how much cpu was used. If you want
to know how much of the cputime available to the virtual box was used
for a process you just have to add/subtract the steal time. Your have
the complete picture what happened. You cannot get the complete picture
if you have do process accounting based on the TOD clock, you never know
how much cpu was actually spent for a process.

[...] With virtual cpus next to nothing, you have to normalize the
numbers with the %steal while the process was running. But even then
it still is not a good number because the %steal changes while a
process is running. The only good solution is to use virtual time for
all cputime values.

the steal percentage is really a concept one step higher in the
scheduling hierarchy. You should be running top (or an equivalent tool)
in the _hypervisor_ context if you want to know about how much time each
virtual box gets. 'mixing' that information with the 'internal' task
statistics of the virtual box is less consistent IMO and leads to the
loss of information. (with the 'mixing' method there's no way to tell
whether a task got all CPU time it asked for - and _that_ is which an
admin is interested in just as much as the time allocation between
virtual boxes.)

Not really. If I look at a process I want to know how much cpu it used.
Not virtual but real cpu. And we learned this the hard way, trying to do
accounting with numbers that have to get normalized by numbers from the
hypervisor is just plain broken.

so in say KVM you determine the steal percentage by looking at 'top'
output in the hypervisor context. (or looking at steal% in the internal
top output) Looking at 'top' in the guest tells you everything internal
to that guest, without mixing external scheduling information into it.

The information about accounting numbers that are internal to the guest
is 99.99% useless. You always have to take the external scheduling into
account. We choose to do the accounting in a way that does not require
additional steps to get to useful numbers.

--
blue skies,
Martin.

"Reality continues to ruin my life." - Calvin.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [patch] sched: accurate user accounting
    ... Currently we only do cpu accounting to userspace based on what is ... actually happening precisely on each tick. ... Both ad-hoc and `/proc/stat' (idle) ...
    (Linux-Kernel)
  • Re: Just in Time levelling
    ... versions thereof, Work in Process accounting, job costing, invoicing, links ... The software dosen't exist becasue it wouldn't sell, because the scheduling idea for a vast ... majority of the world is cost accounting based, ... has an elaborate Production ...
    (microsoft.public.project)
  • Re: Just in Time levelling
    ... versions thereof, Work in Process accounting, job costing, invoicing, links ... The software dosen't exist becasue it wouldn't sell, because the scheduling idea for a vast ... majority of the world is cost accounting based, ... has an elaborate Production ...
    (microsoft.public.project)
  • Re: [patch] sched: accurate user accounting
    ... scheduler first and check the results. ... to before RSDL and have tested it - find the patch below. ... Currently we only do cpu accounting to userspace based on what is ...
    (Linux-Kernel)
  • Re: [patch] sched: accurate user accounting
    ... we want to do this - and we should do this to the vanilla scheduler ... I've back-merged the patch to before RSDL ... Currently we only do cpu accounting to userspace based on what is ...
    (Linux-Kernel)