Re: CPU load

On 12/02/07, Vassili Karpov <av1474@xxxxxxxx> wrote:
Hello,

How does the kernel calculates the value it places in `/proc/stat' at
4th position (i.e. "idle: twiddling thumbs")?

For background information as to why this question arose in the first
place read on.

While writing the code dealing with video acquisition/processing at
work noticed that what top(1) (and every other tool that uses
`/proc/stat' or `/proc/uptime') shows some very strange results.

Top claimed that the system running one version of the code[A] is
idling more often than the code[B] doing the same thing but more
cleverly. After some head scratching one of my colleagues suggested a
simple test that was implemented in a few minutes.

The test consisted of a counter that incremented in an endless loop
also after certain period of time had elapsed it printed the value of
the counter. Running this test (with priority set to the lowest
possible level) with code[A] and code[B] confirmed that code[B] is
indeed faster than code[A], in a sense that the test made more forward
progress while code[B] is running.

Hard-coding some things (i.e. the value of the counter after counting
for the duration of one period on completely idle system) we extended
the test to show the percentage of CPU that was utilized. This never
matched the value that top presented us with.

Later small kernel module was developed that tried to time how much
time is spent in the idle handler inside the kernel and exported this
information to the user-space. The results were consistent with our
expectations and the output of the test utility.

Two more points.

a. In the past (again video processing context) i have witnessed
`/proc/stat' claiming that CPU utilization is 0% for, say, 20
seconds followed by 5 seconds of 30% load, and then the cycle
repeated. According to the methods outlined above the load is
always at 30%.

b. In my personal experience difference between `/proc/stat' and
"reality" can easily reach 40% (think i saw even more than that)

The module and graphical application that uses it, along with some
short README and a link to Usenet article dealing with the same
subject is available at:
http://www.boblycat.org/~malc/apc

The kernel looks at what is using cpu _only_ during the timer
interrupt. Which means if your HZ is 1000 it looks at what is running
at precisely the moment those 1000 timer ticks occur. It is
theoretically possible using this measurement system to use >99% cpu
and record 0 usage if you time your cpu usage properly. It gets even
more inaccurate at lower HZ values for the same reason.

--
-ck
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: questions on NAPI processing latency and dropped network packets
    ... We require kernel mods for our userspace to run, and I doubt I'd be able to get the time to port all the changes forward to git head. ... It's conceivable that userspace is starving the kernel, but we have do about 45% idle on one cpu, and 7-10% idle on the other. ... Any idea of the size of sctp hash size you have? ...
    (Linux-Kernel)
  • CPU load
    ... How does the kernel calculates the value it places in `/proc/stat' at ... For background information as to why this question arose in the first ... for the duration of one period on completely idle system) we extended ... According to the methods outlined above the load is ...
    (Linux-Kernel)
  • Re: questions on NAPI processing latency and dropped network packets
    ... We require kernel mods for our userspace to run, and I doubt I'd be able to get the time to port all the changes forward to git head. ... It's conceivable that userspace is starving the kernel, but we have do about 45% idle on one cpu, and 7-10% idle on the other. ... Based on profiling and instrumentation it seems like the cost of sctp_endpoint_lookup_assocmore than triples, which means that the amount of time that bottom halves are disabled in that function also triples. ...
    (Linux-Kernel)
  • Re: high cpu temperature with linux (on pavilion ze4268)
    ... > can change my processor speed with cpufreq (powernow-k7 module). ... always using a tiny bit of the CPU so that it's never idle. ... processor is disabled either in the BIOS or the kernel, ...
    (comp.os.linux.misc)
  • Re: Dual core processors
    ... processes, one for each CPU: ... my kernel either, and it is showing both cores in top (Core 2 Duo ... cpu0 only 45-50% idle at this time. ... hyperthreadin after reading Bill's response and checked sysctl to find ...
    (freebsd-questions)
Loading