Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108


* Roman Zippel <zippel@xxxxxxxxxxxxxx> wrote:

On Thu, 14 Sep 2006, Ingo Molnar wrote:

i have one very fundamental question: why should we do this
source-intrusive method of adding tracepoints instead of the dynamic,
unintrusive (and thus zero-overhead) KProbes+SystemTap method?

Could you define "zero-overhead"?

zero overhead when not used: not a single instruction added to the
kernel codepath that is to be traced, anywhere. (which will be the case
on 99% of the systems)

Actual implementation aside having a core number of tracepoints is far
more portable than KProbes.

the key point is that we want _zero_ "static tracepoints". Firstly,
static tracepoints are fundamentally limited:

- they can only be added at the source code level

- modifying them requires a reboot which is not practical in a
production environment

- there can only be a limited set of them, while many problems need
finegrained tracepoints tailored to the problem at hand

- conditional tracepoints are typically either nonexistent or very
limited.

But besides the usability problems, the most important problem is that
static tracepoints add a _constant maintainance overhead_ to the kernel.
I'm talking from first hand experience: i wrote 'iotrace' (a static
tracer) in 1996 and have maintained it for many years, and even today
i'm maintaining a handful of tracepoints in the -rt kernel. I _dont_
want static tracepoints in the mainline kernel.

enter KProbes+SystemTap. It needs no changes at the source code level at
all, so no maintainance overhead to generic kernel code. Tracepoints can
be added and removed while the system is running. Trace actions and
filters can be added based on a scripting language, so tracing is as
dynamic as it gets.

(check out http://lwn.net/Articles/198557/ if you have an lwn
subscription - it's subscriber-only for a few weeks)

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108
    ... source-intrusive method of adding tracepoints instead of the dynamic, ... kernel codepath that is to be traced, ... have less overhead when it's actually used. ... static tracepoints are fundamentally limited: ...
    (Linux-Kernel)
  • Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108
    ... Static tracepoints can be achieved by various means, ... static tracepoints add a _constant maintainance overhead_ to the kernel. ... want static tracepoints in the mainline kernel. ...
    (Linux-Kernel)
  • Re: [PATCH 0/4] RFC: jump label - (tracepoint optimizations)
    ... tracepoints are implemented using a conditional. ... the overhead of this check is small, ... direct kernel functionality. ... Could you run your patches though scripts/checkpatch.pl and clean up any ...
    (Linux-Kernel)
  • Re: [patch 4/4] KVM-trace port to tracepoints
    ... Tracepoints allow dormat instrumentation, like the kernel markers, but also ... This patch depends on the "Tracepoints" patch. ... Is it a specific property of KVM-trace that causes this LOC blow-up? ... strings into the kernel code, which, to some, looks like debugging code. ...
    (Linux-Kernel)
  • Re: [PATCH] ftrace: add an fsync tracer
    ... markers, or by tracepoints converted to markers, or by tracepoints ... The tracepoint declaration is linked to the kernel code that is meant to ... I never said it would be harder to generate the trace data. ...
    (Linux-Kernel)