Re: automated test? (was Re: Linux 2.6.17.7)

On Tue, 25 Jul 2006, Matthias Andree wrote:

On Tue, 25 Jul 2006, Arjan van de Ven wrote:

well you can do such a thing withing statistical bounds; however... if
the patch already is in -git (as is -stable policy normally).. it should
have been found there already...

The sad facts I learned from Debian bug #212762 (not kernel related) that
culminated in CVE-2005-2335 (remote root exploit against older
fetchmail) and from various qmail bugs Guninski discovered:

- a bug need not necessarily be found soon after introduction

- a bug report may not convey the hint "look at this NOW, the shit
already hit the fan"
(sorry, I meant to write: look NOW, it's urgent and important)

- an automated test to catch non-trivial mistakes is non-trivial in
itself, and - what I've seen with another project I was involved with,
and more often than I found amusing - is that the test itself can be
buggy causing bogus results.

That doesn't mean I object to automated tests, but "it should have been
found by now" (because the source is open, someone could have tested it,
whatever) just doesn't work.

what I was intending with my origional question was a series of simple 'does it compile' tests that try all the config options that are affected by the patchset in question. the purpose being to catch simple errors like the one here where the patch was diffed against the wrong tree and the result doesn't compile

i.e. if the change is the the e1000 driver, try compiling a kernel with it on, off, and as a module

obviously such a test can't be done on the huge -rc patches (they would approach an exhaustive test of all config permutations), but for -stable patches (or better still the indivdual patches that form a -stable release)

so the first part of the question boils down to

1. Given a patch that modifies file X is it possible to know what config options could be affected?

and the second part would be

2. would it make sense for the LTP or one of the big compile farms to do a series of compiles prior to the release of a -stable kernel to catch mistakes like this?

David Lang
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: stuck with 2.6.23.14 on x86_64
    ... fix the splice bug I've had to apply by hand the patch. ... the lastest that I can compile is 2.6.23.14; ... SCSI controller: ... If patches, what base tree are they applied to? ...
    (Linux-Kernel)
  • Re: stuck with 2.6.23.14 on x86_64
    ... fix the splice bug I've had to apply by hand the patch. ... the lastest that I can compile is 2.6.23.14; ... SCSI controller: ... If patches, what base tree are they applied to? ...
    (Linux-Kernel)
  • Re: stuck with 2.6.23.14 on x86_64
    ... fix the splice bug I've had to apply by hand the patch. ... the lastest that I can compile is 2.6.23.14; ... SCSI controller: ... If patches, what base tree are they applied to? ...
    (Linux-Kernel)
  • Re: Pedants
    ... it was a genuine bug. ... no reason to assume that gcc-specific headers will compile with any ... If some gcc-specific header didn't compile with gcc, ... general container library for instance. ...
    (comp.lang.c)
  • Re: Mystery Access VB bug
    ... one of the things that triggers this corruption is editing the module ... Your code runs, and hits a bug. ... the text version of the code visible in your editing window, the Compile ... > would get if I had a missing reference. ...
    (microsoft.public.access.formscoding)