Re: Announce: Linux-next (Or Andrew's dream :-))

On Tue, Feb 12, 2008 at 10:26:53AM -0800, Linus Torvalds wrote:
On Tue, 12 Feb 2008, Greg KH wrote:

I may be a bit defensive here, but I hope that all of the recent
kobject/kset/driver core changes have been done with the thought of
"what are we doing wrong".

.. but are we expecting it to be finished?

Nope, never.

That's the point.

Not it isn't. To quote you a number of years ago:
"Linux is evolution, not intelligent design"

We need to constantly be able to make these kinds of changes in order to
continue to remain relevant.

This whole "Linux-next" discussion so far has almost been predicated on
the whole assumption that this is an on-going concern. And it really
should NOT be.

If it's an on-going concern, we need to tackle *that* issue, not the issue
that cross-subsystem merges are hard. They simply seem to happen too much.

In other words, I'm not AT ALL interested in the merges we've already
done. That's over and done with, and we'll never ever do those merges
again. Who cares? I don't.

I'm purely and _only_ interested in the merges of the future. You don't
need to be defensive about the things that led up to this discussion, I'm
more hoping that we can aim at fixing the problem at the source, rather
than trying to work around it.

We simply shouldn't have all that many conflicts. We've had *way* too many
of them lately, and I think it's because people have felt it wasn't too
painful.

Oh, it's been painful at times, but they are, overall, very rare.

If you look at the rate of change we are currently running at, it's
amazing that we do not get _more_ of these kinds of problems.

Remember, we are currently clocking along at the steady rate of:
4000 lines added every day
1900 lines removed every day
1300 lines modified every day

And the rate of change in each major portion of the kernel (drivers,
arch, core, network, etc) is exactly proportional to the amount of the
kernel that that portion takes up (I have detailed numbers if people
really want to see them.)

Because of this rate of change, we are surviving, and evolving into what
is exactly needed at this point in time from a kernel. To try to quench
this change, and force us to stick with things that are no longer
optimal, is going to cause us to make the same mistakes that AIX and
Solaris and Vista have. To stop this, is to die :)

Put another way: back when we worked with just patches, we avoided renames
like hell, and we also tried to simply even re-architect the whole tree so
that you didn't have so many patch conflicts. One main reason as far as I
was concerned for things like per-directory Kconfig files and the whole
initcall() stuff was the fact that the old single Kconfig file and the old
crazy init/main.c file were total *nightmares* when it came to conflict
resolution.

So we split things up more, and we didn't do renames (or were very careful
about it). We avoided the things that caused pain.

I think we need to remember that: yes, we'll always have to have ways to
fix the pain that does happen, but even more importantly, we should strive
for models where it doesn't happen in the first place!

We have done very good at that. The problem comes in when one subsystem
depends on another one.

Like the kobject core, or driver core. Or network core. Changes there
trickle out into the other portions of the kernel that depend on them.
And we fix them up, and move on. And so far, we're doing this very
well, at a rate faster than any other software project ever has.

And simply avoiding cross-subsystem API changes unless there is a major
*MAJOR* reason for them is the obvious thing to do. Simply face the fact
that even in open source there are major reasons to stay with an old
interface even if it's not optimal.

I strongly disagree here. We lived with that kset/ktype crap for years,
and I finally broke down and cleaned it up, simplifying things, removing
code, making the kernel smaller, leaner, and easier for others to change
and use in the future. With your statement, such a change should have
never taken place as it what we had at the time was "not optimal", but
good enough to live with.

We absolutely MUST NOT have the mindset that "cross-subsystem conflicts
happen all the time".

They usually don't, by virtue of our current development model and how
we have the kernel structured.

But they do happen about once or twice a kernel release, just by virtue
of the way things need to happen. And since the last kernel had 10353
different commits, only 2 conflicts is a pretty good rate of merge
issues :)

And when they happen, so far _everyone_ involved instantly rushes to fix
them up. I say the fact that we are able to detect these problems, fix
them, and continue to move on and work, is a testament to the fact that
our current model is working _very_ well.

I think that you aren't seeing all of the "whacks" that Andrew is
constantly giving us that break his -mm tree with problems at times, so
when it gets to you, things are cleaned up.

The goal of -next is to take that "whacking" chore off of Andrew's
shoulders, and onto others, so Andrew can move off doing something like
development better.

My point is that the -next tree needs to work like -mm does today, a
policy of "merge broke, drop the tree" isn't going to help us subsystem
maintainers out in the end, we need a way to shim it up for a while as
merges and other communication happens, just like -mm provides, so that
when we get to the 2 week merge window, things are properly worked out
and what goes to you is all clean and purty.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: WTF: Stack Size 4k
    ... > hell are those of us who use and love the SCSI boards based on the ... > kernel with the 8k stack size by choosing to not allow that as an option ... I, for one, see great folly in upgrading ANY datacenter to Fedora Core 2 ...
    (Fedora)
  • Re: SMP Kernel
    ... since the memory is shared between all the cores the Kernel can be ... invoked by any core receiving an interrupt and thus executed by that ... Are there an independent scheduler per cpu? ... To unsubscribe from this list: ...
    (Linux-Kernel)
  • Re: Is C close to the machine?
    ... If a mismatch in workloads is noticed, the pipeline may be repartitioned, generally by moving one kernel across threads. ... Thus, if core 0 has more work than core 1, C may move to core 1, resulting in partition,, ... You're going to have to worry about copying the data at least once and probably twice (data gets created, copied into buffer, data gets copied from buffer, data gets used). ... The overhead of coroutining is not the state save, its the loss of locality of the caching and prediction structures. ...
    (comp.arch)
  • Re: Processor type kernel option for Core Duo (not Core 2)
    ... >> I've got a Centrino Core Duo laptop; ... >> can't work out which processor type option to use for the kernel. ... the CPU type selection within the kernel configuration (make ... > hardware and the software support. ...
    (comp.os.linux.setup)
  • Received signal 13. (no core)
    ... I am running ssh-3.1.0 from ssh.com on a linux box kernel version ... core)" error and sshd has to be restarted. ... Any ideas how to fix this? ...
    (comp.security.ssh)