Re: [crash, bisected] Re: [PATCH 3/4] x86_64: Fold pda into per cpu area

Eric W. Biederman wrote:
Nope. It achieves that affect with a magic set of relocations instead
of linker magic.

Well, the code gcc generates for -fstack-protector emits a literal "%gs:40", so there's no relocations at all.

At present, the x86-64 only uses %gs-relative addressing to reach the pda, which
are always small positive offsets. It always accesses per-cpu data in a
two-step process of getting the base of per-cpu data, then offsetting to find
the particular variable.

x86-32 has no pda, and arranges %fs so that %fs:variable gets the percpu variant
of variable. The offsets are always quite large.

As a practical matter I like that approach (except for extra code size
of the offsets).

Yes, and there's no reason we couldn't do the same on 64-bit, aside from the stack-protector's use of %gs:40. There's no code-size cost in large offsets, since they're always 32-bits anyway (there's no short absolute addressing mode).

The powerpc guys tried using gcc-level thread-local storage, but it doesn't work
well. per-cpu data and per-thread data have different constraints, and its hard
to tell gcc about them. For example, if you have a section of preemptable code
in your function, it's hard to tell gcc not to cache a "thread-local" variable
across it, even though we could have switched CPUs in the meantime.

Yes, I completely agree with that. It doesn't mean however that we
can't keep gcc ignorant and generate the same code manually.

Yes, I see. I haven't looked at that specifically, but I think both Rusty and Andi have, and it gets tricky with modules and -ve kernel addresses, or something.

Well I was thinking threads switching on a cpu having the kinds of problems you
described when it was tried on ppc.

Uh, I think we're having a nomenclature imprecision here. Strictly speaking, the kernel doesn't have threads, only tasks and CPUs. We only care about per-cpu data, not per-task data, so the concern is not "threads switching on a CPU" but "CPUs switching on (under) a task". But I think we understand each other regardless ;)

If we manually generate %gs-relative references to percpu data, then it's no different to what we do with 32-bit, whether it be a specific symbol address or using the TLS relocations.

J
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [crash, bisected] Re: [PATCH 3/4] x86_64: Fold pda into per cpu area
    ... Given the previous description my hunch is that the bug is occurring ... If vmlinux is good and the compressed kernel is bad. ... At present, the x86-64 only uses %gs-relative addressing to reach the pda, which are always small positive offsets. ... It always accesses per-cpu data in a two-step process of getting the base of per-cpu data, then offsetting to find the particular variable. ...
    (Linux-Kernel)
  • Re: [crash, bisected] Re: [PATCH 3/4] x86_64: Fold pda into per cpu area
    ... It achieves that affect with a magic set of relocations instead ... two-step process of getting the base of per-cpu data, ... The offsets are always quite large. ... For x86_64 the kernel lives at a fixed virtual address. ...
    (Linux-Kernel)
  • Re: Bogus load average and cpu times on x86_64 SMP kernels
    ... Brian Gerst wrote: ... > up up bad data from the non-present cpus. ... (still pointing to the original per-cpu data, ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)