Re: percpu memory setup and num_possible_cpus() vs. nr_cpu_ids

Hello, Jan.

Jan Beulich wrote:
Tejun,

this code of yours, as I learned the hard way during some Xen kernel work,
assumes that cpu_possible_map is not sparse. While on native x86 this is
certainly true, I wonder whether such a dependency should really exist in
architecture neutral code. Below, for reference, the places I needed to
change (4k allocator only). However, there are more instances of this in the
other allocators - those seem more problematic, as they directly drive input
to alloc_bootmem() (hence a lot of memory could be wasted, or allocation
could even fail, with a very sparse cpu_possible_map if one would simply
also use nr_cpu_ids here). In the remap (x86 - under the assumption that
if this change gets made, it should also be made in x86 code, albeit not
directly affected) allocator's case things may not be that bad, since the
memory gets freed again at the end of setup_pcpu_remap().

There's other related larger issue. It looks like possible but
offline configuration can be quite common with virtualization and
allocating for all possible cpus on boot can waste problematic amount
of space, so the plan is to allocate only for online cpus on boot and
allocate the rest as cpus come online. This will require audit of
percpu users and probably a new cpumask representing cpus which have
ever been up.

The issue you're describing should be solved when the above is
implemented, right?

Thanks.

--
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: [PATCH 5/5] cpumask: reduce cpumask_size
    ... allocate less bits safely. ... I cc'd you because I remembered you being involved in that libnuma issue ... number of cpus it reads from /sys/.../cpumap is more than the cpumask size ... (libnuma is OK on CONFIG_CPUMASK_OFFSTACK=y because the sysfs output is ...
    (Linux-Kernel)
  • Re: PROBLEM: Off-by-one bug on sun4u SMP
    ... then a chunk of kernel memory ... is wasted tor the phantom CPU #0. ... There are certain arrays which we have to allocate before we know ... Everything else is only allocated for cpus actually present. ...
    (Linux-Kernel)
  • Re: percpu memory setup and num_possible_cpus() vs. nr_cpu_ids
    ... allocating for all possible cpus on boot can waste problematic amount ... so the plan is to allocate only for online cpus on boot and ... allocate the rest as cpus come online. ...
    (Linux-Kernel)
  • Re: [PATCH] irq: sparse irqs, fix #2
    ... I haven't looked at callers of this, ... Some cpus do not strictly honor the set of cpus ... priority interrupt delivery mode. ... this cpu to the set of cpus I need to allocate a vector on. ...
    (Linux-Kernel)
  • Sybase allocation of CPUs
    ... We have a 24way S80 in which we run several Sybase servers and allocate some ... We are looking at moving to a p670 with fewer, more powerful CPUs. ... If you have received this electronic transmission in error, please notify the sender immediately by a "reply to sender only" message and destroy all electronic and hard copies of the communication, including attachments. ...
    (AIX-L)