Re: malfunctioning applications under limited stack size



Recently, I observed several applications to malfunction when
stack size is limited to anything fewer then exactly 2 GB ...

What can be done to debug this?

Look in /var/log/messages for signs of the OutOfMemory (oom) killer.
Enable coredumps (ulimit -c; also set /proc/sys/kernel/core_uses_pid
and/or /proc/sys/kernel/core_pattern) then inspect the results with
"readelf --all <corefile>" to show the address space at the crash.
"cat /proc/<pid>/maps" to see what the address space looks like
when the process is running.
Track the address space using "strace -e trace=mmap,munmap,mprotect".
(Move the compiled binary executable somewhere else, then create an
executable shell script, using the original name, that invokes strace
on the moved binary.)
In the worst case, install a signal handler for SIGSEGV and SIGBUS,
where the handler does "cat /proc/self/maps" by itself without
using system() or stdio: open, {read, write}*, abort.

--
.



Relevant Pages

  • Re: xmalloc
    ... OOM (out-of-memory) condition. ... The 'static solution' may cause a 'stack' overflow. ...
    (comp.lang.c)
  • Re: Andrea VM changes
    ... >> And I don't see how you can avoid oom killing to ever happen if the apps ... >> recurse on the stack and growsdown some hundred megs. ... eventually fallback to sigkill. ...
    (Linux-Kernel)
  • Large memory application exhuasts buffers during write.
    ... We have a large memory application which is being killed by the OOM ... This is a 2.4 based kernel with many of the redhat patches applied. ... the OOM killer kicks in and kills the job. ...
    (Linux-Kernel)
  • Re: Large memory application exhuasts buffers during write.
    ... > We have a large memory application which is being killed by the OOM ... Once swap has been heavily utilized, ... > the OOM killer kicks in and kills the job. ...
    (Linux-Kernel)
  • Re: [RFC] [PATCH] Cgroup based OOM killer controller
    ... killer kill a task hung in D state any better than userspace. ... mask, that may be not enough though, but it tries more than userspace. ... memory reserves with TIF_MEMDIE, it doesn't drop any unkillable state. ... It's a way to prevent invoking the kernel oom killer by allowing userspace ...
    (Linux-Kernel)