Re: Fork bombing a Linux machine as a non-root user

• Previous message: Scot L. Harris: "Re: no eth0 after updating to new kernel (2.6.10-1.770_FC3)"
• In reply to: David Curry: "Re: Fork bombing a Linux machine as a non-root user"
• Next in thread: Scot L. Harris: "Re: Fork bombing a Linux machine as a non-root user"
• Reply: Scot L. Harris: "Re: Fork bombing a Linux machine as a non-root user"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Mon, 21 Mar 2005 10:23:49 -0600
To: For users of Fedora Core releases <fedora-list@redhat.com>

David Curry wrote:
> Discussion in this thread frequently reflected an unwarranted,
> underlying assumption. Namely, that linux/unix experts are intalling
> the OS, know how the system will be used, and act immediately after
> installation to reset default installation resource limits to
> appropriate levels. It is obvious to me from thread discussion that that
> assumption is invalid. Rather the discussion suggests many/some
> respondents had the author's (and my) grasp of the resource limit
> issue. It is past time for the linux enthusiats touting linux as more
> secure than windows to get acquainted with the result of parsing the
> word assume into ass-u-me!

While approach of applying the sane limits out-of-the-box is bettern
than the current approach of not having any limits at all, it is still
not the right approach.

The correct approach would be to have kernel that is able to handle
abuses correctly. Basically, what happens during fork bombing or any
other kind of attack on the resources, is that one process is able to
block all other processes on the system from gaining access to one of
systems limited resources (CPU time, physical memory, and so on). If
you took operating system classes at university, this was called
starvation of resources. An ideal operating system should never allow
for process to starve. No matter how high the load on the system is,
process should get its share of resources.

Now, for the operating system to handle this correctly, it would need to
keep some statistics on multiple levels of what is going on the system
for each process and group of processes (at least grouped by owner and
again by common parrent, and possibly by other criteria) for each of the
system's limited physical resources. That way, OS can penalize the
abusive process, and protect other processes from starvation. In
laboratory, you'll see performance degradation (lower benchmark
results), of course. However, in real world (you know, the place where
we run production servers), the performance degradation will be minimal,
and the system will run way more stable and smoother. Actually, the
system will be able to run multiple services without danger of one
service affecting other services. I find this way more important to
have in real world than meaningless benchmark numbers from lab testings.

An example from real world where I regulary see Linux boxes going down
is not fork bombing. It is "swap bombing". You just make a Linux box
swap a lot, and it becomes dead (to the point where your SSH sessions
will get disconnected on network timeouts). I have one box in my
basement where simply typing "yum update" will get it down. The very
same issue as with fork bombing. Linux does not protect user space
processes from each other. And unlike fork bombing, setting ulimits is
not going to save you here. Linux ignores ulimit for maximum number of
pages a single process can have in physical memory (unless it is mmaped
region with specific value of madvise applied to it, as per ulimit man
page).

Now, what should (ideally) kernel do in this case? It should detect
that there is a single process attempting to monopolize memory for
itself. It should dynamically limit the number of physical memory pages
that process can have, and lower that process priority for accessing
swap. The affected process (the one that went on memory rampage) will
get slowed down by this. How much? Not much, it was going to crawl
anyhow because it will spend most of its time waiting for slow disk.
However, all other processes on the system will be spared from really
not needed swapping. The similar protection can be applied to ther
resources, such as CPU time. Small penalty for resource intensive
process is (IMO) acceptable price to pay to get rock stable system.

The "yum update" problem that crashes the system was on an old MMX box
with 96 megs of RAM. Before anybody says "don't run yum on such an old
box with so 'little' RAM", I recently made a small programing mistake
that brought down P4 box with 256 megs of RAM basically the same way
"yum update" kills that old MMX box. Basically 6 lines of code:

    open() - create an empty file
    pwrite() - write a single byte at 1 gig offset
    mmap() - map whole file (1 gig) into memory as shared
    madvise() - lie to kernel we'll access the file "randomly"
    memset() - entire 1 gig region
    munmap() & close()

So, nothing fancy. The usuall sequence of calls that you can find in so
many programs (well, apart from calling memset on mmaped region right
away). I think I told kernel access will be random in madvise (might be
some other hint, don't remember anymore what was exactly my stupid
mistake at this point). This is different than mallocing entire virtual
memory kind of attack. This does not consume any swap space, all other
programs will have more than enough virtual memory to continue running
(but Linux is going to allow you to steal all physical memory, and you
are going . Theoretically, I'm not attacking any system resources here.
  Practically, I did. And I did it so efficiently, that all I could do
is unplug the system from the power, and reboot.

The system is going to die somewhere at memset part. Unless your
physical memory is in gigabytes range, so that whole region (or at least
most of it) can be fitted into the memory. In that case, either start
several of those, or if you are running 64-bit processor, simply change
steps 2-4 to use 1 terabyte region. Most systems don't have anywhere
near terabyte of memory, for sure. The system will recover from this.
Eventually. But while we are in memset, it will be almost frozen for
anything else. So frozen that even if sysadmin was logged in on the
console, he can't do anything to stop this. He can just watch disk
light constantly on as system swaps as crazy. Stupid kernel...

Put this in loop, insert fork() somewhere, do I really need to draw
everything? And all of this possible becase Linux kernel does not care
what is going on the system (so that is can be the "fastest" system when
benchmarked)... The fastest to go down, if you ask me...

-- 
Aleksandar Milivojevic <amilivojevic@pbl.ca>    Pollard Banknote Limited
Systems Administrator                           1499 Buffalo Place
Tel: (204) 474-2323 ext 276                     Winnipeg, MB  R3T 1L7
-- 
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list

• Previous message: Scot L. Harris: "Re: no eth0 after updating to new kernel (2.6.10-1.770_FC3)"
• In reply to: David Curry: "Re: Fork bombing a Linux machine as a non-root user"
• Next in thread: Scot L. Harris: "Re: Fork bombing a Linux machine as a non-root user"
• Reply: Scot L. Harris: "Re: Fork bombing a Linux machine as a non-root user"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]