Re: CPU load due to IP networking

From: Craig Partridge (craigp_at_std.com)
Date: 06/23/05


Date: Thu, 23 Jun 2005 11:54:12 +0000 (UTC)


"jbl" <levinjb@gmail.com> writes:

>I've been looking for some information on the web, in white papers and
>even on Usenet, but if it exists I can't seem to put together a search
>that finds it. I'm asking here if anyone knows of something along
>these lines and can point me to it.

There was a lot of ink spilled on this topic in the 1990s -- a bunch of
good papers. Unfortunately, the results aren't as clear cut as you'd
like.

It also gets much tougher if you try to make a linux box act like
a router (which it sounds as if you're interested in doing).

But let's be simple minded and say you're running as a host. No OSPF
or BGP -- just TCP or UDP in and out to a default route.

Then assuming a reasonably good TCP implementation and a poor quality buffering
system, you'll expend about 3 or 4 instructions (or, 3 or 4 memory accesses,
whichever is slower) per word of data sent via TCP, NOT INCLUDING DEVICE
DRIVER/INTERRUPT COSTS. Note that a high quality TCP with a good
buffering system/zero copy memory will be under half that cost, while
a poor TCP can easily push it higher.

Device driver and interrupt costs vary so widely based on your interface
hardware that it is hard to estimate. (In the old days, a factor of two
difference was common using different cards in the same box with the same
software -- haven't done tests on interfaces in ages, so can't say today).

UDP costs are more variable and can be higher or lower than TCP depending
on buffering design. Most UDP implementations are not optimized very
much.

In many ways you're probably best off running a test of TCP tests
where the application writes at different sizes and plotting size vs.
CPU costs. Make sure to have other applications running (so TCP has
at least some competition for memory) and have several other TCP
connections running with at least low levels of traffic (so your
TCB cache gets some work).

Craig



Relevant Pages

  • Re: Interplatform (interprocess, interlanguage) communication
    ... a minor issue with TCP for IPC though is that sometimes the buffering does something very annoying: ... no matter how long one waits, TCP will not send the data until a certain amount has been written to the socket. ... UDP also has some merit, but the big annoying hassle of having to pack ones' messages into UDP datagrams. ...
    (comp.lang.java.programmer)
  • NFS problem with recent 2.6 kernels (also serial console weirdness)
    ... 100000 2 tcp 111 portmapper ... 100000 2 udp 111 portmapper ... mounted filesystem with ordered data mode. ... Mounted root (ext3 filesystem) readonly. ...
    (Linux-Kernel)
  • Solaris 9 <---> linux (2.6.8) NFS file locking problem?
    ... to the same file placed on nfs filesystem. ... 100000 4 tcp 111 portmapper ... 100000 4 udp 111 portmapper ... 100021 1 udp 4045 nlockmgr ...
    (SunManagers)
  • Urgent help with Secure NFS.
    ... have that option - I'm just attempting to tunnel all NFS traffic to the ... 100000 4 tcp 111 rpcbind ... 100000 4 udp 111 rpcbind ... 100021 1 tcp 49153 nlockmgr ...
    (SSH)
  • Re: nfs error
    ... kernel: nfs: server ... So if your system uses ypbind be sure that is working properly before ... 100000 2 tcp 111 portmapper ... 100000 2 udp 111 portmapper ...
    (comp.sys.sun.admin)