Delayed TCP messages on 2.4 during poll command

From: Leif (
Date: 04/21/05

Date: 21 Apr 2005 07:32:47 -0700

We are having some trouble with a web server application. After
rebooting the machine, response times are excellent.
However, after several hours the system starts to slow down. This
slowdown is not generally observed with all requests, yet more and
more requests take several seconds to be processed. Having run an
strace on a bare Apache installation (shipped with Red Hat Enterprise
Linux), we disovered that there is a delay on the following system
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN, revents=POLLIN}],
2, -1) = 1

While this call usually takes <0.01 seconds, the delayed ones take
several seconds (the one above 2.972207).

In addition, we ran an IP trace and made the following observation:
while usually the client contacts the server, receives an ACK and
sends its request directly after receiving the ACK, all delayed
requests did have this delay between the ACK and the sending of the
request itself. While this might indicate a client rather than a
server issue, we've had this problem with all kinds of clients (Linux,
Windows, Mac), so I assume its rather a server-side issue (and
response times are fine when running the server on a Windows machine).
The observation on the server matches this trace result: the web
server receives a request, yet when trying to read the data the delay

Does anyone have suggestions what might cause this? Since a reboot of
the server accelerates the system, my first guess would be something
like a leak somewhere... but where?

dmesg output of our kernel:
Linux version 2.4.21-27.0.2.ELsmp (
(gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-47)) #1 SMP Wed Jan
12 23:35:44 EST 2005

Its a Dual-Xeon 2.4 GHz machine with HT enabled and 2gb RAM.

The web server itself has no impact on the situation, at least once
the slowdown has appeared. We have made the observation with a
proprietary Java web server (JDK 1.5) as well as with an Apache 2.0.