Delayed TCP messages on 2.4 during poll command

From: Leif (matrix_2004_at_gmx.net)
Date: 04/21/05


Date: 21 Apr 2005 07:32:47 -0700

We are having some trouble with a web server application. After
rebooting the machine, response times are excellent.
However, after several hours the system starts to slow down. This
slowdown is not generally observed with all requests, yet more and
more requests take several seconds to be processed. Having run an
strace on a bare Apache installation (shipped with Red Hat Enterprise
Linux), we disovered that there is a delay on the following system
call:
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN, revents=POLLIN}],
2, -1) = 1

While this call usually takes <0.01 seconds, the delayed ones take
several seconds (the one above 2.972207).

In addition, we ran an IP trace and made the following observation:
while usually the client contacts the server, receives an ACK and
sends its request directly after receiving the ACK, all delayed
requests did have this delay between the ACK and the sending of the
request itself. While this might indicate a client rather than a
server issue, we've had this problem with all kinds of clients (Linux,
Windows, Mac), so I assume its rather a server-side issue (and
response times are fine when running the server on a Windows machine).
The observation on the server matches this trace result: the web
server receives a request, yet when trying to read the data the delay
occurs.

Does anyone have suggestions what might cause this? Since a reboot of
the server accelerates the system, my first guess would be something
like a leak somewhere... but where?

dmesg output of our kernel:
Linux version 2.4.21-27.0.2.ELsmp (bhcompile@tweety.build.redhat.com)
(gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-47)) #1 SMP Wed Jan
12 23:35:44 EST 2005

Its a Dual-Xeon 2.4 GHz machine with HT enabled and 2gb RAM.

The web server itself has no impact on the situation, at least once
the slowdown has appeared. We have made the observation with a
proprietary Java web server (JDK 1.5) as well as with an Apache 2.0.

Thanks,
Leif



Relevant Pages

  • Re: best way to serve wsgi with multiple processes
    ... although presumably, due to the beauty of WSGI, this shouldn't matter.. ... the server required - ideally with a simple example] and most reliable ... embedding mini web server that understands XML-RPC. ... server would then make XML-RPC requests against the backend daemon ...
    (comp.lang.python)
  • Re: best way to serve wsgi with multiple processes
    ... although presumably, due to the beauty of WSGI, this shouldn't matter. ... the server required - ideally with a simple example] and most reliable ... embedding mini web server that understands XML-RPC. ... server would then make XML-RPC requests against the backend daemon ...
    (comp.lang.python)
  • Re: write with cURL
    ... execute permissions. ... This is assuming that the PHP script runs ... of potential security risks from other users on the same server. ... web server itself is part of the group. ...
    (alt.php)
  • Re: Reg: Immediate requirement for multiple skills from our direct client.Only phone interview
    ... Visual Studio can preview ASP.NET pages but Dreamweaver ... caught by the server side code behind, but in terms of pure HTML, ... Of course, if you are using IIS as the web server, ...
    (comp.lang.cobol)
  • RE: System.Data.SqlClient "Timeout expired" causing ASP.net web applic
    ... There are many values here that can shutdown the aspnet_wp. ... > update tables on a Web Server running SQL Server 2000. ... > formation(DataSet currentBalances): Timeout expired. ...
    (microsoft.public.dotnet.framework.aspnet.webservices)