Re: linux nfs server tuning

From: P Gentry (rdgentry1_at_cablelynx.com)
Date: 06/12/04


Date: 12 Jun 2004 09:29:31 -0700


"Chris Walker" <cwalkatron@gmail.com> wrote in message news:<cadpj8$54b@odah37.prod.google.com>...
> The following shell clause shows the send and receive buffers for the
> NFS server:
>
> #!/bin/sh
>
> while true; do
> netstat -nua | egrep :2049
> sleep 1
> done
>
> The output is this (on a loaded machine):
>
> (Proto Recv-Q Send-Q Local Address Foreign Address)
> udp 81340 5456 0.0.0.0:2049 0.0.0.0:*
> udp 167328 0 0.0.0.0:2049 0.0.0.0:*
> udp 202188 0 0.0.0.0:2049 0.0.0.0:*
> udp 160356 0 0.0.0.0:2049 0.0.0.0:*
> udp 190568 0 0.0.0.0:2049 0.0.0.0:*
> udp 202188 0 0.0.0.0:2049 0.0.0.0:*
> udp 167328 0 0.0.0.0:2049 0.0.0.0:*
> udp 202188 0 0.0.0.0:2049 0.0.0.0:*
> udp 125496 0 0.0.0.0:2049 0.0.0.0:*
> udp 109228 0 0.0.0.0:2049 0.0.0.0:*
> udp 169652 0 0.0.0.0:2049 0.0.0.0:*
> udp 202188 0 0.0.0.0:2049 0.0.0.0:*
> udp 202188 0 0.0.0.0:2049 0.0.0.0:*
>
> You'll notice that size of the recv buffer never seems to exceed
> 202188.
> I thought that this would be the size of the recv buffer, but it
> doesn't

See below in general ... but there can be a number of
settings/variables involved with any _particular_ observation.

> appear to be (cat /proc/sys/net/core/rmem_max shows 131071). I couldn't

65535 * 2 = 13170 plus there are several "conflicting/confusing"
settings -- I have trouble keeping them sorted out ;-(

> find any indication that the recv buffer size was be set before the nfs
> server starts and reset after.
>
> Does anyone know why the recv buffer seems to be topping out at 202188
> bytes? Can this ceiling be raised?

These figures are reflective of the interaction of the IP stack _and_
the specific app.

> This is 2.4.24 on Redhat 7.1. The system is a dual 2.8GHz Xeon with 4GB
> of RAM and 100Mbit ethernet.

$man netstat
...
OUTPUT
...
Recv-Q
The count of bytes not copied by the user program connected to
this socket.

Ie., it is not _determined_ by the size of rmem buffers alone.
netstat is OK as far as it goes, but I usually prefer to break out the
sniffer early on and see just what's happening on a packet level.
Disk I/O will ultimately limit the tranfer rates of larger files --
especially packets read from wire, then _written_ to disk.

So far as "tuning" some of the net variables check these out:
file:/usr/src/linux-2.4.?/Documentation/networking/ip-sysctl.txt
http://ipsysctl-tutorial.frozentux.net/ipsysctl-tutorial.html
http://lartc.org/howto/lartc.kernel.obscure.html
I think these pretty much require a "tune/change by observation"
approach except for the more "obvious" settings ;-)

I assume you're looking to improve throughput with larger transfers
;-)

The NFS HOWTO has some pretty good -- general -- tuning pointers:
http://www.tldp.org/HOWTO/NFS-HOWTO/performance.html

Also will depend on the specific NFS server you're running.

BTW, you may also have some peculiaritues relating to the smp setup
you have. You may want to double check just how smp friendly your NFS
version is -- upgrade? update? cpu affinity? I have no specific
knowledge of running NFS with smp/RH7.1 -- this is just a general
suggestion.

hth,
prg
email above disabled



Relevant Pages

  • linux nfs server tuning
    ... NFS server: ... You'll notice that size of the recv buffer never seems to exceed ...
    (comp.os.linux.networking)
  • Re: Why is my animation flickering?
    ... buffer but you didn't specify if you want GLUT_RGBA or GLUT_RGB ... perhaps flickering is the wrong word. ... Do you get a frame rate of something like 60 or 70 FPS? ... I looked in the settings but souldn't find what ...
    (comp.graphics.api.opengl)
  • Re: Why is my animation flickering?
    ... buffer but you didn't specify if you want GLUT_RGBA or GLUT_RGB ... perhaps flickering is the wrong word. ... Do you get a frame rate of something like 60 or 70 FPS? ... I looked in the settings but souldn't find what ...
    (comp.graphics.api.opengl)
  • Re: [Full-disclosure] screen 4.0.3 local Authentication Bypass
    ... I tested using the default settings as per OS X ... While I could not replicate the bypass, I did find that screen will ... Even copying the password into the buffer (all variations ... experiencing an authentication bypass, then it is likely that you ...
    (Full-Disclosure)
  • update: problems rendering mpeg2 through buffer
    ... I thought maybe some of my settings on the demultiplexer were wrong because I ... had to configure them manually as I was using it in push mode. ... sending it through the buffer. ... Microsoft says that the stream buffer engine can handle MPEG2 video streams ...
    (microsoft.public.win32.programmer.directx.video)