NFS performance issue

From: ASB (vmlinuz_at_abisen.com)
Date: 09/24/04

  • Next message: Lloyd H. Meinholz: "RE: redhat-config-httpd question" 
    Date: Fri, 24 Sep 2004 11:36:23 -0500
To: Npaci Rocks <npaci-rocks-discussion@sdsc.edu>, redhat-list@redhat.com

    Hello,

    I have a NFS server set-up on a RHEL AS 3.0 machine with 4 Pentium IV
    Xeon's and 4 Gig of RAM that is just acting as a NFS server to around 2
    TB of disk space (2 LUNS). The NFS server has connects to a Gigabit
    switch with 4 Aggregated Gigabit NIC's. And the storage is connectd
    using Fiber connects. This NFS share is mounted by 48 node cluster
    (Rocks 3.2.0 / based on RHEL 3.0 base srpms). On the NFS server there
    are 8 instances of nfsd serving the requests and i dont see the network
    contention (no even near contention) on the NFS server and/or Disk's.
    The requests that NFS server is serving are 1000's of small
    read/write/stat calls.

    What i need to know is how can i improve the NFS server performance what
    should be the options on the NFS export and NFS clients that mount the
    NFS share (i.e. should i use udp or tcp for NFS, hard vs soft mounts
    etc.. my applications should not get a file not found or error because
    they are too touchy to errors)

    1. Is the number of nfsd's on a 4 processor machine enough, should i
    enable HT or not. During loads when i do a cat /proc/net/rpc/nfsd this
    is the output i get but am unable to determine that do i need more
    instances of nfsd's.

    # cat /proc/net/rpc/nfsd
    rc 119962 25377040 42222820
    fh 3690 66565276 0 140 3637
    io 3570911731 3462845160
    th 8 30591462 10278.250 4629.100 3809.780 0.000 3457.170 2708.270
    2035.320 1717.180 0.000 19783.720
    ra 16 3645786 226692 72466 132437 59243 60455 41709 14720 25792 13253
    1901791
    net 67719822 67702906 16916 33
    rpc 67719822 0 0 0 0
    proc2 18 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    proc3 22 2 9122660 314574 9685297 13420650 0 6195787 19583200 2475564
    902 0 0 1772088 717 1349957 0 82830 102511 103 294 0 3312072

    after 1 day of operation

    rc 432346 54116787 118493699
    fh 5102 167914277 0 724 48367
    io 1430949976 1209324547
    th 8 110245501 17130.230 6509.020 5264.770 0.000 4811.310 4098.190
    3469.520 3636.230 0.000 83702.300
    ra 16 7044467 559769 182947 290285 115112 142084 99659 35098 61013 28165
    5258146
    net 173042832 173025916 16916 33
    rpc 173042832 0 0 0 0
    proc2 18 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    proc3 22 2 25577479 3740196 30203894 38264086 0 13818455 37670132
    5648930 1252 0 0 4041598 957 3446068 0 721617 495716 276 294 0 8962512

    2. Should i change the mount option in my fstab the current settings are

    nfsserver:/export/lun1 /global/data/lun1 nfs hard,udp,defaults 0 0
    nfsserver:/export/lun2 /global/data/lun2 nfs hard,udp,defaults 0 0

    What should i concentrate on in order to improve / optimize for
    performance and reliability.

    Thanks

    Anand 

    -- 
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list
  • Next message: Lloyd H. Meinholz: "RE: redhat-config-httpd question"

    Relevant Pages

    • Re: Some clients cant see NFS after re-boot
      ... Peter Smithson wrote: ... I suspect that the mounts that aren't working are not set to automount. ... > They can all ping the server and they can all see another nfs server. ... mount server:/remotmountpoint /localmountpoint ...
      (comp.unix.admin)
    • Re: Very slow mount over NFS?
      ... Install the portmapper service on *BOTH* your NFS server and ... > easily five minutes trying to mount. ... It's equally slow mounting Samba shares. ...
      (Debian-User)
    • Re: 2003 R2, Services For Unix, NFS Server
      ... I have the NFS server up and running and can mount NFS ... When I mount home directories from another linux box, ... I have had no luck with the GID. ...
      (microsoft.public.windows.server.scripting)
    • Re: Help needed with server setup at work
      ... Most of the employees mount their home dirs manually, ... and emplyoee Britney knows she belongs to NFS server 3 and so on. ... About the union thing I first thought of somehow union mouting all the ...
      (Debian-User)
    • NFS and "Permission denied" error
      ... I'm trying to set up an NFS server, with MDK 10.1 and nfs-utils-1.0.7, ... but from whichever machine I try to mount the exported directory, ... empty), exportfs updates /var/lib/nfs/etab. ... May 23 13:40:55 localhost rpc.mountd: authenticated mount request from ...
      (comp.os.linux.misc)