Too Many Files Open

From: Shadus (shadus_at_shadus.org)
Date: 10/05/05 

Date: Wed, 05 Oct 2005 09:29:46 -0500

I'm running Gentoo on a 2.4-Hardened kernel, I'm having an on going
problem that I can't seem to resolve, and wondered if perhaps anyone
else had some ideas.

The on going problem occurs with the Sybase v7 ASA DB Engine and the
error I'm getting is "To many files open" when it attempts to open a
database or transaction log. The machine has 4g of ram, duel processor
xeon w/ht, using a xfs file system.

I've tried the following:
echo 1048576 > /proc/sys/fs/file-max

Setting sysctl.conf:
fs.file-max = 1048576
kernel.shmmax = 102400000

Raising the ulimit to 4096 (from 1024):
ulimit -n 4096 (in the script that calls db and in roots login shell
(since some of the admins manually restart it))

The sybase database engine seems to take up about 750 file handles on
it's own.. it also runs 25 database processes total. The server crashes
somewhere in the open file handle range of 25000-27500 (my personal
guess would be between 25600-26350 [eg: 25*1024 or 25*1024+750]). It
varies by a bit. Also as it nears this limit the database itself slows
to a crawl.

I'm out of ideas on what I need to do to correct this problem other than
move databases off to another server to lower the number of connections,
but that really isn't an acceptable solution at the level of load the
server/memory usage is at.

Any ideas or suggestions on where to go from here? I've done some
searching on google, but not found a good answer to my problem, I'm
beginning to think the limitation may be in the kernel itself, but I've
had no real luck finding it there either.

Thoughts, comments, and suggestions are greatly appreciated.

Shadus

Relevant Pages

  • Re: reiser4 plugins
    ... Shoving the RDBMS into the kernel doesn't solve this, ... > database with easy standard tools like a file manager or from command ... operating systems with funny "just as applications require" ... > of KDE where the filesystem view is implemented even in a higher level ...
    (Linux-Kernel)
  • Re: OT] Joerg Schilling flames Linux on his Blog
    ... The current problems in Linux are just a result of mixing unrelated interfaces ... >> A database may kill a hdd in many other ways. ... >> your kernel. ...
    (Linux-Kernel)
  • Re: NFS intr/nointr: SIGKILL may leave pages locked forever
    ... On Oct 29, 2008, at 12:38 PM, Chuck Lever wrote: ... Looks like this week's kernel version is 2.6.27-rc7, but I will need to confirm that. ... Since 2.6.25, the problem is the "sql shutdown abort" command, which is designed to trigger an immediate database shutdown, causes the database instance to hang. ... On single processor clients, SIGKILL works correctly and the database shuts down without corrupting its data files. ...
    (Linux-Kernel)
  • Re: Dual-Licensing Linux Kernel with GPL V2 and GPL V3
    ... but the signature isn't part of the kernel, and the code that checks the signature is completely independant. ... if your argument was true then Oracle releasing a database appliance would require Oracle to give you the source to their database since it's part of 'bringing about a certain result' namely operating as a database server. ... if your argument was true then releasing a GPL package for windows would require that the windows kernel source be released, after it it's nessasary for 'brining about a certain result' namely letting your code run. ...
    (Linux-Kernel)
  • Re: Large file write pauses to for kbmemfree to replenish
    ... Why such an old kernel? ... way to mount an NFS volume to use direct I/O? ... Interestingly, but somewhat unrelated, we tried telling the database ... and direct_io was slightly slower on both. ...
    (comp.os.linux.misc)