efficacy of Linux w/o swap

phil-news-nospam_at_ipal.net
Date: 05/16/05 

Date: 16 May 2005 07:55:54 GMT

I'm wanting to put together a machine to do a single dedicated purpose
that will use Linux and commodity hardware. The purpose would be to
drive a firewire based digit video (DV) encoder/decoder much like a
PVR, but in this case somewhat different in functionality. However,
in this case, unlike a PVR, performance is an absolute must. The video
stream must be input or output continuously without any loss. In order
to achieve that goal I'm expecting the process doing the actual work
would have to be locked into RAM to be sure it doesn't ever fail to get
it's needed CPU cycles to keep the stream going.

One issue that I've experienced plaguing Linux in the past (especially
in many early 2.4 kernels) is that heavy writing to disk resulted in
large scale caching of the written data in RAM, resulting in fewer
usable RAM pagses for process VM, resulting in more processes being
swapped out. When I tested these scenarios with swap disabled, things
actually got worse, rather than better. It seems the caching still was
being done, but this time the kernel choked on RAM being totally full
and spent large amounts of CPU doing some kind of activity looking for
available pages. The way I observed this was through virtual console
(X was not running) switching. When I was in tty1 and pressed Alt-F2
to go to tty2, it would pause and not switch for quite some time, then
evetually complete the switch. Worst case event took 2.5 minutes.

Are these problems definitely cured in 2.6 at least at the current
versions?

Does writing with O_DIRECT absolutely avoid this data from being cached?
Since the idea is to write from user space rather than copying data to
kernel space first, it seems this data could not be cached without some
kind of copying taking place. Once the data is written, in theory the
process can put new data there. So the kernel would have to go out of
its way to get it cached, and I hope that is not the case.

Any other suggestions? Should I configure the kernel more like an
embedded system even though this will just be running on commodity
computers (i386 or maybe some day PPC)? 

-- 
-----------------------------------------------------------------------------
| Phil Howard KA9WGN       | http://linuxhomepage.com/      http://ham.org/ |
| (first name) at ipal.net | http://phil.ipal.org/   http://ka9wgn.ham.org/ |
-----------------------------------------------------------------------------

Relevant Pages

  • Re: Kernel memory leak?
    ... However, the remaining RAM decreases day by day, and after 2 or 3 ... This behaviour has been seen on Red Hat Enterprise Linux 3 with a 2.4 ... kernel and on SuSE Linux 10 with a 2.6.13-15-default kernel. ... Is it possible that processes are invisible in /proc or in the procps tools? ...
    (Linux-Kernel)
  • Re: New hardware, advice wanted
    ... Only between your MoBo and the RAM, not between RAM and linux. ... Sound output ... An ATA drive is recommended for kernel 2.4. ...
    (alt.os.linux.suse)
  • Re: Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)
    ... we upgraded a Server from 32 GB RAM to 64 GB. ... Now we try to get a Linux ... My friend Bjoern has forgotten to write that we actually checked the memory with only 32GB and kernel booted normally. ...
    (Linux-Kernel)
  • Re: New to Linux
    ... A good, and comprehensive, overview of available Linux distributions ... Any distro includes the Linux kernel. ... For Windows 98, 32 MB of RAM would be _very_ tight, IMO; ...
    (comp.os.linux.questions)
  • Re: differences between kernel-tree and kernel-source and kernel image
    ... > will this install the kernel or try to patch an existing 2.6 kernel or ... affix-source - Driver source for the Affix Bluetooth protocol stack for Linux ... atlas-doc - Automatically Tuned Linear Algebra Software,documentation ...
    (Debian-User)