Re: How to set up a Linux machine that occupies the minimum memory footprint ?
From: Penang (penang_at_myrealbox.com)
Date: 11/30/04
- Next message: Penang: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Previous message: Penang: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- In reply to: Jean-David Beyer: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Next in thread: General Schvantzkoph: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Reply: General Schvantzkoph: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: 29 Nov 2004 23:01:06 -0800
Thanks for your reply.
Jean-David Beyer <jdbeyer@exit109.com> wrote in message news:<10qm7bl82jrqk0f@corp.supernews.com>...
> >>>>Anyway, is there not some limiting device in Linux to put a cap on the
> >>>>mount of memory each user is allowed to use?
> >>>Sure. But how do you know how much he is using? Plenty of it will be
> >>>shared - is he using it or only 1/n'th of it? And then what about
> >>>memory he has been promised but will never ask for? Do you count memory
> >>>that is copy-on-write but has not been written to?
> >>Agreed in general. However there must be some limitation of a 'user'
> >>program that obtains then uses all available memory until the swap
> >>partition is full. sort of imagine a program that calls gobs of
> >>Mallocs, writes them with zeroes and keeps going until the kernel is
> >>virtually paralysed.
> >>If the user in question runs his rendering software in 'user' mode
> >>with say a limit of 3.8G out of 4G RAM, then it can optimise its
> >>functions around this rather than fondly imagining it has (say) 8G
> >>memory and hence causes the kernel to 'thrash' thus slowing down the
> >>application.
> > I read the above with great interest, because it addresses the
> > potential traps about the very task that I am about to commit.
> > However, I must apologize for my own stupidity, because I do not fully
> > comprehend what is stated above.
> > Therefore, if there's anyone who can explain it to me - in simpler
> > English, por favor, svp, please - I would be extremely grateful.
> I think the poster is saying that if you have a process that asks for 8
> GBytes of RAM on a machine with 4 GBytes, it will think it has got 8
> GBytes of RAM, but in fact it will get at most 4, and the rest will be
> paged. Now if the working set at any time is smaller than 4 GB, you can be
> OK, but otherwise, the paging will probably kill you in terms of performance.
> But if your program is written to request only a total of 4GB (including
> its instructions and data), by itself it not need page much. The amount of
> RAM needed to avoid paging problems is not the static size of the program,
> but its working set, which is (rougly) the amount of memory it needs at
> any one time. For example, if the program needs 3GB to initialize itself,
> 3GB to run, and 3GB to finish off and exit, it does not have a 9GB working
> set (which is what matters), but a 3GB working set. But unless your
> program is really simple, it is difficult to determine its working set
> other than by running it and watching it (with top, for example).
The one thing that worries me the most is the potential of application
crashing in the middle of a rendering job. A normal render job may
take from hours to days, and it would kill me if it crashes on the nth
day - and everything that it has done is gone forever.
If there's a graceful way for the program to "exit" without a sudden
crash - while have everything saved - then the delay wouldn't be too
severe. I am looking for ways to achieve that, and in the meantime,
I'm concentrate on looking for a uni-proc mobo that can handle at
least 8GB of RAM, as insurance.
> > The software in question is a beast - originally run under Solaris -
> > and although we've managed to ported it to run under Linux, we haven't
> > have the chance to throw everything on it, yet.
> > True, we have successfully get it to render spnippets of
> > not-so-very-detailed test runs, and because we haven't really tapped
> > out it's full potential, I can't really tell you how hungry the beast
> > can be, if we are to render something like what we are about to do.
> > The 4GB figure is just a rough calculation - it could be more ! - and
> > that's why I am worried about whether the entire thing could be
> > achieved via the PC platform.
> I am afraid there is no alternative to measuring it when actually running
> with typical tasks and data.
That's what I am facing. Because the rendering going to HUGE, I really
have absolutely no idea what I am talking about, in terms of memory
usage.
> > About optimization - we thought about it, but unfortunately we neither
> > have the talent nor the time, budget, etc to achieve anything
> > meaningful at that front, so right now what I am doing is to bite the
> > bullet and set up a working system and start throw everything at it.
> > After reading the above quoted message, my main worry is that if I set
> > up a system with 8GB, would it hinder the rendering job _MORE_ than a
> > mere 4GB configuration ?
> Probably not enough to notice. Not compared to paging on a 4GB machine.
Just one idea: Anyone ever heard of a "hard-drive" made up of DRAM
MODULES ? If the cost is reasonable, maybe I'd even try to put one
16GB-size "HD" made up of DRAM into the test machine - paging delay
shouldn't be a huge problem if that's possible.
- Next message: Penang: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Previous message: Penang: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- In reply to: Jean-David Beyer: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Next in thread: General Schvantzkoph: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Reply: General Schvantzkoph: "Re: How to set up a Linux machine that occupies the minimum memory footprint ?"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
