Re: Loading an I/O intensive program into memory
- From: Maxwell Lol <nospam@xxxxxxxxxxx>
- Date: 27 May 2008 07:20:36 -0400
"C++ Newbie" <newbie.cpp@xxxxxxxxxxxxxx> writes:
OK, that's handy to know. It would still be interesting to compare the
relative performance of letting Linux do the RAM buffering and
explicitly loading the lot into RAM.
You can lock programs into memory, but that may or may not help. It
would eliminate the time when the program is swapped out to disk
because another process has higher priority. If that's happening, you
have other problems.
Usually more RAM will allow better disk buffering. The system will
tend to keep as much "disk" in memory as possible.
What's the profile of the CPU when the program is running? Is it CPU
bound, or I/O bound?
Do you mean DIY writing of equivalent programs to grep, sed & awk or
just #including them? I've always thought of #including to be a bit of
a cop-out as you're really just using black boxes :-).
That's really the wrong way to do things. What you want is to have
the flexibility to re-write your program to be as efficient as
possible, which is easier because of sh/sed/grep/awk/sort.
It's easier to redo the main loop of code in shell than in C.
Optimizing in C too early is bad.
The general rule is that 90% of the time is spent in 10% of the code.
That's what you want to optimize. Don't guess. Know before you re-write.
I would try to use shell/awk/sort/etc before C to get the logic optimized.
But you are using C, so .....
Have you done any profiling of your C code?
Where is your program spending 90% of its time?
It's been a while, but you do something like
1) recompile all of your programs using the "-pg" option for
profiling. (and maybe -fprofile-arcs according to the gprof(1) man
page).
2) Run your program with real input to gather the profile data.
3) run the "gprof" program, which calculates the amount of time spend
in each routine. If you only have a few, you many need to add more
just to profile the code.
The real thing to do is to consider the loops of execution. Can the
number of loops be eliminated? Can you remove code from inside the
loop to outside the loop. Are you sorting data, and can you make it
more efficient?
And, or couse, have you compiled the program with optimization enabled?
But it really sounds as if you are doing it backwards. For instance,
if you are using sed, grep and awk, can one of these programs be
eliminated? How many times are they executed in your program?
.
- Follow-Ups:
- Re: Loading an I/O intensive program into memory
- From: Jean-David Beyer
- Re: Loading an I/O intensive program into memory
- References:
- Loading an I/O intensive program into memory
- From: C++ Newbie
- Re: Loading an I/O intensive program into memory
- From: The Natural Philosopher
- Re: Loading an I/O intensive program into memory
- From: C++ Newbie
- Loading an I/O intensive program into memory
- Prev by Date: Re: Loading an I/O intensive program into memory
- Next by Date: Re: Loading an I/O intensive program into memory
- Previous by thread: Re: Loading an I/O intensive program into memory
- Next by thread: Re: Loading an I/O intensive program into memory
- Index(es):
Relevant Pages
|
