Re: help with huge memory consumption??

On Mar 9, 3:57 pm, Chris Friesen <cbf...@xxxxxxxxxxxxx> wrote:

Sure there is. Given a parallel task that can be done with threads,
there is almost always an equivalent way to do that task using
separate
processes with shared memory, unix sockets, etc.

The design issues between the two approached would be so different
that it would no longer make sense to compare the creation time of a
thread to the creation time of a process.

For example, suppose the threads could be reused but the processes
couldn't. Then you would be creating far fewer threads than processes.
Suppose the processes could easily be reused but not the threads,
again the number would be different.

You cannot take two completely different programs and compare them by
looking at the cost of one operation that one does against some other
one operation the other does. That's just nonsensical.

While POSIX has defined threads and processes in a particular way,
from
an implementation level in linux they are fundamentally a schedulable
entity that shares various amounts of data with other schedulable
entities. Threads share almost everything, processes share almost
nothing. You can create wierd hybrids that are partway in between.

And if you did so, the costs of creating those weird hybrids would be
incomparable to the costs of creating either threads or processes.
It's like comparing the cost to overnight a pound of bananas against
the cost to send by boat a ton of eggs. There is almost no conceivable
case where you would want to do one or the other the same number of
times.

As has been mentioned in another post, there is nothing stopping you
from creating a "process pool" equivalent. From the other direction,
there is nothing stopping you from making a "pthread_create after
accept" version.

Except that such a program would be pointless. Obviously you can
create poor performance in any scheme by doing stupid things. The
response is simply, "that's stupid, don't do that", not, "completely
change to another architecture that does that at less cost".

Suppose you want to simultaneously issue statfs() calls for each
filesystem on the machine. One method would be to set up a shared
memory mapping for the results and fork() one process for each
filesystem. Alternately, you could pthread_create() one thread for
each
filesystem. The two are equivalent. There is slightly more overhead
involved in the fork() version, but the result is the same.

Let's assume they are equivalent. That means, in both cases we keep
the threads/processes around if we're going to reuse them (or we don't
in either case). That means the total creation cost in both cases is
negligible. If we aren't reusing them, then if the cost is high, it's
because of our bad design choices, not because of the cost of creating
threads or processes.

Alternately, suppose you have a long-running compute process, which
updates a display periodically. This could be implemented as two
threads, one for the compute and the other driving the GUI.
Alternately, it could be done via two separate processes
communicating
via sockets.

In both cases, the creation cost is negligible. So again there's no
reason to compare the creation cost of a process to that of a thread.

I don't deny that it might be possible, with extreme cleverness, to
create a case where this makes some kind of sense, but it's completely
unreasonable to claim that this comparison has any kind of
significance in general.

It's utterly nonsensical to start out with such a comparison in a
general talk about processes versus threads. It's a pure distraction.

DS

.

Relevant Pages

  • Re: What would you do to save the environment?
    ... You wonder if humans can design, ... including the cost of running the humans running the factories, ... But the point is not to compare ...
    (rec.arts.sf.science)
  • Re: ANN: New SBIG Camera-Control Software for Intel Macs
    ... be careful to compare comparably equipped and specced machines. ... What's the cost of adding FireWire 800 to a Dell? ... like crazy, compared to my PCs, because of poor ventilation design. ... The reliability of Apple computers has been at or near the top ...
    (sci.astro.amateur)
  • Re: help with huge memory consumption??
    ... | Because thread creation is in no way analogous to process creation. ... So it's nonsensical to compare ... Linux clone() call is not too far from this. ... The cost of a thread ...
    (comp.os.linux.development.system)
  • Re: Quad Core PC for less than a Mac Mini
    ... A meaningless one when compared to computers. ... Cars are quite easy for a friend of mine to assemble, ... compare equals. ... home is an example of a cost avoidance of eating out. ...
    (comp.sys.mac.advocacy)
  • Re: help with huge memory consumption??
    ... Given a parallel task that can be done with threads, ... Just because design issues are different (they are for lots of choices ... | You cannot take two completely different programs and compare them by ... | looking at the cost of one operation that one does against some other ...
    (comp.os.linux.development.system)