Re: help with huge memory consumption??

"David Schwartz" <davids@xxxxxxxxxxxxx> writes:
On Mar 9, 3:57 pm, Chris Friesen <cbf...@xxxxxxxxxxxxx> wrote:
Sure there is. Given a parallel task that can be done with threads,
there is almost always an equivalent way to do that task using
separate
processes with shared memory, unix sockets, etc.

The design issues between the two approached would be so different
that it would no longer make sense to compare the creation time of a
thread to the creation time of a process.

As I understand this statement, it means 'everything that is not
related to process or thread creation will be different if done
differently, and therefore, the process and thread creation times
cannot be compared'. But that is just plain illogical.

[...]

As has been mentioned in another post, there is nothing stopping you
from creating a "process pool" equivalent. From the other direction,
there is nothing stopping you from making a "pthread_create after
accept" version.

Except that such a program would be pointless. Obviously you can
create poor performance in any scheme by doing stupid things. The
response is simply, "that's stupid, don't do that", not, "completely
change to another architecture that does that at less cost".

Well, that you believe something would be stupid does neither mean it is
not quantifiable nor that the quantatities determined that way cannot
be sensibly compared to other quantities.

Suppose you want to simultaneously issue statfs() calls for each
filesystem on the machine. One method would be to set up a shared
memory mapping for the results and fork() one process for each
filesystem. Alternately, you could pthread_create() one thread for
each
filesystem. The two are equivalent. There is slightly more overhead
involved in the fork() version, but the result is the same.

Let's assume they are equivalent. That means, in both cases we keep
the threads/processes around if we're going to reuse them (or we don't
in either case). That means the total creation cost in both cases is
negligible.

That means that the creation cost per operation (however defined)
would asymptotically approach zero. It does not say anything about the
cost itself.

Alternately, suppose you have a long-running compute process, which
updates a display periodically. This could be implemented as two
threads, one for the compute and the other driving the GUI.
Alternately, it could be done via two separate processes
communicating
via sockets.

In both cases, the creation cost is negligible. So again there's no
reason to compare the creation cost of a process to that of a
thread.

'Negligible' is not a quantity but an opinion about the impact of a
quantity. You could, for instance, measure and compare the cost to
determine how long the program would need to run so that the quantity
divided by the number of somehow defined 'work units' becomes smaller
than some constant.

I don't deny that it might be possible, with extreme cleverness, to
create a case where this makes some kind of sense, but it's completely
unreasonable to claim that this comparison has any kind of
significance in general.

You contradict yourself: If the comparision is impossible, as you have
claimed, it cannot have a 'significance' by virtue of not existing.
.