Re: help with huge memory consumption??

David Schwartz wrote:

...thread creation is in no way analogous to process creation.
There is no case where you think, "hmm, I could create a thread or a
process, I wonder which is cheaper". So it's nonsensical to compare
their costs.

Sure there is. Given a parallel task that can be done with threads, there is almost always an equivalent way to do that task using separate processes with shared memory, unix sockets, etc.


It means that you need to create a thread when, and only when, you
need more threads than you have already created. However, you need to
create a process not just when you need more processes but when you
need different processes.

While POSIX has defined threads and processes in a particular way, from an implementation level in linux they are fundamentally a schedulable entity that shares various amounts of data with other schedulable entities. Threads share almost everything, processes share almost nothing. You can create wierd hybrids that are partway in between.

As a trivial example, consider a thread pool versus a 'fork after
accept' server. The thread pool server will create some maximum number
of threads over its life and will reuse them. The cost of a thread
creation per client is effectively nearly zero. A 'fork after accept'
server will create one process for each client it handles. The cost of
a process creation is a per connection cost.

As has been mentioned in another post, there is nothing stopping you from creating a "process pool" equivalent. From the other direction, there is nothing stopping you from making a "pthread_create after accept" version.

So it makes no sense whatsoever to compare the cost of one to the cost
of the other. There is essentially no actual circumstance where one
has the choice to do one or do the other the same number of times.

Sure there are.

Suppose you want to simultaneously issue statfs() calls for each filesystem on the machine. One method would be to set up a shared memory mapping for the results and fork() one process for each filesystem. Alternately, you could pthread_create() one thread for each filesystem. The two are equivalent. There is slightly more overhead involved in the fork() version, but the result is the same.

Alternately, suppose you have a long-running compute process, which updates a display periodically. This could be implemented as two threads, one for the compute and the other driving the GUI. Alternately, it could be done via two separate processes communicating via sockets.

Chris
.

Relevant Pages

  • Re: Macs not used in business, part II
    ... collaboration software (and the accompanying server technology.) ... surrounding it, would have cost US$100,000, Magosky says. ... Office SharePoint Server 2007 Standard CAL - $94 ... Windows solution didn't cost $100K. ...
    (comp.sys.mac.advocacy)
  • Re: Fault Tolerence on SBS2003 Prem.
    ... Perhaps we could convice Microsoft to bring the cost down a little. ... There is always a compromise involved in any business ... > server equipment that improved the recovery time from an annual event from ...
    (microsoft.public.windows.server.sbs)
  • Re: backup email to a mail hosting company.
    ... There must be a way to have a live pop backup of exchange email! ... Mostly the cost. ... > server, ...
    (microsoft.public.backoffice.smallbiz2000)
  • Re: Fax pooling performance
    ... your cost per page just skyrocketed :-) ... 'proper' fax server software, ISDN lines and multiple ISDN FAX devices. ... This facility is not available in SBS ... I was a support guy for a modem manufacturer who was also ...
    (microsoft.public.windows.server.sbs)
  • Re: backup email to a mail hosting company.
    ... There must be a way to have a live pop backup of exchange email! ... Mostly the cost. ... > server, ...
    (microsoft.public.exchange2000.clients)