Re: help with huge memory consumption??

"David Schwartz" <davids@xxxxxxxxxxxxx> writes:
On Mar 6, 6:49 am, phil-news-nos...@xxxxxxxx wrote:
Programmers should choose to use threads _only_ when they fully
understand _why_ their application benefits from threads at a
level greater than the
risks that threads pose to their application, whatever size it might be.
They should also consider process separation where possible
(and I believe
it is more possible than commonly done today). Making processes talk to
each other, of course, does involve more work, and using less efficient
methods that can have an impact when there is a higher volume of data to
go between tasks and work units (e.g. an IM or IRC server).

I agree with you completely. The problem is that for a huge category
of real-world problems, threads are currently the only alternative to
'process per client/task'.

'Threads' are a way to have multiple threads of execution in a single
address space, nothing more and nothing less.

I would love to see more work done developing process-pool
technologies that would allow programs to use pools of processes as
easily as they use pools of threads. Ways to manage a pool of file
descriptors across processes and easily manage dynamic objects living
in shared memory would go a long way towards this end.

Something like that could be implemented almost trivially in C++ (or any
other language that can access memory directly, do systemcalls and
supports operator overloading). I would assume that it certainly is
already implemented in some library/ framework and the reason it isn't
universally used is that the complexity isn't really worth it.

[...]

In a 64-bit address space, globally-unique address space is not a
scarce resource.

And 640K is certainly enough for everybody for all times. That was as
conceivably true by the time Billy the Gates coined it as your
statement appears to be today. But for globally unique addresses, you
need either globally unique address mangement, which would be a
non-starter, or global address space segmentation, leading to the
question how many globally unique segments would be needed. Since 2^32
different global IPv4-addresses are considered to be to little, 2^32
different global distributed applications should be as well.

One of the issues that I complain about a lot that you are kind of
touching on is the irrational worship of performance.

'worship' implies 'irrational' because it is a religious
term. Appropriateness of the label would need to be proven for every
single, existing case.

It costs some time to create a process, and threads don't have that
time, so threads must be better.

On a system that has only ever supported 1:1 threading as default
model, 'process creation' being much slower than 'thread creation'
would be a strong hint at a deficiency in the kernel. But reportedly, the
difference is negligible on Linux, so your assumption should be
wrong. And it is a strawman, except if you constrict yourself to a set
of real-world cases where the reason you try to shoot down actually
was the reason to use threads.
.