Re: large binary immediately SEGV's
From: Jean-David Beyer (jdbeyer_at_exit109.com)
Date: 02/21/04
- Next message: Jean-David Beyer: "Re: large binary immediately SEGV's"
- Previous message: General Schvantzkoph: "Re: Redhat v7.3"
- In reply to: Nick Landsberg: "Re: large binary immediately SEGV's"
- Next in thread: P.T. Breuer: "Re: large binary immediately SEGV's"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Sat, 21 Feb 2004 13:50:51 -0500
Nick Landsberg wrote:
>
> Jean-David Beyer wrote:
>
>> As it was designed, we could even change the implementation of
>> running processes and test them without taking the system down.
>> Certain users could run the new implimentation and the rest (most
>> of them) got the default implimentation. We just had to obey the
>> interface specification, and if we goofed that up, only our own
>> stuff failed. This was a wonderful way to test new algorithms and
>> stuff on live data without hurting the innocent.
>
>
> Ah, this is where our worlds diverge. It has been many years since I
> worked on systems which dealt directly with end-users.
It was a telecommunication system, sort of like a glorified PBX, but it
handled voice, text, and image with equal facility, converting
automatically between them if required (e.g., if someone with a
voice-only terminal wanted to examine an e-mail, a speech synthesizer
read it to him). The call processing software knew nothing about it as
it was handled at a lower level. The users were not allowed to program
it in the usual sense. They could customize the features, their "address
books", etc. Only extremely privileged users could change the software
in any way.
> Rather, I work on systems which process large amounts of requests
> from other systems. These other systems are always connected. This
> is what I meant by "it's ok at startup", below. We chose a
> "subsystem" style of architecture, e.g. "COMM handlers" for the
> external communications, "LOGIC handlers" to handle the various types
> of requests, and a DBMS abstraction layer (which can be either linked
> in or run as client-server). There are maybe about 20 processes
> doing the "heavy lifting" and another 80 or so which are also started
> at init time which hang around to do periodic maintenance activities.
>
Our system had three conceptual levels of operation:
The application level (where the most different kinds of processes;
i.e., programs existed, and the most instances of each),
The server level, where there were expected to be four kinds of servers:
an interface server, a process server, a communication server, and a
data server. If you were running on one machine, there would be one
instance of the PS, CS, and DS, and one instance of the IS for each user
actually running. Now when we distributed the system, there was a PS,
CS, and DS on each machine and they all talked with TCP/IP.
The resource level, where hardware and os-dependent stuff went on. These
tended to be the klugiest, the most heterogenious. What made this
tolerable is that only the servers were allowed to use the resource
level, and only the applications could use the servers. The idea was to
push the complexity down.
> A few processes are transients invoked by cron, very few.
>
> We don't have the luxury of testing a new version of a LOGIC handler
> on a few guineau pig users.
>
We could do a lot. I do not think we could change any of the server
programs on a hot system, though it may have been possible to change
interface servers. One could certainly change any of the applications
and have the new versions work only for specified users. A decade has
passed since I worked on that system, so details are hazy.
In our system, what is typically an extremely difficult thing to do,
call processing, was a application. If a user wanted to do call
processing, he would do what needed to be done to get one running. Note
that he not really need be there. If an incoming call comes in to him,
an instance of a call processing program would be invoked, it would not
he is not there, decide what to do (forward the call, give custom
answer, voice mail it, no answer, I forget what all). Now in a normal
electronic central office, call processing is a gigantic mass of
spaghetti trying to keep track of the progress of all calls through the
system, possibly 25,000 calls, possibly more (depending on the size of
the CO). That is really difficult, especially with all the fancy
customizable features the telcos like to offer. With our system, call
processing needs to keep track of only a few calls, typically one.
So if we want to experiment with call processing, two system testers
just install a new call processing program, diddle their profiles so
that when they do call processing they get the new one, and everyone
else gets the default one. Now if their stuff works, it can be made the
default. And if it is lousy or crashes, their calls, and only their
calls, go wrong. The reason the call processing programs are relatively
simple is that they do not have to run the switch. The call processing
program uses the Connection Server to manage connections. And the CS is
rather simple, since it uses the Switch resource to actually run the
switch. We have used Dialogic, RedCom, and a few other switches, and
only the Resource level needed diddling. Try that with your #5ess
software with a switch transplant. ;-)
>>> P.S. - I would tend to disagree that process creation is "cheap"
>>> on ANY O/S. If it is ONLY done on startup, then it's OK. If
>>> programs fork()/exec() others willy-nilly, it's real pig.
>>>
>> A pig compared to sqrt(integer) no doubt. But not compared to UNIX
>> in about 1970. It is much much better now, when we do paging
>> instead of process swapping, when the processors have memory
>> management units at their disposition, ... .
>>
> But in the system I described, we budget the CPU cycles very
> carefully. The LOGIC handlers must complete in <100 microseconds and
> the DBMS lookup in <500 microseconds. A fork/exec pair is still
> measured in milliseconds of CPU time.
In ours, we did not actually budget CPU cycles. Even though we were
running on a 1989 DELL i386 (or whatever was state of the Intel art at
the time) box, with a few megs of memory, a Motorola box with the
88100/88200 chip set (I think it was), and another box full of Dialogic
boards to drive POTS telephones, X-Terminals, telco trunks, T1-links,
and to switch calls among all that, and to snatch digitized voice to
hard drives (to manage voice mail), and so on. Now if we were close to
hurting, we probably would have done CPU cycle budgeting, but when we
were anywhere near close, we normally figured out a better
implementation of whatever was hogging. Our biggest problem with UNIX at
the time was running out of space in message input queues.
>
> That's another place where our experiences diverge.
>
> (No, we do not use a disk database, it's all in memory, all 23 GB of
> it, with another 23 GB reserved for schema updates on the fly, and
> the remaining 2 GB for executables. We design not to page during
> steady state).
>
Times have changed in the last 30-40 years, have they not? I did some
database research on UNIX in the early mid to late 1970s. We put all the
business, professional, and government listings of the Suffolk County,
NY telephone directory on a hard drive of a PDP-11/45 machine running
UNIX (Pre-SVR anything: it would have to be called SIIIR something).
That was about 100,000 listings, and the government ones were especially
complex. The hard drive (we had 4, IIRC) held 40 megabytes. The machine
I am builing in my kitchen as I type this, has 4GBytes RAM in it, and
that RAM cost only $1200. Those hard drives were 2400rpm and cost well
over $10K each.
-- .~. Jean-David Beyer Registered Linux User 85642. /V\ Registered Machine 73926. /( )\ Shrewsbury, New Jersey http://counter.li.org ^^-^^ 1:15pm up 46 days, 36 min, 2 users, load average: 2.15, 2.15, 2.10
- Next message: Jean-David Beyer: "Re: large binary immediately SEGV's"
- Previous message: General Schvantzkoph: "Re: Redhat v7.3"
- In reply to: Nick Landsberg: "Re: large binary immediately SEGV's"
- Next in thread: P.T. Breuer: "Re: large binary immediately SEGV's"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
