Re: remote access during runaway process

On 2006-01-04, Todd W <toddrw69@xxxxxxxxxx> wrote:
>
> On a remotely hosted Linux server, a developer's code went in to deep
> recursion, and all of the services on the system became unavailable,
> requiring a call to the hosting provider.
>
> I'd like to make it so ssh is still available in this situation.
>
> After some research (I'm much more of a developer than a sysadmin myself), I
> came up with putting "snice -20 sshd" in /etc/rc.d/rc.local, but I don't
> know for sure if this would give the desired results.
>
> Will someone please advise me as to whether this is okay or point me to the
> proper solution if I am way off base?

Well, it might work okay, but if your load starts hitting 20 or 30, even
sshd will go dog-slow. (I had this once--it took three minutes to go
from initial connect to shell prompt. Killing the process was even more
painful.) One other option might be to set up a serial console, but
you'll still run into the problem that you need to get a shell, find out
which process is going crazy, and kill it. The only potential advantage
is that, over a serial console, you might (*might*) be able to send
Ctrl-Alt-Del to the server, so that you can reboot without having to log
in and get a shell and issue shutdown -r. But doing a reboot may not be
a good option for you anyway.

Another potential solution is to use ulimits. If this is a command-line
program, ulimit the user's shell. If it's a CGI, ulimit (or RLimit)
apache (or the webserver). It won't make sshd more available, but it
will kill off the process automatically if it uses too much resources.

--keith


--
kkeller-usenet@xxxxxxxxxxxxxxxxxxxxxxxxxx
(try just my userid to email me)
AOLSFAQ=http://wombat.san-francisco.ca.us/cgi-bin/fom
see X- headers for PGP signature information

.

Relevant Pages

  • Too Many Open Files --- error 24
    ... I have ulimit for shell /plimit for process (for no of file descr) set ... /etc/system contains (after reboot) ...
    (comp.unix.programmer)
  • Re: ulimit - Bash question
    ... running in a Bash shell might run very slowly if 'ulimit -s' is set to 'unlimited', but run an order of magnitude faster if 'ulimit' is set to some finite value? ... Although, it seems strange that 'unlimited' would cause the stack to grow infinitely at run-time. ...
    (comp.unix.shell)
  • Re: ulimit revisited
    ... | file would be a universal fix, but it appears not to be the case. ... Ulimit applies to the current shell and any descendants. ... and an opportunity for every app to be much slower. ...
    (Fedora)
  • Re: [SLE] file size limit issue
    ... it's not a bash limit. ... by the ulimit command ... This means commands that control such ... parameters must be implemented by the shell, ...
    (SuSE)
  • Re: ulimit open file descriptor.
    ... ulimit -n 32765 then exit, as shell is exit, the value is lost. ... Then don't exit the shell. ... Use 'su' to become root, change ulimit, then exec a shell ... The wrapper should set the ulimit to what ...
    (comp.unix.programmer)