Re: why does scp not use all my cores on a multi-cpu machine?



Rahul wrote:
On a unloaded multi-cpu machine why does scp still use a single processor when copying a bunch of files? Isn't this an "embarassingly parallel" problem? There is no link betweek encrypting file1 and file2 is there? So why not use all my cores / cpus? I used top to check the loads and only one cpu seems to be in play.

Does this need to be enabled somehow or is the current behavour the desirable one?

When I ran my little test earlier, I observed that while I typed scp, both scp and ssh got "energized" and in fact ssh used more time than scp. Since there were two processes (at each end), they might well have been assigned to two processors. Neither of those process is, I imagine, written in a multithreaded way, but each gets its own slot in the process table, so the kernel scheduler is free to assign them to different processors, especially if any of them are relatively idle.

Now since encrypting and transmitting (ssh) has some elements of paralellity (if there were such a word), it might be of interest to have a separate process for each, and I do not believe that is how ssh currently works.

I did notice that both scp and ssh stopped using any cpu time while the network kept busy for quite a while afterwards. I infer that ssh finished its work quite a bit before the network could take itall.

Note that on my system, the receiving computer had only a 100 MHz NIC, and that one of its processors was nealy saturated.

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 16:20:01 up 34 days, 22:26, 4 users, load average: 4.63, 4.40, 4.44
.



Relevant Pages

  • Re: nice and ssh/scp
    ... >>Is there a way to nice the remote end of an ssh or scp session? ... >>even enrypting for scp. ... the other is CPU bound and ...
    (Debian-User)
  • Re: NAS Drive ?
    ... is singlethreaded so makes no use of multicores and is very CPU ... I've not checked scp, I'd ... On reflection I was probably using rsync over ssh at the time, ... The same applies for ssh - it may have been using a slower crypto algorithm. ...
    (uk.comp.sys.mac)
  • Re: NAS Drive ?
    ... Chris Ridd wrote: ... is singlethreaded so makes no use of multicores and is very CPU ... I've not checked scp, I'd ... The same applies for ssh - it may have been using a slower crypto algorithm. ...
    (uk.comp.sys.mac)
  • Re: Poor PostgreSQL scaling on Linux 2.6.25-rc5 (vs 2.6.22)
    ... 10ms, then if your ssh only needs average of 5ms of CPU time per second, ... CPU time per second, then it has to wait for 2 seconds anyway to be run ... regardless of whether the timeslice was 10ms or 20ms. ... Just because tpc-c runs are set up so the number of server threads ...
    (Linux-Kernel)
  • Re: Is SSH worth it??
    ... > We would be using SSH and SCP. ... SCP for automated scripts. ... > client will not be prompted for a password. ... remote machine, but imho it is better to swap client+server and give ...
    (Security-Basics)