Re: why does scp not use all my cores on a multi-cpu machine?



Rahul wrote:
On a unloaded multi-cpu machine why does scp still use a single processor when copying a bunch of files? Isn't this an "embarassingly parallel" problem? There is no link betweek encrypting file1 and file2 is there? So why not use all my cores / cpus? I used top to check the loads and only one cpu seems to be in play.

Does this need to be enabled somehow or is the current behavour the desirable one?

When I ran my little test earlier, I observed that while I typed scp, both scp and ssh got "energized" and in fact ssh used more time than scp. Since there were two processes (at each end), they might well have been assigned to two processors. Neither of those process is, I imagine, written in a multithreaded way, but each gets its own slot in the process table, so the kernel scheduler is free to assign them to different processors, especially if any of them are relatively idle.

Now since encrypting and transmitting (ssh) has some elements of paralellity (if there were such a word), it might be of interest to have a separate process for each, and I do not believe that is how ssh currently works.

I did notice that both scp and ssh stopped using any cpu time while the network kept busy for quite a while afterwards. I infer that ssh finished its work quite a bit before the network could take itall.

Note that on my system, the receiving computer had only a 100 MHz NIC, and that one of its processors was nealy saturated.

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 16:20:01 up 34 days, 22:26, 4 users, load average: 4.63, 4.40, 4.44
.