Oct-07-2019, 08:02 AM
(Oct-06-2019, 06:48 PM)wavic Wrote: I have somewhere a script which downloads in parallel more than 200 web pages and I tested it on 4 core system with a different number of processes. The best time I got was with around 30 processes set. If I found it, I can run the script here again to see what is going on and share it here. This system has 6 cores though.
I have no I / O limit, because the speed of the algorithm does not depend on the use of a HDD or SSD (in the C- program I had 4x acceleration on SSD). Therefore, starting processes more than the number of cores does not make sense.