Hello again,
this is a repost of the former "When will the GIL be removed?" thread, which has been lost by the hack attack.
There have been a number very useful links to videos of the Pycon Conference 2016 and especially from the Gilectomy project guy. I hope, they will be posted again, that future readers of this thread will have the same chances to follow the discussion. I got the following out of this videos:
1. The general interpreter lock (GIL) locks all (especially internal) variables for others threads.
2. This makes sequential python code run fast.
3. At the same time it inhibits threads to be (naturally) processed in parallel on the CPU, which means, threaded python code only runs on one CPU core, the other cores remain unused. (Recent CPUs have 4 to 8 cores).
4. The removal of the GIL (gilectomy) is a big deal, because the straight forward mulitiple interpreter locks approach following the CPython garbage collection framework of reference counting is slow. Thus, most probably garbage collection/ reference counting also must be redesigned in one breath.
(Please correct me, where I am wrong).
I am a numerical python programmer and I need parallel execution of my programs. So, how are you dealing with the current GIL situation, when you need parallel python execution, so that you won't need to reprogram all your programs in python 4.0, when the GIL is possibly removed and the dirty GIL workaround by the package multiprocessing is dispensible. AFAIK calling a process instead of calling a thread makes up a lot of additional processing ballast.
Consuli
P.S.:
In the former thread, there has been a very nice code example using the package concurrent.futures or similar, posted by one of the mods. Would be nice to be linked again.
this is a repost of the former "When will the GIL be removed?" thread, which has been lost by the hack attack.
There have been a number very useful links to videos of the Pycon Conference 2016 and especially from the Gilectomy project guy. I hope, they will be posted again, that future readers of this thread will have the same chances to follow the discussion. I got the following out of this videos:
1. The general interpreter lock (GIL) locks all (especially internal) variables for others threads.
2. This makes sequential python code run fast.
3. At the same time it inhibits threads to be (naturally) processed in parallel on the CPU, which means, threaded python code only runs on one CPU core, the other cores remain unused. (Recent CPUs have 4 to 8 cores).
4. The removal of the GIL (gilectomy) is a big deal, because the straight forward mulitiple interpreter locks approach following the CPython garbage collection framework of reference counting is slow. Thus, most probably garbage collection/ reference counting also must be redesigned in one breath.
(Please correct me, where I am wrong).
I am a numerical python programmer and I need parallel execution of my programs. So, how are you dealing with the current GIL situation, when you need parallel python execution, so that you won't need to reprogram all your programs in python 4.0, when the GIL is possibly removed and the dirty GIL workaround by the package multiprocessing is dispensible. AFAIK calling a process instead of calling a thread makes up a lot of additional processing ballast.
Consuli
P.S.:
In the former thread, there has been a very nice code example using the package concurrent.futures or similar, posted by one of the mods. Would be nice to be linked again.