Python Forum
Class variables and Multiprocessing(or concurrent.futures.ProcessPoolExecutor)
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Class variables and Multiprocessing(or concurrent.futures.ProcessPoolExecutor)
#1
Hello everyone!

The last code that I was working on was very taxing and slow, so I decided to optimize it by using multiprocessing. When I did this, I discovered that somehow class variables that have been changed in code before calling ProcessPoolExecutor aren't taken into account when instantiating the processes. As an example, I prepared the following code:
import concurrent.futures

class test():
    class_variable = None

    def test_method(self,iter):
        print(test.class_variable)
    
if __name__=='__main__':
    test.class_variable = 1
    inst = test()
    print(f"Class variable before pool: {test.class_variable=}")
    with concurrent.futures.ProcessPoolExecutor() as executor:
        executor.map(inst.test_method,[x for x in range(5)])
If you run this code, the results are:
Class variable before pool: test.class_variable=1
test.class_variable=None
test.class_variable=None
test.class_variable=None
test.class_variable=None
test.class_variable=None

As I said at the start, even if we change the class variable before using multiprocessing they will be ignored and the code will use the original value (None in this case).

I wanted to know if someone knows a way to make the processes take into account the change in class variables.
Reply


Messages In This Thread
Class variables and Multiprocessing(or concurrent.futures.ProcessPoolExecutor) - by Tomli - Nov-11-2021, 09:17 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Unchangeable variables in a class? Calab 12 1,541 Sep-15-2023, 07:15 PM
Last Post: deanhystad
  Concurrent futures threading running at same speed as non-threading billykid999 13 1,826 May-03-2023, 08:22 AM
Last Post: billykid999
  Weird ProcessPoolExecutor initializer behavior davetapley 2 1,663 Mar-13-2023, 06:49 PM
Last Post: davetapley
  PyRun_SimpleFile calling multiprocessing Python Class cause endless init loop Xeno 2 1,046 Sep-19-2022, 02:32 AM
Last Post: Xeno
  How to pass variables from one class to another hobbyist 18 10,719 Oct-01-2021, 05:54 PM
Last Post: deanhystad
  Problem with concurrent.futures. thunderspeed 3 2,054 Sep-01-2021, 05:21 PM
Last Post: snippsat
  concurrent.futures help samuelbachorik 2 1,749 Aug-22-2021, 07:20 AM
Last Post: bowlofred
  Acess variables from class samuelbachorik 3 1,899 Aug-20-2021, 02:55 PM
Last Post: deanhystad
  Multiprocessing, class, run and a Queue Object SeanInColo 0 1,544 Jul-12-2020, 05:36 PM
Last Post: SeanInColo
  Class variables menator01 2 2,008 Jun-04-2020, 04:23 PM
Last Post: Yoriz

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020