Concurrent futures threading running at same speed as non-threading

[billykid999]

As the below code shows, I was not correct, regarding 1, that you don't execute anything

Actually, I was not correct, regarding 1, so strike that

from concurrent.futures import ThreadPoolExecutor

def add(a, b):
    print('inside add')
    return a + b

def sub(a, b):
    print('inside sub')
    return a - b


def pow(a, b):
    print('inside pow')
    return a ** b
   
functions = [add, sub, pow]
a=9
b=3

with ThreadPoolExecutor(max_workers=5) as executor:
    print('start submit')
    future = [executor.submit(func, a, b) for func in functions]
    print('end submit, get result')
    for result in concurrent.futures.as_completed(future):
        print(result.result())

Output:
start submit
inside add
inside sub
end submit, get result
inside pow
6
12
729

One thing I noticed is it looks like I'm trying to execute different threads underneath a loop which iterates over the infile list 1 by one, I could just parse the entire file, and then run the threads, but I want to add additional connection drivers in the future outside of Netmiko, and the program will choose the best driver depending on conditions. I'm not sure if this would have an effect. I tried using a context manager underneath __name__ == __main__ and no improvement. This may all be due to this is the fastest it is going to run. I'm running about 15-20 commands on a device for 5 devices and each device takes up about 5-7 seconds so maybe this is what should be expected, and threading may be better for a production environment with many devices?

Corrction, there are 9 commands per device in a given class, which equates to about a second a command. Maybe this is the best I will get (it take about 32 seconds for 6 devices).


This iteration wheich I was getting the result for the executor.map() which I assume means I was running it properly, doesn't show an improvement in time.

        for data in self.json_data:
            ip = data["ip"]
            # port = data["port"]
            username = data["username"] if data["username"] else ""
            password = data["password"] if data["password"] else ""
            secret = data["secret"] if "secret" in data else False
            device_type = data["device_type"] if data["device_type"] else ""

            if data["header"] == "Netmiko":
                print("The variables being passed:  " + ip, username, password, device_type)
                ConnectHandler = netmiko.ConnectHandler(
                    device_type=device_type,
                    host=ip,
                    username=username,
                    password=password,
                    port=22,
                    secret=data["secret"] if "secret" in data else False
                )
                if ConnectHandler:
                    try:
                        ConnectHandler.enable()
                    except Exception as e:
                        print("Could not connect to {}".format(ip))
                for i in inlistx:
                    executor = ThreadPoolExecutor(max_workers=50)
                    res = executor.map(i(ConnectHandler, ip))
                    res.result()
                    executor.shutdown()




if __name__ == "__main__":
    from Connect_Handler import Hardware, OSPF
    from Connect_Handler import Connect
    inlist = sys.argv[1]
    outlist = inlist.split(",")
    result = [i.strip("[]") for i in outlist]
    inlist = []
    for i in result:
        inlist.append(functions[i])
    Connector = Connect()
    print(time.perf_counter())
    # print(time.perf_counter())
    Connector.connect_parser(inlist)
    print(time.perf_counter())