Python Forum
Concurrent futures threading running at same speed as non-threading
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Concurrent futures threading running at same speed as non-threading
#11
import concurrent.futures
from concurrent.futures import ThreadPoolExecutor

def add(args):
    print('inside add')
    return sum(args)

args = [(4, 2), (9, 3), (16, 2)]

with ThreadPoolExecutor(max_workers=5) as executor:
    for result in executor.map(add, args):
        print(result)
Output:
inside add inside add 6 inside add 12 18
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#12
(May-03-2023, 07:40 AM)buran Wrote: Still I think you want executor.submit(i, ConnectHandler, ip), not executor.submit(i(ConnectHandler, ip))

using executor.map would expect that you pass iterable as second argument



                for i in inlistx:
                    executor = ThreadPoolExecutor(max_workers=50)
                    res = executor.submit(i, ConnectHandler,ip)
                    res.result()
It came out to the same 32 seconds as before. I also tried res = executor.submit(i, ConnectHandler) and it parsed correctly at about 30ish seconds. It's okay I tried not the end of the world. I will keep working at it, at least I learned something new.
Reply
#13
Note that it may be multithreading vs multiprocessing issue (i.e. as I am not 100% sure I understand what you do, I don't know if multithreading vs multiprocessing should be used). Try using ProcessPoolExecutor instead
billykid999 likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#14
(May-03-2023, 08:05 AM)buran Wrote: Note that it may be multithreading vs multiprocessing issue (i.e. as I am not 100% sure I understand what you do, I don't know if multithreading vs multiprocessing should be used). Try using ProcessPoolExecutor instead

these are all I/O operations that send commands to devices and wait for a result, so normally that would be that last place I would look at because of what i've read working with network devices ~ I will however tryi it.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  threading native_id returning same value for all threads billykid999 2 1,019 May-04-2023, 06:40 AM
Last Post: billykid999
  Trouble with threading and reading variable from a different script Lembas 14 3,071 Apr-26-2023, 11:21 PM
Last Post: Lembas
  using threading.Timer for function korenron 1 1,207 Dec-20-2022, 01:09 PM
Last Post: ndc85430
  [Solved]Help with Threading Extra 7 1,861 Sep-05-2022, 05:29 PM
Last Post: Extra
Question Opencv and threading ethernel 4 145,432 Feb-25-2022, 06:06 PM
Last Post: ethernel
  Inconsistent counting / timing with threading rantwhy 1 1,774 Nov-24-2021, 04:04 AM
Last Post: deanhystad
  Class variables and Multiprocessing(or concurrent.futures.ProcessPoolExecutor) Tomli 5 3,903 Nov-12-2021, 09:55 PM
Last Post: snippsat
  Mult-threading and locking file mr_byte31 4 2,603 Oct-16-2021, 01:54 AM
Last Post: Larz60+
  Matplotlib Animation with Threading peterjv26 4 7,234 Oct-08-2021, 05:51 PM
Last Post: peterjv26
  Tutorials on sockets, threading and multi-threading? muzikman 2 2,130 Oct-01-2021, 08:32 PM
Last Post: muzikman

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020