Python Forum
multiprocessing Pipe.poll very slow
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
multiprocessing Pipe.poll very slow
#1
I reduced the cpu usage of my program by more than half by using select.poll rather than the Pipe's own poll routine:

in __init__
...
        self.pollin = select.poll()
        self.pollin.register(self.pipe, select.POLLIN)
...


    def readline(self):
#        if self.pipe.poll():   # pipe poll is really slow!!!!!  better use our own                                             
        if self.pollin.poll(0):
            return self.pipe.recv()
        return False
I measured the time from before and after calling pipe.poll and it's more than 10 times slower than using the select poll routine but does exactly the same thing from what I can tell except I have to pass 0 to avoid blocking.


Where is the actual implementation? So why would this be so slow?
Reply


Messages In This Thread
multiprocessing Pipe.poll very slow - by seandepagnier - Mar-09-2020, 03:10 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Convert Excel file into csv with Pipe symbol.. mg24 4 1,360 Oct-18-2022, 02:59 PM
Last Post: Larz60+
  BrokenPipeError: [Errno 32] Broken pipe throwaway34 6 9,413 May-06-2021, 05:39 AM
Last Post: throwaway34
  Duplex Named Pipe with Python Server and C# Clients raybowman 1 2,414 Dec-03-2020, 09:58 PM
Last Post: Gribouillis
  2 or more processes on the write end of the same pipe Skaperen 4 3,896 Sep-27-2020, 06:41 PM
Last Post: Skaperen
  STT: recognition connection failed: [Errno 32] Broken pipe GrahamBerends 0 5,101 Jul-18-2020, 11:00 PM
Last Post: GrahamBerends
  Speech (audio file, wav) to Text - Broken pipe Shobha 1 3,796 Nov-27-2018, 12:41 PM
Last Post: Larz60+
  Python tailing file or named pipe stalls after a while nilsk123 4 4,800 Jul-27-2018, 07:14 AM
Last Post: Gribouillis
  should i use os.poll() for 2 reads? Skaperen 2 2,510 Jul-08-2018, 01:52 AM
Last Post: Skaperen
  faster socket with multiprocessing.Pipe sixteenornumber 2 6,624 Dec-10-2016, 06:52 PM
Last Post: sixteenornumber
  Filtering an interactive CLI command via pipe(s) Skaperen 2 4,187 Nov-23-2016, 09:17 AM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020