Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Suspending worker threads
#1
Question is Semaphore a robust means of temporarily suspending worker threads in the following situation? (free of race conditions etc.) Is there a better way?

Context:
One Python process managing work by several child processes (which happen to be coded in a different language, but that shouldn't matter).
To simplify coding, the Python has a "worker" thread for each child process (giving that child commands and waiting for that child to finish each command).
One shared complicated data structure that can be searched for work to do, and updated for claiming work, or completing work.
It is not generally practical to know how many work requests are scattered around that structure, except when the result is zero.
When completing a work item, it is known whether that completion creates more than one new work items and if so, exactly how many (if not more than 1, unknown whether 0 or 1).

Each worker thread repeatedly:
Lock data structure
   Search for work to do
     if none found
       unlock data structure
       *** possible race condition I'm worrying about ***
       [color=#9B59B6]suspend self[/color]
       when released, start over
     if work found
       update data structure to claim that work
       unlock data structure
       do that work
       lock data structure
       update data structure to show that work is done (maybe causing new work)
       if N greater than 1 items of work caused
          [color=#9B59B6]Attempt to wake up N-1 suspended workers[/color]
       unlock data structure
       start over
I'm not worried that a lot of processing is done with that data structure locked (nor that Python multi-threading isn't really multi-threading), because almost all the total CPU time is in the child processes inside the "do that work" portion of this loop.

Using a thread.Semaphore (acquire to suspend and release to awaken) seems to be sometimes wrong in a safe direction that only wastes a little CPU time, and never wrong in the unsafe direction (leave work waiting with workers suspended).

We might have more workers awake than work to do, either because we start with more workers than work, or because a worker finishes an item that creates zero new items, and then looks for the next thing to do. But I think we never waste serious CPU time spinning: Seeing there is nothing to do, soon results in really suspending.
We might wake up a worker, but because of temporary excess workers at that moment, the worker just woken finds nothing to do. That seems to be just a special case of the above, not a serious race condition to worry about.

Worker X might unlock the data structure for lack of work, then worker Y updates to add work and fails to wake any others up, because none were suspended, then worker X reaches the suspend, despite there being work. This is the problem race condition for most conceptual forms of suspend/wake.

My understanding of thread.Semaphore documentation is that it simply covers that scenario with no problem and the race condition doesn't matter. When X suspends right after Y tries to wake X up, the wake up still has the desired effect (X doesn't get suspended). But in the actual program, that is too rare for practical testing to validate.

Am I on the right track? (It is coded and working already, but testing whether it is really robust doesn't seem to be practical).
Reply
#2
Semaphores will allow for synchronization of processes, and may suit your needs.

I suggest downloading and reading the following:
Summary: The Little Book of Semaphores
PDF: http://greenteapress.com/semaphores/Litt...phores.pdf
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Receive data from Redis RQ worker process freak14 0 1,866 Jul-15-2019, 12:39 PM
Last Post: freak14
  Sending a custom signal from a worker mhc 4 4,722 Jul-03-2018, 04:05 PM
Last Post: mhc
  Good way to have a worker queue accessible by multiple process? cheater 2 2,478 Dec-21-2017, 09:30 PM
Last Post: wavic
  Worker thread? micko 1 3,000 Feb-27-2017, 09:52 AM
Last Post: Ofnuts

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020