Python Forum
How to timeout a task using the ThreadpoolExecutor?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to timeout a task using the ThreadpoolExecutor?
#1
Hi all.

I'm wonder what are the possible approaches to handle timeout of a task that was submitted to an executor ThreadPoolExecutor.
I know I can timeout while getting the result like so future.result(timeout=x). But I don't understand how can I timeout a task, in a non-blocking way, when I'm on "fire and forget" mode.

Example:

# Send the task to the pool
future = self.executor.submit(mytask, arg)
future.add_done_callback(...)

# By this time, I can `get` the result with timeout
future.result(timeout=xxx)
# Send the task to the pool
future = self.executor.submit(mytask, arg)
future.add_done_callback(...)

# By this time, I can `get` the result with timeout
future.result(timeout=xxx)
But if I call the result, I'll block, and I won't be able to send more tasks to the pool.
What I'm trying to do is something like:

# Notice the timeout would be defined when sending the task, and not when getting the result
future = self.executor.submit(mytask, arg, timeout=10)

# Assign the callback normally
future.add_done_callback(...)

# Here, I don't care about the result, I want to be able to submit another task
?
I though about opening a new "maintenance" thread, that only waits for the results of all tasks, but again, wouldn't I be blocking for each call?
Say I have 100 tasks periodically being sent to the executor, each just time.sleep(10). How can I make sure that none of those tasks pass 10 seconds?
If it helps, the context is a task queue, hence the "fire and forget" logic.

Any help/suggestion is welcome.
Thanks a lot!
Reply


Messages In This Thread
How to timeout a task using the ThreadpoolExecutor? - by lowercase00 - Feb-07-2023, 04:38 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  count certain task in task manager[solved] kucingkembar 2 1,173 Aug-29-2022, 05:57 PM
Last Post: kucingkembar
  FTp timeout except korenron 2 3,642 Feb-01-2022, 06:51 AM
Last Post: korenron
  Schedule a task and render/ use the result of the task in any given time klllmmm 2 2,147 May-04-2021, 10:17 AM
Last Post: klllmmm
  How to create a task/import a task(task scheduler) using python Tyrel 7 3,808 Feb-11-2021, 11:45 AM
Last Post: Tyrel
  Trying to understand concurrent.futures.ThreadPoolExecutor Laxminarsaiah 0 1,638 Dec-18-2019, 12:43 PM
Last Post: Laxminarsaiah
  ThreadPoolExecutor read file to list DaLiPy 3 6,315 Jun-11-2019, 05:55 AM
Last Post: DaLiPy
  timeout value in subprocess jonesin1974 2 5,092 Dec-01-2017, 02:18 PM
Last Post: snippsat
  AsyncSSH and timeout Standard_user 1 5,542 Nov-03-2016, 06:05 PM
Last Post: micseydel

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020