How to understand asyncio.gather return_exceptions=False?

How to understand asyncio.gather return_exceptions=False? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: How to understand asyncio.gather return_exceptions=False? (/thread-21211.html)

How to understand asyncio.gather return_exceptions=False? - learnpython - Sep-19-2019

In doc of python 3.7.4, the information about asyncio.gather return_exceptions=False says

"If return_exceptions is False (default), the first raised exception is immediately propagated to the task that awaits on gather(). Other awaitables in the aws sequence won’t be cancelled and will continue to run."

I made a simple code like below:

import asyncio, time

async def fn1(x):
    await asyncio.sleep(3)
    print(x)
    return "fn1"

async def fn2(y):
    await asyncio.sleep(2)
    raise asyncio.TimeoutError()
    print(y)
    return "fn2"

async def main():
    print("start:",time.ctime())

    await asyncio.gather(
        fn1("fn1"),
        fn2("fn2"),
        return_exceptions=False,
    )
    print("end:",time.ctime())


asyncio.run(main())

the terminal printed(whole information):

Error:start: Thu Sep 19 17:13:36 2019
Traceback (most recent call last):
  File "aio.py", line 25, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "aio.py", line 20, in main
    return_exceptions=False,
  File "aio.py", line 10, in fn2
    raise asyncio.TimeoutError()
concurrent.futures._base.TimeoutError  #nothing below

when run this code ,after asyncio.TimeoutError() raised, the other awaitable funtion ("fn1") did't continue working. program quited.this is not worked as the doc says.
Is there anything wrong ? or how to understand the doc correctly?

RE: How to understand asyncio.gather return_exceptions=False? - DeaD_EyE - Sep-19-2019

In the documentation it's described as:

Quote:If return_exceptions is False (default), the first raised exception is immediately propagated to the task that awaits on gather(). Other awaitables in the aws sequence won’t be cancelled and will continue to run.

Explained with my own words: If a awaitable raises an error, you won't get results back, the tasks are not cancelled, but the caller have to catch the error.

Quote:If return_exceptions is True, exceptions are treated the same as successful results, and aggregated in the result list.

Here both versions explained with code:

import asyncio, time


async def fn1(x):
    await asyncio.sleep(3)
    print(f"fn1 is called with x={x}")
    return f"Hello from fn1: {x}"


async def raise_error(y):
    await asyncio.sleep(2)
    print(f"raise_error is called with y={y}")
    raise asyncio.TimeoutError() # <- function exits here
    # no result
    return f"Hello from raise_error"


async def main():
    print("start:",time.ctime())
    print("return_exceptions=False")
    try:
        result1 = await asyncio.gather(
            fn1("First call"), # <- 1st result
            raise_error("Second call"), # <- here is the error
            fn1("Third call"), # <- 3rd result
            return_exceptions=False, # <- don't return result, if an exception was raised
            # but tasks are not canceled. You can see it with the print function, that
            # fn1 is still called, but you won't get a return value from gather
            )
    except Exception as e:
        print("Catched Exception from gather")
        print("We don't get the result back, but the tasks are not canceled")
    
    # result1 does not exist
    try:
        print('Result1', result1)
    except NameError:
        print('As expected you get a NameError, result1 does not exist')
    
    print()
    print('Sleeping 2 seconds')
    await asyncio.sleep(2)
    print()
    print('return_exceptions=True')
    
    result2 = await asyncio.gather(
            fn1("First call"), # <- 1st result
            raise_error("Second call"), # <- here is the error
            fn1("Third call"), # <- 3rd result will evaluated aswell
            return_exceptions=True, # <- return exceptions as results
        )
    print("result2:", result2)
    await asyncio.sleep(2)
    print("end:",time.ctime())
    


asyncio.run(main())

RE: How to understand asyncio.gather return_exceptions=False? - learnpython - Sep-20-2019

Thanks for your helping!

I have ran your code.

Firstly, if set

 asyncio.gather(*aws, loop = None, return_exceptions = True)

then we can get the Exception as an object in the result list. that's a good choice to know which awaitable task or coroutine had an exception after asyncio running.

Secondly, if set

 asyncio.gather(*aws, loop = None, return_exceptions = False)

after the asyncio.TimeoutError() raised, "fn1 is called with First call" and "fn1 is called with Third call" both sentences didn't print yet.

terminal's output information is as below:

Output:#nothing above
start: Thu Sep 19 21:24:37 2019
return_exceptions=False
raise_error is called with y=Second call
Catched Exception from gather
We don't get the result back, but the tasks are not canceled
As expected you get a NameError, result1 does not exist   
# nothing below

what makes me confused is on my understanding about the doc

Quote:"If return_exceptions is False (default), the first raised exception is immediately propagated to the task that awaits on gather(). Other awaitables in the aws sequence won’t be cancelled and will continue to run."

https://docs.python.org/3/library/asyncio-task.html#running-tasks-concurrently

if there is an Exception in one of the awaitable tasks or coroutines, the others awatiable tasks will still run. In the examples you gave,

 
async def fn1(x):
    await asyncio.sleep(3)
    print(f"fn1 is called with x={x}") # from the doc says, I think this sentence will still run
    return f"Hello from fn1: {x}"

the print sentence will be running and give a printing on terminal. but nothing happended in the termial.