Python Forum

Hi,

How do you structure code around multiple context managers? I looking to make something like this:

#main.py
import file_io.py
import rest_api.py
main():
    main program flow here
    read something from a log file
    GET something from a REST API
    write it to the log file
    POST something to the REST API

#file_io.py
#maybe this could be a class
def read():
    with open('log.txt') as log_file:
        read last entry in log_file
        return entry

def write(something):
    with open('log.txt') as log_file:
        write something to log_file

#rest_api.py
#maybe this could be a class too
token = 'super secret'
header = {"Authorization":"Bearer {}".format(token)}
url = 'api.some_site/endpoint'

with requests.Session() as rest_session:
    rest_session.headers.update(headers)
    GET/PUT/POST stuff depending on main()
    return data to main

I can live with opening and closing a file all the time for every operation, but can I avoid stuffing all my main() logic in the API context manager? Maybe at some point there will be another website with a different API that the program also needs to work with. Or is this a case where you wouldn't wrap the requests.Session() in a context manager and pass the session object around as needed while making sure you catch all exception? Sorry if this is a bit fuzzy. Please let me know if you need more clarification.

What about passing the other functions a file object it can read/write to, instead of having it open it's own files? Something like

with requests.Session() as rest_session:
    with open("log.txt", "a") as log:
        data = rest_session.get_data()
        unrelated = read(log)
        write(data, log)
        if unrelated == "OK":
            rest_session.POST("spam")

Thanks for your suggestion nilamo. So the unavoidable seems to be that you have to pick one master context and embed all the lesser contexts in it? I will give it a try but I think it will be difficult to cleanly separate out logic bits. As for example in the case of an aditional website with a different API or some third case where a context manager is recommended.

You can write this with one level of indentation:

with requests.Session() as rest_session, open("log.txt", "a") as log:
    data = rest_session.get_data()
    unrelated = read(log)
    write(data, log)
    if unrelated == "OK":
        rest_session.POST("spam")

Since some Python 3.x version it's possible. Before this was introduced, it was also possible with a helper function from the contextlib.
https://docs.python.org/3/library/contex....ExitStack
By the way, it's good to read the whole document. You'll find functions like suppress, closing. etc.

Thanks, so they can essentially share a context manager. Maybe I can also use asynccontextmanager to get the structure I want.

(Nov-15-2018, 04:57 PM)heras Wrote: [ -> ]Maybe I can also use asynccontextmanager to get the structure I want.

I guess you can do this. Using await in the context, requires to be inside a async function.

EDIT:

You can't mix synchronous and asynchronous context managers :-(
You can stack them only.
Calling a blocking synchronous function in async code, will block the whole event-loop.

What works:

In [35]: @contextlib.asynccontextmanager 
    ...: async def async_ctx(): 
    ...:     print('Entering async context') 
    ...:     await asyncio.sleep(5) 
    ...:     yield 
    ...:     print('Leaving async context') 
    ...:                                                                        

In [36]: @contextlib.contextmanager 
    ...: def sync_ctx(): 
    ...:     print('Entering sync context') 
    ...:     yield 42 
    ...:     print('Leaving sync context')                                      

In [37]: async def run(): 
    ...:     async with async_ctx(): 
    ...:         with sync_ctx() as sync_ret: 
    ...:             print(sync_ret)                                            

In [38]: asyncio.run(run())                                                     
Entering async context
Entering sync context
42
Leaving sync context
Leaving async context