Python Forum
Dealing with multiple context managers
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Dealing with multiple context managers
#1
Hi,

How do you structure code around multiple context managers? I looking to make something like this:
#main.py
import file_io.py
import rest_api.py
main():
    main program flow here
    read something from a log file
    GET something from a REST API
    write it to the log file
    POST something to the REST API
#file_io.py
#maybe this could be a class
def read():
    with open('log.txt') as log_file:
        read last entry in log_file
        return entry

def write(something):
    with open('log.txt') as log_file:
        write something to log_file
#rest_api.py
#maybe this could be a class too
token = 'super secret'
header = {"Authorization":"Bearer {}".format(token)}
url = 'api.some_site/endpoint'

with requests.Session() as rest_session:
    rest_session.headers.update(headers)
    GET/PUT/POST stuff depending on main()
    return data to main
I can live with opening and closing a file all the time for every operation, but can I avoid stuffing all my main() logic in the API context manager? Maybe at some point there will be another website with a different API that the program also needs to work with. Or is this a case where you wouldn't wrap the requests.Session() in a context manager and pass the session object around as needed while making sure you catch all exception? Sorry if this is a bit fuzzy. Please let me know if you need more clarification.
Reply
#2
What about passing the other functions a file object it can read/write to, instead of having it open it's own files? Something like
with requests.Session() as rest_session:
    with open("log.txt", "a") as log:
        data = rest_session.get_data()
        unrelated = read(log)
        write(data, log)
        if unrelated == "OK":
            rest_session.POST("spam")
Reply
#3
Thanks for your suggestion nilamo. So the unavoidable seems to be that you have to pick one master context and embed all the lesser contexts in it? I will give it a try but I think it will be difficult to cleanly separate out logic bits. As for example in the case of an aditional website with a different API or some third case where a context manager is recommended.
Reply
#4
You can write this with one level of indentation:

with requests.Session() as rest_session, open("log.txt", "a") as log:
    data = rest_session.get_data()
    unrelated = read(log)
    write(data, log)
    if unrelated == "OK":
        rest_session.POST("spam")
Since some Python 3.x version it's possible. Before this was introduced, it was also possible with a helper function from the contextlib.
https://docs.python.org/3/library/contex....ExitStack
By the way, it's good to read the whole document. You'll find functions like suppress, closing. etc.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#5
Thanks, so they can essentially share a context manager. Maybe I can also use asynccontextmanager to get the structure I want.
Reply
#6
(Nov-15-2018, 04:57 PM)heras Wrote: Maybe I can also use asynccontextmanager to get the structure I want.

I guess you can do this. Using await in the context, requires to be inside a async function.

EDIT:

You can't mix synchronous and asynchronous context managers :-(
You can stack them only.
Calling a blocking synchronous function in async code, will block the whole event-loop.

What works:
In [35]: @contextlib.asynccontextmanager 
    ...: async def async_ctx(): 
    ...:     print('Entering async context') 
    ...:     await asyncio.sleep(5) 
    ...:     yield 
    ...:     print('Leaving async context') 
    ...:                                                                        

In [36]: @contextlib.contextmanager 
    ...: def sync_ctx(): 
    ...:     print('Entering sync context') 
    ...:     yield 42 
    ...:     print('Leaving sync context')                                      

In [37]: async def run(): 
    ...:     async with async_ctx(): 
    ...:         with sync_ctx() as sync_ret: 
    ...:             print(sync_ret)                                            

In [38]: asyncio.run(run())                                                     
Entering async context
Entering sync context
42
Leaving sync context
Leaving async context
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How does open context manager work? deanhystad 7 345 Nov-08-2022, 02:45 PM
Last Post: deanhystad
  UnicodeEncodeError - Dealing with Japanese Characters fioranosnake 2 455 Jul-07-2022, 08:43 PM
Last Post: fioranosnake
  Decimal context stevendaprano 1 468 Apr-11-2022, 09:44 PM
Last Post: deanhystad
  Dealing with duplicated data in a CSV file bts001 10 5,494 Sep-06-2021, 12:11 AM
Last Post: SamHobbs
  TextIOWrapper.tell() with Python 3.6.9 in context of 0D/0A fschaef 0 1,545 Mar-29-2020, 09:18 AM
Last Post: fschaef
  Dealing with a .json nightmare... ideas? t4keheart 10 2,949 Jan-28-2020, 10:12 PM
Last Post: t4keheart
  Time execution of a "for loop" with zip different in 2 equivalent context sebastien 1 1,342 Oct-11-2019, 11:07 AM
Last Post: sebastien
  Is it OK to use a context manager to simplify attribute access? nholtz 0 1,618 Jun-11-2019, 01:19 AM
Last Post: nholtz
  Dealing with Exponential data parthi1705 11 7,132 May-30-2019, 10:16 AM
Last Post: buran
  Smtplib: What does context argument means? Pythenx 1 2,410 Mar-27-2019, 06:25 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020