Dealing with multiple context managers - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: Dealing with multiple context managers (/thread-14102.html) |
Dealing with multiple context managers - heras - Nov-14-2018 Hi, How do you structure code around multiple context managers? I looking to make something like this: #main.py import file_io.py import rest_api.py main(): main program flow here read something from a log file GET something from a REST API write it to the log file POST something to the REST API #file_io.py #maybe this could be a class def read(): with open('log.txt') as log_file: read last entry in log_file return entry def write(something): with open('log.txt') as log_file: write something to log_file #rest_api.py #maybe this could be a class too token = 'super secret' header = {"Authorization":"Bearer {}".format(token)} url = 'api.some_site/endpoint' with requests.Session() as rest_session: rest_session.headers.update(headers) GET/PUT/POST stuff depending on main() return data to mainI can live with opening and closing a file all the time for every operation, but can I avoid stuffing all my main() logic in the API context manager? Maybe at some point there will be another website with a different API that the program also needs to work with. Or is this a case where you wouldn't wrap the requests.Session() in a context manager and pass the session object around as needed while making sure you catch all exception? Sorry if this is a bit fuzzy. Please let me know if you need more clarification. RE: Dealing with multiple context managers - nilamo - Nov-14-2018 What about passing the other functions a file object it can read/write to, instead of having it open it's own files? Something like with requests.Session() as rest_session: with open("log.txt", "a") as log: data = rest_session.get_data() unrelated = read(log) write(data, log) if unrelated == "OK": rest_session.POST("spam") RE: Dealing with multiple context managers - heras - Nov-14-2018 Thanks for your suggestion nilamo. So the unavoidable seems to be that you have to pick one master context and embed all the lesser contexts in it? I will give it a try but I think it will be difficult to cleanly separate out logic bits. As for example in the case of an aditional website with a different API or some third case where a context manager is recommended. RE: Dealing with multiple context managers - DeaD_EyE - Nov-15-2018 You can write this with one level of indentation: with requests.Session() as rest_session, open("log.txt", "a") as log: data = rest_session.get_data() unrelated = read(log) write(data, log) if unrelated == "OK": rest_session.POST("spam")Since some Python 3.x version it's possible. Before this was introduced, it was also possible with a helper function from the contextlib. https://docs.python.org/3/library/contextlib.html#contextlib.ExitStack By the way, it's good to read the whole document. You'll find functions like suppress, closing. etc. RE: Dealing with multiple context managers - heras - Nov-15-2018 Thanks, so they can essentially share a context manager. Maybe I can also use asynccontextmanager to get the structure I want. RE: Dealing with multiple context managers - DeaD_EyE - Nov-16-2018 (Nov-15-2018, 04:57 PM)heras Wrote: Maybe I can also use asynccontextmanager to get the structure I want. I guess you can do this. Using await in the context, requires to be inside a async function.EDIT: You can't mix synchronous and asynchronous context managers :-( You can stack them only. Calling a blocking synchronous function in async code, will block the whole event-loop. What works: In [35]: @contextlib.asynccontextmanager ...: async def async_ctx(): ...: print('Entering async context') ...: await asyncio.sleep(5) ...: yield ...: print('Leaving async context') ...: In [36]: @contextlib.contextmanager ...: def sync_ctx(): ...: print('Entering sync context') ...: yield 42 ...: print('Leaving sync context') In [37]: async def run(): ...: async with async_ctx(): ...: with sync_ctx() as sync_ret: ...: print(sync_ret) In [38]: asyncio.run(run()) Entering async context Entering sync context 42 Leaving sync context Leaving async context |