Python Forum

Python 3.7.1

I use multiprocessing.Pool for paralleling my code, and it work good. But when I add contextvars in working module (for my coroutines, which I created 2 for each process):

import contextvars
line_var = contextvars.ContextVar('my_var')

def my_func():
    line = 1
    line_by_ticker_var.set(line)
    line2 = line_var.get()

I got this error:

Error:
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7fe233cccf90>'. Reason: 'TypeError("can't pickle ContextVar objects")'

How can I use multiprocessing and asynchrony (with the necessary module contextvars) at the same time?

https://docs.python.org/3.7/library/mult...guidelines
https://docs.python.org/3.7/library/mult...-processes

Objects shared between different processes need to be pickleable. ContextVars are apparently not pickleable. So I think the answer to your question is simply: "you can't".

Mixing multiprocessing and asyncio is not very easy.
(contextvars was made for asyncio)

Use for process communication Queues, Manager (with namespace, list, dict).
Inside the process you should use contextvars for asyncio.

(Oct-30-2019, 07:16 PM)DeaD_EyE Wrote: [ -> ]Inside the process you should use contextvars for asyncio.

If I use contextvars, I get the error from topic.