+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: reading 2 files concurrently (/thread-10692.html)
reading 2 files concurrently - Skaperen - Jun-01-2018
i am trying to find the Pythonic way(s) (or popular way(s)) to read from two files or file descriptors at the same time, in such a way that the caller can determine which it came from. i have done this many times in C using poll() or select() to wait for on of the file descriptors to be ready. i suppose i could do it the C way in Python, but i want to see if Python has something better. i am also interested in cases with more than two files or file descriptors but if there is a special way that is specific to exactly two files or file descriptors i am interested in that, too. googling for this just gets answers for questions about iterating two or more files.
use cases:
1. each file is a different socket with a different network connection.
2. each file is a different pipe with the write end being on different processes.
3. two files that are pipes to the same process, one getting stdout, the other getting stderr.
4. two or more files that are (or may be) on different disk devices.
a nice solution might be a class which is given a list of open files. a .read method would return a tuple with the file index and the data (maybe a line or a block or a blob or even a single character). whichever file had data ready to read next would be the one it reads from; its index and data would then be returned. it should also have a timeout means (if no data was available when the timeout event happens it would return (-1,...) to the caller).
RE: reading 2 files concurrently - Skaperen - Jun-01-2018
though not exactly what i was looking for, module selectors looks like the way to make a C-like start solution in Python. the class i was suggesting in the main post could be implemented around this. maybe it's on me to do that.
RE: reading 2 files concurrently - wavic - Jun-01-2018
Threads. You can name the thread and you can provide that name along with the data.
RE: reading 2 files concurrently - Skaperen - Jun-01-2018
the logic i need still needs to have this in a single loop. threads can't do that. the selectors module looks very promising.