Python Forum
reading 2 files concurrently
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
reading 2 files concurrently
#1
i am trying to find the Pythonic way(s) (or popular way(s)) to read from two files or file descriptors at the same time, in such a way that the caller can determine which it came from. i have done this many times in C using poll() or select() to wait for on of the file descriptors to be ready. i suppose i could do it the C way in Python, but i want to see if Python has something better. i am also interested in cases with more than two files or file descriptors but if there is a special way that is specific to exactly two files or file descriptors i am interested in that, too. googling for this just gets answers for questions about iterating two or more files.

use cases:

1. each file is a different socket with a different network connection.

2. each file is a different pipe with the write end being on different processes.

3. two files that are pipes to the same process, one getting stdout, the other getting stderr.

4. two or more files that are (or may be) on different disk devices.

a nice solution might be a class which is given a list of open files. a .read method would return a tuple with the file index and the data (maybe a line or a block or a blob or even a single character). whichever file had data ready to read next would be the one it reads from; its index and data would then be returned. it should also have a timeout means (if no data was available when the timeout event happens it would return (-1,...) to the caller).
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
though not exactly what i was looking for, module selectors looks like the way to make a C-like start solution in Python. the class i was suggesting in the main post could be implemented around this. maybe it's on me to do that.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#3
Threads. You can name the thread and you can provide that name along with the data.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#4
the logic i need still needs to have this in a single loop. threads can't do that. the selectors module looks very promising.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Reading txt files in ssh server. therenaydin 6 315 Aug-06-2020, 05:17 AM
Last Post: ndc85430
  How to stop the reading of files Laplace12 4 318 Jul-27-2020, 04:07 PM
Last Post: Gribouillis
  How can I speed up my openpyxl program reading Excel .xlsx files? deac33 0 425 May-04-2020, 08:02 PM
Last Post: deac33
  Other modules for reading audio files? jedzz 0 328 Mar-25-2020, 11:07 PM
Last Post: jedzz
  Error With Reading Files In Directory And Calculating Values chascp 2 384 Feb-15-2020, 01:57 PM
Last Post: chascp
  Reading DBF files from Amazon s3 throwing error - 'int' object has no attribute 'isa abeesm 1 643 Sep-22-2019, 05:49 PM
Last Post: ndc85430
  Reading and writing files JakeHoward4 1 542 Aug-07-2019, 06:22 PM
Last Post: Yoriz
  Reading PCAP FIles Variables 5 2,672 Apr-26-2019, 06:05 AM
Last Post: buran
  Python 2.7.13 Issue Reading .txt files Properly username1145 3 796 Mar-24-2019, 03:08 PM
Last Post: username1145
  Watching for inputs from multiple sources concurrently anakaine 4 786 Mar-03-2019, 07:38 PM
Last Post: woooee

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020