Jan-26-2023, 10:23 AM
I have multiple files, all containing 10s of thousands of lines of data. I cannot alter the way the data is presented and it appears as seen in the attached screenshot.
The first 2 lines can be ignored; I am only interested in the lines beginning with "hydro".
Currently, my program reads the data in from each file selected as and when I select a graph...
I did consider reading the data into a list as and when each file was selected, but this causes a delay between selecting a file and being able to select the next file, as the data is read into the list.
I thought of reading in on a separate thread, but then how do I ensure that the data added to the list is in the correct order?
I would appreciate any and all suggestions on how I might approach this problem.
The first 2 lines can be ignored; I am only interested in the lines beginning with "hydro".
Currently, my program reads the data in from each file selected as and when I select a graph...
for line in file: if keyword.casefold() in line.casefold(): time = self.convert(line[21:35]) # Timestep delta_t = self.convert(line[36:50]) # Change in timestep mass = self.convert(line[51:65]) # Mass of star (excluding envelope) radius = self.convert(line[66:80]) # Radius of star (excluding envelope) lum_core = self.convert(line[81:95]) # Luminosity of star (excluding envelope) lum_tot = self.convert(line[96:110]) # Total luminosity (including enveloping cloud) flux = self.convert(line[111:125]) # Mass flux ratio = float(line[125:137]) # Ratio of star mass against mass of the Sun # Store the data into the Numpy array 'data' data = np.append(data, np.array([[time, delta_t, mass, radius, lum_core, lum_tot, flux, ratio]]), axis = 0)The problem here is that this takes some time (several minutes) and is required each time I create a new graph based upon this data.
I did consider reading the data into a list as and when each file was selected, but this causes a delay between selecting a file and being able to select the next file, as the data is read into the list.
I thought of reading in on a separate thread, but then how do I ensure that the data added to the list is in the correct order?
I would appreciate any and all suggestions on how I might approach this problem.