Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
MATLAB to Python conversion
#1
Hi - I am new to Python and am converting some MATLAB code over. I have a question about converting MATLAB nested variables (is that the right term for this?) with varying sizes and dimensions into Python. Is there a way in Python to set up my variable below, filedata(x).shiftinfo.numapples(y,z), equivalently in Python? I'm not sure how to ask without an example. My stripped-down MATLAB example code is below.

files = ['C:\temp\day1.dat','C:\temp\day2.dat'];   % Files to read (1 file for each day apple pickers worked)
for x = 1:numel(files)                             % Loop over each file
   [fid,msg] = fopen(files(x),'r','b');            % Open for reading binary data
   numShifts(x) = fread(fid,1,'uint32');           % Read # of worker shifts on that day
   for y = 1:numShifts(x)                          % Now loop over the shifts
      filedata(x).numFarmers(y) = fread(fid,1,'uint32');  % Read number of farmers working that shift
      for z = 1:filedata(x).numFarmers(y)          % Now read number of apples picked by each farmer on that shift.
         filedata(x).shiftinfo.numapples(y,z) = fread(fid,1,'uint32'); 
      end
   end
end
Right now I'm just storing one set of z values at a time and bookkeeping x,y indices for that entry. Is there a better way?
Reply
#2
TL;DR version:
Can I replicate the following MATLAB variable somehow in Python, where ny can change with x and nz can change with y?
var(x).inner.innerinner(y,z)
Reply
#3
I am not MATLAB person so for me it would better to explain what do you want to accomplish instead of providing code.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#4
and in addition to what @pefringo said - show the format of input file.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#5
I believe the input file is a binary file, with only 32-bit unsigned integers. (I am new to Matlab so don't be cross with me if I am wrong.)
There are several ways to store the data. I would advise to create a 3-dimensional dictionary. But one can argue about that. Perhaps creating a class for storing the data is even more elegant.
As far as I understand your Matlab code I believe you could write it in this way in Python.
import struct                                      # for reading binary data
files = ['C:\temp\day1.dat','C:\temp\day2.dat'];   # Files to read (1 file for each day apple pickers worked)
filedata = {}                                      # prepare empty dictionary for filedata
for file in files:                                 # Loop over each file
   filedata[file] = {}                             # Create new dict for this file
   fid = open(file,'rb')                           # Open for reading binary data
   numShifts = struct.unpack('i', fid.read(4))     # Read # of worker shifts on that day
   for shift in range(1, numShifts):               # Now loop over the shifts
      filedata[file][shift] = {}                   # Create new dict for this shift
      numFarmers = struct.unpack('i', fid.read(4)) # Read number of farmers working that shift
      for farmer in range(1, numFarmers):          # Now read number of apples picked by each farmer on that shift.
         filedata[file][shift][farmer] = struct.unpack('i', fid.read(4))
   close(fid)
I have no data to test it so there may be errors in it. In this case the numShifts and numFarmers are not stored in the 3-D dictionary but I believe you only need them in the process.
I hope I understood your problem right.
Reply
#6
Thank you for the help! I appreciate the time. That example nailed what I am trying to do perfectly.

The format of the actual input file consists of a mix of u8, u4, f8, and f4 variables in multi-frame image data with image header data then image data for each frame. There's about 250 header variables in total before each frame of image data. Some header data are single values and some are vectors, and they are stored in different sub-levels of header information.

The struct.unpack() looks like a good alternative to what I've been trying - variable.append(np.fromfile()) - which had some limitations for setting up arrays of different dimensionality.

The only issue I see is that some variables may need to be indexed as vectors later on - I just need to figure out converting the resulting dictionaries to vectors for this approach as needed - like converting
filedata[file][shift][:], or
filedata[:][numshifts]
to vectors. I'm not sure about the easiest way to retrieve those two examples of vectors.
Reply
#7
Looking this over again, I'm not sure that will work because I have multiple variables at each level of the hierarchy. This solution just seems to assign values to indices alone, not allowing for multiple sets of variables at each level of the dictionary - which would work if I only have one variable at each level of the tree. Perhaps I need to use nested classes. The problem is I can't figure out how to properly pose the problem.
Reply
#8
again, post example input file. where it comes from (what software generated it)? explain the data in the file. desribe what you want to achieve in plain english, not in matlab concepts.
at the moment you try to mimic matlab code verbatim, but python has different programming paradigms/patterns.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#9
Input file is 10GB so I cannot post. I explained what's in the actual data 2 or 3 posts up.

In my simplified apple picking example, the file would have number of shifts as uint32.
Within that it would have number of workers per shift as uint32.
For each worker in a shift, I define the number of hours they work for that shift (float), number of apples they pick (uint32), and worker's GPS location (float * 2).
The file might also have the number of trucks used in the fleet for a given shift (uint16)
For each truck, I might have amount of gasoline (float), GPS location (float x2), and number of miles it has on it (float).
And there is a manager employee ID associated with the file.

So I need these variables
file.numshifts
file.shift.numworkers
file.shift.worker.numapples
file.shift.worker.hours
file.shift.worker.location [x,y]
file.shift.numtrucks
file.shift.truck.gasoline
file.shift.truck.location [x,y]
file.shift.truck.miles
file.managerid
Reply
#10
As I said before: there are a lot of ways to store data. Some are more legible than others. I always prefer the most legible solution.
What do you think about the following solution? (It is the same principle as I offered before.)
>>> filedata = {}
>>> filedata['file1'] = {}
>>> filedata['file1']['numShifts'] = 8
>>> filedata['file1']['numWorkers'] = 5
>>> filedata['file1']['worker1'] = {}
>>> filedata['file1']['worker1']['hours'] = 8
>>> filedata['file1']['worker1']['location'] = [47.123, 15.123]
>>> filedata['file1']['numTrucks'] = 3
>>> filedata['file1']['truck1'] = {}
>>> filedata['file1']['truck1']['gasoline'] = 150
>>> filedata['file1']['truck1']['location'] = [47.123, 15.123]
>>> filedata['file1']['truck1']['miles'] = 15
>>> filedata['file1']['managerid'] = 'Mickey Mouse'
>>> 
>>> filedata
{'file1': {'numShifts': 8, 'numWorkers': 5, 'worker1': {'hours': 8, 'location': [47.123, 15.123]}, 'numTrucks': 3, 'truck1': {'gasoline': 150, 'location': [47.123, 15.123], 'miles': 15}, 'managerid': 'Mickey Mouse'}}
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  I need to add data types to cython conversion python to c Good_AI_User 1 973 Aug-19-2022, 07:52 AM
Last Post: Gribouillis
Photo Matlab to Python Sateesh 0 1,636 Nov-04-2020, 09:11 AM
Last Post: Sateesh
  From Matlab to Python erbab 1 1,970 Oct-27-2020, 02:16 PM
Last Post: jefsummers
  Arrays in MATLAB and PYTHON cokhuatlanyeuthuongconhetmuc 2 2,156 Jul-24-2020, 10:47 AM
Last Post: cokhuatlanyeuthuongconhetmuc
  Conversion of Oracle PL/SQL(packages, functions, procedures) to python modules. DivyaKumar 2 6,413 Jul-09-2020, 04:46 PM
Last Post: srikanth7482
  Matlab to Python -- Parallel Computing zistambo 1 1,934 Jun-10-2020, 04:59 PM
Last Post: pyzyx3qwerty
  C to Python code conversion print problem anakk1n 1 2,137 May-22-2020, 04:15 PM
Last Post: deanhystad
  Python uppercase conversion conditions Jaypeng 7 2,893 Apr-29-2020, 11:24 AM
Last Post: jefsummers
  Python v MatLab for graphs and plots CynthiaMoore 4 2,979 Apr-22-2020, 02:13 PM
Last Post: CynthiaMoore
  python equivalent to MATLAB xcov chai0404 2 3,817 Apr-02-2020, 10:29 PM
Last Post: chai0404

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020