Python Forum

Hello

I want to import data in Arangodb so I need .json files. All of my data are in .xlsx file (big ones 10 to 20 MB each) with 25 to 35 sheets each. So I created a loop with this code:

#sheets is alist with all the sheet names from the file 
for i in sheets:
    df = pd.read_excel(file, sheet_name = i, index = None, header = 1)
    json_file = df.to_json(("{}.json").format(i))

I have the following questions:
1. The code works I have some of the .json files to prove it, but after a few of the sheets the program crashes. Can anyone help with why is this happening? I am using jupyter notebook and I got an error that thee kernel is dead.

2. Obviously when the loop is running it reads every time the .xlsx file and creates the new dataframe. Is there a way to load the file only once in the memory and get the dataframes from that instead of loading it every time ?

For debugging I would comment out line 4 to see if the crash is happening on reading or writing. Then I would add "print(i)" between lines 3 and 4 to see if a particular sheet is causing the choke.

(Apr-03-2020, 05:59 PM)jefsummers Wrote: [ -> ]For debugging I would comment out line 4 to see if the crash is happening on reading or writing. Then I would add "print(i)" between lines 3 and 4 to see if a particular sheet is causing the choke.

Thank you for your answer.

Unfortunately I don't need debugging. I already know that the crashing happens while reading the file for the n-th time when it starts to read a particularly big sheet. I know this because when I run the loop it always crashes at the 10th sheet (which is one of the biggest) after it has written in .json files the first nine sheets of the spreadsheet.