Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Exporting a huge dataFrame
#6
Ha thanks, I figured it out. It was kinda simple. Here's the solution for anyone else facing the same problem.

df1 = pd.DataFrame(historicalData[0:1000000])
df2 = pd.DataFrame(historicalData[1000000:2000000])
df3 = pd.DataFrame(historicalData[2000000:3000000])
df4 = pd.DataFrame(historicalData[3000000:4000000])
df5 = pd.DataFrame(historicalData[4000000:5000000])
df6 = pd.DataFrame(historicalData[5000000:6000000])
writer = pd.ExcelWriter('15 Year Backtest.xlsx', engine='xlsxwriter')
df1.to_excel(writer, sheet_name='Sheet1')
df2.to_excel(writer, sheet_name='Sheet2')
df3.to_excel(writer, sheet_name='Sheet3')
df4.to_excel(writer, sheet_name='Sheet4')
df5.to_excel(writer, sheet_name='Sheet5')
df6.to_excel(writer, sheet_name='Sheet6')

writer.save()
Now if I can just sort out this threading problem... Its like after an hour, the program forgets its running... Or to be more accurate, one of the threads just forgets what its supposed to do after a period of time.
Reply


Messages In This Thread
Exporting a huge dataFrame - by stylingpat - Mar-21-2021, 02:34 PM
RE: Exporting a huge dataFrame - by perfringo - Mar-21-2021, 04:10 PM
RE: Exporting a huge dataFrame - by stylingpat - Mar-21-2021, 08:22 PM
RE: Exporting a huge dataFrame - by perfringo - Mar-21-2021, 11:17 PM
RE: Exporting a huge dataFrame - by supuflounder - Mar-22-2021, 09:57 AM
RE: Exporting a huge dataFrame - by stylingpat - Mar-23-2021, 12:13 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Exporting Stock Fundamental Data to a CSV file with yahoo_fin DustinKlent 2 4,730 Aug-01-2022, 06:08 PM
Last Post: paulyan
  Exporting dataframes to excel without loosing datetime format Rafa 0 1,238 Oct-27-2021, 10:42 AM
Last Post: Rafa
  exporting all lines YazeedbnMohmmed 2 2,127 Feb-24-2021, 03:29 AM
Last Post: YazeedbnMohmmed
  Pip prints huge error when installing p5 hayden2s 1 2,254 Aug-08-2020, 02:30 PM
Last Post: snippsat
  Huge CSV Reading and Sorting for comparison akshaynimkar 3 2,484 Aug-04-2020, 11:20 AM
Last Post: Larz60+
  Python code for exporting table using Selenium gj31980 4 3,007 Aug-04-2020, 01:29 AM
Last Post: gj31980
  Exporting data from python into excel Zankawah 5 3,386 Jun-02-2020, 03:17 AM
Last Post: buran
  How to scan huge files and make it in chunks ampai 2 2,606 May-28-2020, 08:20 PM
Last Post: micseydel
  convert huge xml to csv using python srikanta_p 2 2,036 Feb-08-2020, 07:16 PM
Last Post: srikanta_p
  Exporting list with dictionary to Excel veromi22 0 3,050 Oct-15-2019, 12:54 AM
Last Post: veromi22

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020