Python Forum

Hi All,

I have one query that needs to be called from python script. That query will return 2 million records.Is there any document in python which can explain best way to handle /process huge data .

Basically I will prepare excel report with those records.

Is this useful: http://code.activestate.com/recipes/5773...cx_oracle/

Hi Larz ,
Appreciate your effort in providing documents.

As per this document what I am understanding is I should export my data in csv and then I should read the csv file and put whatever logic I want to put further.
When we read from csv we get data in list format and that will be easy to process

Please correct me if my understanding is incorrect

I didn't read the entire document, just providing a link.

No need to write to csv file. Here is Oracle article expalining best practices http://www.oracle.com/technetwork/articl...01587.html
Note the part about cursors being iterator objects.
And useful SO link too
https://stackoverflow.com/questions/594/...result-set

You probably don't really want that many results.  Normal humans can't consume that much data... so modify the query to return paginated results, and only display 100ish at a time?