Feb-28-2017, 04:46 AM
I don't know if Python is a suitable language for what I want to do. In summary:
Read a file of about 6GB into an array (list I presume). The array/list has 6,000,000 entries each of about 1,000 bytes containing a mix of text, integer and real numbers.
Process the data. Loading it all into RAM is very desirable because effectively random subsets of the data must be processed up to about 1,000 times. Anyway, the subsets are very large.
In my youth, about 50 years ago, I used Fortran, Algol and Pascal. All of these have no important problems, but I was curious to learn something of Python to see if a claim made in a tutorial, that it can be used for anything, is correct.
Comments?
Read a file of about 6GB into an array (list I presume). The array/list has 6,000,000 entries each of about 1,000 bytes containing a mix of text, integer and real numbers.
Process the data. Loading it all into RAM is very desirable because effectively random subsets of the data must be processed up to about 1,000 times. Anyway, the subsets are very large.
In my youth, about 50 years ago, I used Fortran, Algol and Pascal. All of these have no important problems, but I was curious to learn something of Python to see if a claim made in a tutorial, that it can be used for anything, is correct.
Comments?