Python Forum
Reading floats and ints from csv-like file
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Reading floats and ints from csv-like file
#1
Hello :)

I have stored numbers in a file from another program, and I'd like to extract the values in lists to use with another python program I already wrote, here is how the data file looks like:

��1.12005000,1.11800000,14574
1.11947000,1.11811000,8285
1.12035000,1.11749000,7979
1.11812000,1.11597000,18181
1.13148000,1.11499000,30360
1.13176000,1.12344000,57786
1.12441000,1.11997000,24175
1.12261000,1.12067000,14455
1.12466000,1.12198000,10255
1.12643000,1.12209000,29588
For some unknown reason i have these strange characters at the beginning. I came up with the following code to get the values and get rid of the commas and EOL characters:

f = open('EU-1H.txt', 'r')
a = f.readline()
a = a[2:-1] #removing the two odd characters from he first line as well as the EOL character
fullarray, t = [], []

for numb in a[:-1].split(','):
    #print numb
    t.append(numb)
#print t
fullarray.append(t)

for i, line in enumerate(f):
    t = []
    for numb in line[:-1].split(','): #just making life easier by removing the EOL
        t.append(numb)
    fullarray.append(t)
    if i == 10: break #I don't need to process the whole thousands of lines to know if the code works

#print fullarray
To describe my issue, we'll just run this with the first two "print", namely "print numb" and "print t". This outputs:

1.12005000
1.11800000
14574
['1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00', '\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00', '\x001\x004\x005\x007\x004\x00\r']
And if I try to convert to float by replacing t.append(numb) with t.append(float(numb)) I get the following output:
1.12005000
Traceback (most recent call last):
  File "reader.py", line 9, in <module>
    t.append(float(numb))
ValueError: invalid literal for float(): 1
To make my life easier, I wondered if I coult convert it back to a string inside the list to make the conversion to float/int later. So I tried by changing t.append(float(numb)) to t.append(str(numb)) which yielded as output:
1.12005000
1.11800000
14574
['1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00', '\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00', '\x001\x004\x005\x007\x004\x00\r']
For some reasons, some of this code actually works when run directly in an interpreter in a terminal. Example:

>>> a = '1.12005000,1.11800000,14574' #copypasted from the source file
>>> a
'1.12005000,1.11800000,14574'
>>> a.split(',')
['1.12005000', '1.11800000', '14574']
But if I read the data from the data file:
f = open('EU-1H.txt', 'r')
a = f.readline()
a = a[2:-1]
print a
print a.split(',')
gives when run:
1.12005000,1.11800000,14574
['1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00', '\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00', '\x001\x004\x005\x007\x004\x00\r\x00']
In interpreter:
>>> f = open('EU-1H.txt', 'r')
>>> a = f.readline()
>>> a[2:-1]
'1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00,\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00,\x001\x004\x005\x007\x004\x00\r\x00'
>>> print a[2:-1]
1.12005000,1.11800000,14574
>>> a[2:-1].split(',')
['1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00', '\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00', '\x001\x004\x005\x007\x004\x00\r\x00']
>>> print a[2:-1].split(',')
['1\x00.\x001\x002\x000\x000\x005\x000\x000\x000\x00', '\x001\x00.\x001\x001\x008\x000\x000\x000\x000\x000\x00', '\x001\x004\x005\x007\x004\x00\r\x00']
I have a lot of troubles understanding what's happening there.
My main goal is to just have something working (I expected to take no more than 10mn to write the storing algorithm and the data-retrieving one, apparently I couln't be more wrong, nothing went right), but I'd also like to understand what is going on obviously (so that I don't get stuck into this kind of issues again)
Any hints?
Thanks in advance.

Python 2.7.9 / Debian Jessie / i686
Reply


Messages In This Thread
Reading floats and ints from csv-like file - by Krookroo - Sep-04-2017, 05:06 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
Sad problems with reading csv file. MassiJames 3 658 Nov-16-2023, 03:41 PM
Last Post: snippsat
  When is it safe to compare (==) two floats? Radical 4 745 Nov-12-2023, 11:53 AM
Last Post: PyDan
  Reading a file name fron a folder on my desktop Fiona 4 931 Aug-23-2023, 11:11 AM
Last Post: Axel_Erfurt
  Reading data from excel file –> process it >>then write to another excel output file Jennifer_Jone 0 1,119 Mar-14-2023, 07:59 PM
Last Post: Jennifer_Jone
  Reading a file JonWayn 3 1,113 Dec-30-2022, 10:18 AM
Last Post: ibreeden
  Reading Specific Rows In a CSV File finndude 3 1,004 Dec-13-2022, 03:19 PM
Last Post: finndude
  Excel file reading problem max70990 1 906 Dec-11-2022, 07:00 PM
Last Post: deanhystad
  Replace columns indexes reading a XSLX file Larry1888 2 998 Nov-18-2022, 10:16 PM
Last Post: Pedroski55
  [split] why can't i create a list of numbers (ints) with random.randrange() astral_travel 7 1,531 Oct-23-2022, 11:13 PM
Last Post: Pedroski55
  Failing reading a file and cannot exit it... tester_V 8 1,838 Aug-19-2022, 10:27 PM
Last Post: tester_V

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020