Python Forum
Reading floats and ints from csv-like file
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Reading floats and ints from csv-like file
#8
import csv
import os

filename  = os.path.abspath('/media/dfuu/Data/mt4/EU-1H.txt')

def read_csv_data(filename):
    try:
        buffer = None
        with open(filename) as csvfile:
            buffer = csv.reader(csvfile, delimiter=',')
            for row in buffer:
                print(row)
    except csv.Error:
        print('Error reading {filename}')
   
if __name__ == '__main__':
    read_csv_data(filename)
and terminal is:
dfuu@pupute:/media/dfuu/Data/mt4$ python reader.py 
Error reading {filename}
dfuu@pupute:/media/dfuu/Data/mt4$ ls
EU-1H.txt  reader.py  reader.py~
I was expecting it wouldn't change anything as the directory I'm in with my terminal is the working directory in general. I'm really suspecting a problem with my input file which is not a true csv (I don't know what a "true csv" really is, I just described above how I wrote the file using another language, that other language being mql5 with which everything seems a pain to do... Unfortunately I don't have access to another language to extract this data. there is an option in it to write data to files in csv format but it was making ridiculous things and I also had that issue of the first few messed up characters, so I decided to make a txt output formated as csv).
I think it would be easier to attach the file so that you can see what it is to work with that.
I'm trying atm to work with the other lines and not the first one, but I can't find how to convert a single one to a float. This is getting ridiculous, I didn't expect to be stuck for 10 hours to write numbers in a file and retrieving them with python...

Edit: I just went to ask that other program to write data in "csv" format.
1.12035	1.11749	7979
1.11812	1.11597	18181
1.13148	1.11499	30360
1.13176	1.12344	57786
1.12441	1.11997	24175
1.12261	1.12067	14455
1.12466	1.12198	10255
1.12643	1.12209	29588
1.12887	1.12095	33628
1.12243	1.11729	35058
1.11896	1.11737	12049
1.11971	1.11828	8376
1.11934	1.11714	9451
1.12193	1.11732	26088
Data looks like this. Your above code raises the same error.

Okay, last post in there, and the bottom line:

When I open the data file in emacs and copy/paste the content in a leafpad, then save it, all the propositions above works (mine included).
It appears my data file was encoded in iso435143625642562 which led to character representation unbearable apparently.
Apparently, other users of mql5 have stopped trying to open files for writing in utf8 (when I said it had the reputation of making everything become a pain...)
I'll try to see if there is a way in python to reencode this stuff properly. There should be.
Thanks a lot for your time and effort, it was greatly appreciated :)
Reply


Messages In This Thread
RE: Reading floats and ints from csv-like file - by Krookroo - Sep-04-2017, 11:42 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
Sad problems with reading csv file. MassiJames 3 791 Nov-16-2023, 03:41 PM
Last Post: snippsat
  When is it safe to compare (==) two floats? Radical 4 877 Nov-12-2023, 11:53 AM
Last Post: PyDan
  Reading a file name fron a folder on my desktop Fiona 4 1,054 Aug-23-2023, 11:11 AM
Last Post: Axel_Erfurt
  Reading data from excel file –> process it >>then write to another excel output file Jennifer_Jone 0 1,215 Mar-14-2023, 07:59 PM
Last Post: Jennifer_Jone
  Reading a file JonWayn 3 1,185 Dec-30-2022, 10:18 AM
Last Post: ibreeden
  Reading Specific Rows In a CSV File finndude 3 1,073 Dec-13-2022, 03:19 PM
Last Post: finndude
  Excel file reading problem max70990 1 959 Dec-11-2022, 07:00 PM
Last Post: deanhystad
  Replace columns indexes reading a XSLX file Larry1888 2 1,057 Nov-18-2022, 10:16 PM
Last Post: Pedroski55
  [split] why can't i create a list of numbers (ints) with random.randrange() astral_travel 7 1,668 Oct-23-2022, 11:13 PM
Last Post: Pedroski55
  Failing reading a file and cannot exit it... tester_V 8 1,916 Aug-19-2022, 10:27 PM
Last Post: tester_V

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020