Python Forum
read_csv error and rows/columns missing
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
read_csv error and rows/columns missing
#7
Here's a sample of what the actual data looks like, and some code that will read it:
raw data sample:
Output:
0631960537; 10.01.1990 09:35:37;Intern;0x2207;Batt-SN ungueltig 0631960539; 10.01.1990 09:35:39;CHRG;0x0400;Netzfrequenz zu klein 0631960539; 10.01.1990 09:35:39;CHRG;0x0402;Netzspg. zu klein 0631960539; 10.01.1990 09:35:39;CHRG;0x0607;UBatt Min 0631960539; 10.01.1990 09:35:39;CHRG;0x0616;Uzwk Min 0631963784; 10.01.1990 10:29:44;Intern;0x2207;Batt-SN ungueltig 0631963787; 10.01.1990 10:29:47;CHRG;0x0400;Netzfrequenz zu klein 0631963787; 10.01.1990 10:29:47;CHRG;0x0402;Netzspg. zu klein 0631963905; 10.01.1990 10:31:45;Intern;0x2207;Batt-SN ungueltig 0631963909; 10.01.1990 10:31:49;CHRG;0x0400;Netzfrequenz zu klein;631963907: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 18 BF1: 00041213 BF2: 00100140 Ph: 0 LN: 0 Fan: 342 0631963909; 10.01.1990 10:31:49;CHRG;0x0402;Netzspg. zu klein;631963909: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 911 BF1: 00041813 BF2: 00100140 Ph: 0 LN: 0 Fan: 342 0631964140; 10.01.1990 10:35:40;Intern;0x2207;Batt-SN ungueltig 0631964182; 10.01.1990 10:36:22;WR;0x0515;Spg. N->PE Fehler;631964180: ENS::St: 30 SF: 0000 BF: 0006 BAS: 0000 BAI: 0000 WR::St: 900 SF: 0020 BF: 0006 BAS: 0000 BAI: 0000 EMS::St: 31 BF1: 00045213 BF2: 00000140 Ph: 0 LN: 0 Fan: 374 0631966026; 10.01.1990 11:07:06;Intern;0x2207;Batt-SN ungueltig 0631966029; 10.01.1990 11:07:09;CHRG;0x0402;Netzspg. zu klein;631966027: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 18 BF1: 00041213 BF2: 00100140 Ph: 0 LN: 0 Fan: 372 0631966029; 10.01.1990 11:07:09;CHRG;0x0403;Netzspg. zu gross 0631966029; 10.01.1990 11:07:09;CHRG;0x061D;Dry-Kontakt;631966029: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 911 BF1: 00041813 BF2: 00100140 Ph: 0 LN: 0 Fan: 372
simple code to read it
Note: ASll this does is read one record at a time into a list named row, and waits for you to press enter before reading next record,
but it reads reliably, so you can modify for your own purposes. Since each record is in a list, you can access individual parts with row[index], index vbeing an integer starting with 0 for first field, or to read all fields one by one just use for field in row:
import csv
import os


class ReadFiles:
    def __init__(self):
        # assure in proper directory
        os.chdir(os.path.abspath(os.path.dirname(__file__)))
        # you can add your file list code here
        list_raw_files = ['130000054.csv']
        for filename in list_raw_files:
            self.read_file_data(filename)

    def read_file_data(self, filename):
        with open(filename) as fp:
            csv_reader = crdr = csv.reader(fp, delimiter=';')
            for row in crdr:
                print(row)
                input()

if __name__ == '__main__':
    ReadFiles()
results of first few records:
Output:
['0631960537', ' 10.01.1990 09:35:37', 'Intern', '0x2207', 'Batt-SN ungueltig'] ['0631960539', ' 10.01.1990 09:35:39', 'CHRG', '0x0400', 'Netzfrequenz zu klein'] ['0631960539', ' 10.01.1990 09:35:39', 'CHRG', '0x0402', 'Netzspg. zu klein'] ['0631960539', ' 10.01.1990 09:35:39', 'CHRG', '0x0607', 'UBatt Min'] ['0631960539', ' 10.01.1990 09:35:39', 'CHRG', '0x0616', 'Uzwk Min'] ['0631963784', ' 10.01.1990 10:29:44', 'Intern', '0x2207', 'Batt-SN ungueltig'] ['0631963787', ' 10.01.1990 10:29:47', 'CHRG', '0x0400', 'Netzfrequenz zu klein'] ['0631963787', ' 10.01.1990 10:29:47', 'CHRG', '0x0402', 'Netzspg. zu klein'] ['0631963905', ' 10.01.1990 10:31:45', 'Intern', '0x2207', 'Batt-SN ungueltig'] ['0631963909', ' 10.01.1990 10:31:49', 'CHRG', '0x0400', 'Netzfrequenz zu klein', '631963907: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 18 BF1: 00041213 BF2: 00100140 Ph: 0 LN: 0 Fan: 342'] ['0631963909', ' 10.01.1990 10:31:49', 'CHRG', '0x0402', 'Netzspg. zu klein', '631963909: ENS::St: 30 SF: 0000 BF: 000a BAS: 0000 BAI: 0000 WR::St: 1 SF: 0000 BF: 0005 BAS: 0000 BAI: 0000 EMS::St: 911 BF1: 00041813 BF2: 00100140 Ph: 0 LN: 0 Fan: 342'] ['0631964140', ' 10.01.1990 10:35:40', 'Intern', '0x2207', 'Batt-SN ungueltig']
Reply


Messages In This Thread
RE: read_csv error and rows/columns missing - by Larz60+ - Nov-07-2019, 11:59 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Merging rows and adding columns based on matching index pythonnewbie78 3 865 Dec-24-2023, 11:51 AM
Last Post: Pedroski55
  Pandas read_csv markf7319 0 1,296 Mar-03-2022, 04:59 AM
Last Post: markf7319
  groupby on var with missing values error zenvega 0 1,790 May-07-2021, 07:40 PM
Last Post: zenvega
  pandas read_csv can't handle missing data mrdominikku 0 2,530 Jul-09-2020, 12:26 PM
Last Post: mrdominikku
  Checking a filename before reading it with pd.read_csv karlito 2 2,243 Oct-30-2019, 09:46 AM
Last Post: karlito
  Drop rows if a set of columns has a value dervast 1 2,016 Sep-12-2019, 04:18 PM
Last Post: sd_0912
  display graph in columns and rows william888 1 1,881 Jul-02-2019, 10:19 AM
Last Post: dataman
  utf-8 error with pandas read_csv logues 0 3,891 Oct-23-2018, 05:25 PM
Last Post: logues
  Dropping all rows of multiple columns after the max of one cell Thunberd 2 2,990 Jun-01-2018, 10:18 PM
Last Post: Thunberd
  pandas read_csv, numbers in footer mechanic310 1 2,840 May-22-2018, 10:38 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020