Python Forum
Reading an Unconventional CSV file
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Reading an Unconventional CSV file
#1
Hi Everyone,

I'm killing myself to find a way to read a CSV file and convert to a format I can import to InfluxDB.

The CSV has a very unconventional format and it grows horizontally. I tried to make sense of reading it with the CSV module and PANDAS but no luck in making sense of it. The file is an export from the Xiaomi Mi Flower Care sensor (example below). And I want to load the date into a InfluxDB while I learn how to do that using Phyton.

An example of he file is:
And the format I need the output is:
Output:
Device Column 1 (L) Column 2(S) Column 3 (T) Column 4 (E) Time finger light=74,moisture=61,temperature=11.3,fertility=812 1557784799999750000 finger light=244,moisture=61,temperature=12.3,fertility=798 1557788399999750000 finger light=325,moisture=62,temperature=14.2,fertility=773 1557791999999750000 finger light=379,moisture=62,temperature=16.2,fertility=753 1557795599999740000 finger light=495,moisture=63,temperature=18.2,fertility=739 1557799199999740000 finger light=436,moisture=63,temperature=18.3,fertility=750 1557802799999740000 finger light=264,moisture=63,temperature=17.8,fertility=772 1557806399999740000 finger light=323,moisture=63,temperature=17.7,fertility=785 1557809999999740000 finger light=217,moisture=63,temperature=17.5,fertility=796 1557813599999740000 finger light=37,moisture=63,temperature=16.6,fertility=815 1557817199999740000 finger light=4,moisture=63,temperature=15.6,fertility=829 1557820799999740000 finger light=5,moisture=62,temperature=14.9,fertility=839 1557824399999740000 finger light=5,moisture=62,temperature=13.6,fertility=854 1557827999999740000 finger light=5,moisture=61,temperature=11.7,fertility=872 1557831599999740000 finger light=5,moisture=61,temperature=10.9,fertility=873 1557835199999740000 finger light=5,moisture=61,temperature=10.7,fertility=861 1557838799999740000 finger light=5,moisture=61,temperature=11.1,fertility=839 1557842399999740000 finger light=5,moisture=60,temperature=11.5,fertility=825 1557845999999740000 finger light=6,moisture=60,temperature=11.9,fertility=810 1557849599999740000 finger light=5,moisture=60,temperature=11.9,fertility=805 1557853199999740000 finger light=5,moisture=60,temperature=11.5,fertility=805 1557856799999740000 finger light=6,moisture=60,temperature=12.2,fertility=789 1557860399999740000 finger light=5,moisture=60,temperature=12.3,fertility=783 1557863999999740000 finger light=7,moisture=60,temperature=12.2,fertility=782 1557867599999740000
O tried the CSV module but couldn't find a way to read the file that would make sense.

Any help?

Thanks
Reply


Messages In This Thread
Reading an Unconventional CSV file - by OzSbk - May-17-2019, 01:13 AM
RE: Reading an Unconventional CSV file - by buran - May-17-2019, 08:14 AM
RE: Reading an Unconventional CSV file - by MvGulik - May-17-2019, 12:15 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Excel File reading vanjoe198 1 2,007 Mar-31-2021, 11:53 AM
Last Post: snippsat
  reading from a file looseCannon101 14 4,754 Jul-18-2020, 11:29 AM
Last Post: GOTO10
  Weird problem with reading from file and performing calculations pineapple999 1 2,956 Jul-25-2019, 01:30 AM
Last Post: ichabod801
  Handling IO Error / Reading from file Expel 10 4,719 Jul-18-2019, 01:21 PM
Last Post: snippsat
  reading text file and writing to an output file precedded by line numbers kannan 7 10,247 Dec-11-2018, 02:19 PM
Last Post: ichabod801
  Reading of structured .mat (matlab) file sumit 2 3,369 May-24-2018, 12:12 PM
Last Post: sumit
  File Reading toxicxarrow 9 5,104 May-07-2018, 04:12 PM
Last Post: toxicxarrow
  reading all lines from a text file seadoofanatic 2 2,889 Mar-13-2018, 06:05 PM
Last Post: Narsimhachary
  Reading a text file fivestar 7 5,509 Oct-13-2017, 07:25 AM
Last Post: gruntfutuk
  Dictionary + File Reading palmtrees 2 4,818 Nov-15-2016, 05:16 PM
Last Post: Ofnuts

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020