Python Forum
Read file Into array with just $0d as Newline
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Read file Into array with just $0d as Newline
#1
I have a Textfile that contains several Lines, but just contains $0d as new Line instead of $0d $0a

When i load this into a List with

	with open(filename)as f:
		lines =f.readlines()
	print(lines)
The result is a list (lines) which contains just one Item which looks like :

"8890 \r11.19.19\rGRIDDYP-+1D/EX\rPRG\r46 \r8890 \r11.19.19\rTESTFILE2018\rPRG\r132 \r8890 \r11.19.19\rTESTFILE2019\rPRG\r132"

But how do i convert this to an array which uses the \r as splitter ? or how to read this onto an array with just $0d as new line Indicator ?
Reply
#2
just strip it off, it's not really needed
for line in lines:
    line = line.strip()
Reply
#3
(Feb-03-2020, 04:37 AM)Larz60+ Wrote: just strip it off, it's not really needed
for line in lines:
    line = line.strip()
I tried this before but this results in a single line that contains just

75INK+1H/EX2/GP

The List which i read the previous commands results in a List with just one Item so probably that is the reason why line.strip doesnt work that way. The obove result is just a cut from the List. I attached the file to parse, then you probably may verify what i mean

Attached Files

.txt   ud- 1.txt (Size: 25.18 KB / Downloads: 8)
Reply
#4
Can do a test,look in file you see that there is a Unicode mess in some places.
You can have made this eg not saving in utf-8 or a problem from where it comes from.
Output:
2!ÔÄOWN A+2ÄÆ/ÅØ PRG 171 8890 11.19.19 3!ÔÄOWN A+2ÄÆ/ÅØ
Just ignore this now and replace errors with ?.
lst = []
with open('ud- 1.txt', encoding='utf-8', errors='replace') as f:
    for line in f:
        line = line.strip()
        lst.append(line)
Look okay if this is what you want.
Output:
# Take out first 10 >>> lst[:10] ['8890', '11.19.19', 'GRIDDYP-+1D/EX', 'PRG', '46', '8890', '11.19.19', 'TESTFILE2018', 'PRG', '132'] # Take out last 10 >>> lst[-10:] ['9194', '2.1.20', '4!HAWKSMILL+2/GP', 'PRG', '63', '9197', '2.1.20', 'MBLINK+1H/EX', 'PRG', '75'] # First last >>> lst[0] '8890' >>> lst[-1] '75'
Reply
#5
This is looking good so far. But the unicode mess is from the System where it comes from and sadly has to be saved later on aswell else the names wont be displayed properly

To be specific. The file comes from c64 and i want to reverse the data and write a new file for later use on the c64 again.

So i think i need to write the List back, compare Files and check what i need to do for custom encoding or probably byte exchange the written File to retrieve my Unicode messed Data again Big Grin
Reply
#6
I was testing with this the Evening and it seems that i need to Convert the messy foreign c64 Chars to UTF before processing by the Script.

So my thought was to parse the whole textfile and replace the utf-8 unknown Chars (bytes) with the correct utf-8 Chars

I wanted to create a conversation table which would convert e.g.
193 ($c1) to 64 ($41)
194 ($c2) to 65 ($42)
.....
97 ($61) to 64 ($41)
98 ($62) to 65 ($42)

should this be done best with a external conversation table or parsing the file byte by byte and use some algo that converts ?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas read csv file in 'date/time' chunks MorganSamage 4 1,629 Feb-13-2023, 11:24 AM
Last Post: MorganSamage
  Can't read text file with pandas zinho 6 11,941 May-24-2020, 06:13 AM
Last Post: azajali43
  Read json array data by pandas vipinct 0 1,887 Apr-13-2020, 02:24 PM
Last Post: vipinct
  Read csv file from Yahoo Finance ian 3 4,586 Sep-22-2019, 06:47 AM
Last Post: ndc85430
  read complex file with both pandas and not Diedro 1 2,851 Jan-29-2019, 05:26 PM
Last Post: Larz60+
  read from txt. file in certain manner SchroedingersLion 13 7,322 Aug-07-2018, 04:14 PM
Last Post: SchroedingersLion
  save 2d array to file and load back ian 3 18,122 May-18-2018, 05:00 AM
Last Post: scidam
  loading a 3D array from a bin file Mark3232 1 4,649 May-04-2018, 11:36 AM
Last Post: j.crater
  access a very large file? As an array or as a dataframe? Angelika 5 4,822 May-18-2017, 08:15 AM
Last Post: Angelika
  Need help in framing data read from wav file Vishweshkumar 1 3,613 Feb-10-2017, 01:45 PM
Last Post: sparkz_alot

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020