readline() and readlines()

[rpaskudniak]

Greetings.

This is my first thread on this forum. I am learning Python from a book named "Learn to Program Using Python" (I am a VERY experienced programmer and DBA, BTW.) In the chapter on I/O I wasted no time diverging from the book's example.  My code reads a line from a file and then spits it out with a line count and length.  Here is the code, with my line numbers:

  1 #!/usr/bin/python
  2 # files1.py - first program to play with files
  3 #
  4 filename = "justsome.txt"
  5
  6 inp = open(filename, "r")
  7 lc = 0
  8
  9 for one_line in inp.readlines() :
 10   lc += 1     # Bump up lines-read count
 11   one_line = one_line.rstrip('\n')
 12   ll = len(one_line)
 13   print "Input line[%d] is %d chars: %s" % (lc, ll, one_line)
 14
 15 # Done printing...
 16 #
 17 inp.close

Here, the output looks just as expected:
$ ./files1.py

Input line[1] is 22 chars: This is the first line
Input line[2] is 18 chars: and this is second
Input line[3] is 35 chars: I'm a little teapot short and stout
Input line[4] is 36 chars: Pour me out into the forest primeval
Input line[5] is 36 chars: THe murmuing poines and the hemlocks

Next, the book discusses using readline() rather than readlines().  Of course I gotta try that (in line 9). And of course it barfs in my face.  Here is the output of that:
Input line[1] is 1 chars: T
Input line[2] is 1 chars: h
Input line[3] is 1 chars: i
Input line[4] is 1 chars: s
Input line[5] is 1 chars:
Input line[6] is 1 chars: i
Input line[7] is 1 chars: s
Input line[8] is 1 chars:
Input line[9] is 1 chars: t
Input line[10] is 1 chars: h
Input line[11] is 1 chars: e
Input line[12] is 1 chars:
Input line[13] is 1 chars: f
Input line[14] is 1 chars: i
Input line[15] is 1 chars: r
Input line[16] is 1 chars: s
Input line[17] is 1 chars: t
Input line[18] is 1 chars:
Input line[19] is 1 chars: l
Input line[20] is 1 chars: i
Input line[21] is 1 chars: n
Input line[22] is 1 chars: e
Input line[23] is 0 chars:

There are two weirdnesses about this:

1. It is accepting only one character at a time.  I have googled this issue and found the answers wanting.  And searching this forum with so general a string will get me a haystack when I need the needle.
   
2. It stopped reading after the first line.  Otherwise the output would have been a line for every blessed character in the input file.
   

So my obvious questions are:

• How do I make readline() read a full line at each call?
  
• Why did it stop after the first line?
  
• Not that serious here but why isn't the len() function a string method?
  

Thank you.

-- Rasputin (my middle name is Concise ) Paskudniak

[metulburr]

9 for one_line in inp.readlines() :

im assuming you just change it from pl;ural to singular on this line?

If so when you are looping readlines() you are getting each line on each iteration. Whereas if you omit the s you are looping char by char of the line.

readlines() just returns the entire file on a list with each index split by newline. readline() returns the line the file pointer is on, then sets it to the next line. It reads one line at a time.

https://docs.python.org/3/tutorial/input...le-objects

[rpaskudniak]

metulburr said:

readline() returns the line the file pointer is on, then sets it to the next line. It reads one line at a time.

Indeed, when I go interactive, here's the first line of activity:

>>> filename = "justsome.txt"
>>> inp = open(filename, "r")
>>> lc = 0
>>> one_line = inp.readline()
>>> lc += 1
>>> one_line = one_line.rstrip('\n') # Chomp (For Perl geeks)
>>> ll = len(one_line)
>>> print "Input line[%d] is %d chars: %s" % (lc, ll, one_line)
Input line[1] is 22 chars: This is the first line

So I can see that readline() does indeed read a whole line. Exactly as documented. But when I ran the list object in the for loop , it retrieved only one character at a time from the method.  This appears to go against the documentation.  But it gave me a glimmer of a suspicion.  Observe:

>>> for myline in "qwerty" :
...   print myline
...
q
w
e
r
t
y

AHA! It was taking the first input line as a list of characters, one at a time, rather than a whole string object!  It also starts to explain why it stopped after pulling in that first line.

So let me rephrase the question:  OK, readline() reads an entire line at each call.  But when returning values to the caller, readline() is treating that line as a string, returning one character per call, as in the querty example above. How can I make readline() actually return to the caller that entire blessed line as a string object when I use it as the list in the FOR loop?  And then proceed to the next line, of course.

Alternatively, perhaps readline() is simply the wrong method to call in the context of the loop header. Among all those gadzillion read methods available to the file object, which one will return to the caller a complete line, without reading the entire file into memory (as readlines() would do)?

-- Rasputin (my middle name is Concise  ) Paskudniak

[metulburr]

readline is the least used. You would only use it in the event the file was too large that readlines() is too much to put into RAM or too slow to do so. And then you would most likely use a database instead. You can also control how many bytes are read if you give it an argument.

You wouldnt use readline in a for loop. Something like this

f = open('text.txt')
line = f.readline()
while line:
    print(line, end='')
    line = f.readline()

where readlines would be

with open('text.txt') as f:
    lines = f.readlines()

Even if you wanted to read the first 5 lines of a file would would just splice readlines()

[iFunKtion]

Hi there, it seems that the readline/readlines conundrum is actually cleared up in this thread, so I would like to clarify why you were getting a letter at a time, perhaps for my own clarification, who knows.

>>> for myline in "qwerty" :
...   print myline
...
q
w
e
r
t
y

With python, you create variables on the fly, so what has happened here, is you created a variable "myline", and as you are looping through something, python looks for a list or tuple or dictionary, otherwise it will split up text into characters, if you want to print text in either words or lines (other than using readline) you can use split:

some_text = "This is a string of words to be split up individually"
a_list = some_text.split()

for a_word in a_list:
    print(a_word)

this will split up a sentence into words turn them into a list and then print out a word at a time. I think this can be a little confusing at first, but is actually a really nice way of working with text as you can use split to split text up with most characters and punctuation should you want to, something like:

some_text.split(",") 

or

some_text.split("w")

if you want to. Hope this helps.

[buran]

if f is the file object:
f.readline() - reads one line and returns str. So when you iterate over f.readline(), you iterate over the string, returned by calling it, thus it yields one char at a time.
f.readlines() - reads all lines from the file and returns a list (i.e. each line is a separate element). So when you iterate over f.readlines() you iterate over a list, thus it yield one element at a time.

Also, you don't need to use f.readlines() at all and to read all the file in the memory. You can just iterate over the file object:

with open('yourfile.txt', 'r') as f:
    for line in f: # iterate over the file object, without reading the whole content in the memory
        print(line)

[rpaskudniak]

iFunKtion and metulburr,

Thanks for the explanations. Indeed, I have changed to code to use a WHILE loop. Here is the new code:

1 #!/usr/bin/python
2 # files1.py - first program to play with files
3 #
4 filename = "justsome.txt"
5
6 inp = open(filename, "r")
7 lc = 0
8
9 one_line = "Anything, just not null :-)"
10 #for one_line in inp.readline() :
11 while one_line :
12   one_line = inp.readline()
13   one_line = one_line.rstrip('\n')
14   ll = len(one_line)
15   if ll < 1 :
16     break
17   lc += 1 # Not null: Bump up lines-read count
18   print "Input line[%d] is %d chars: %s" % (lc, ll, one_line)
19
20 # Done printing...
21 #
22 inp.close

And voila! the output matches exactly what the readlines() version produced.  Actually, slightly better, since the new version skips the final blank line that I got from readlines().

So now I can mark this thread as "Answered" but I don't see that option. Thanks SO much, folks, for clarifying a confusing basic issue!

[snippsat]

It work,try not to copy line numbers 
There are stuff that can be improved quite a lot if we talk about Pythonic code.
Some point with open() close file-object automatic 
enumerate() is important when want to use or manipulate index in a loop.
The old string formatting (%s%d) should not be used anymore.

with open('justsome.txt') as f:
    for line_nr,line in enumerate(f, 1):
        line = line.strip()
        print('Input [{}] is {} chars: {}'.format(line_nr,len(line), line))

Output:
Input [1] is 22 chars: This is the first line
Input [2] is 18 chars: and this is second
Input [3] is 35 chars: I'm a little teapot short and stout
Input [4] is 36 chars: Pour me out into the forest primeval
Input [5] is 36 chars: THe murmuing poines and the hemlocks

The advisable version is Python 3.6,that also has f-string.
It's make code even nicer than with format().

with open('justsome.txt') as f:
    for line_nr,line in enumerate(f, 1):
        line = line.strip()
        print(f'Input [{line_nr}] is {len(line)} chars: {line}')

[rpaskudniak]

snippsat,

That formatting style is an eye-opener for me   !  Much thanks!

Like I said in my initial post, I am just picking up Python for the first time and I used what the book used.  I hope the next book I pick up describes these constructs better.

All that said, how do mark this thread as [Answered]?  The PerlGuru forum has a button to click but I do't see that here.

Thanks all!  I hope to pose smarter questions as I go along.

(Hey that was a short post!  What's the matter with me?  )

-- Rasputin P.

[metulburr]

There are a lot more formatting options you can do besides just plugging in values into a string
https://python-forum.io/Thread-Basic-str...xpressions