Python Forum
TypeError: file must have 'read' and 'readline' attributes
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
TypeError: file must have 'read' and 'readline' attributes
#1
I am trying to make the below code work since morning. I have read tutorials on google about python, I have done 0 progress. Can you help me with this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 65, in <module>
    dict1 = pickle.load(f1,encoding='bytes')
TypeError: file must have 'read' and 'readline' attributes
The code that explodes is this part:

class load_train_data:
    os.open("/home/mojito/Desktop/CNN/datasets/training_data/images", os.O_RDONLY)
    def __enter__(self):
        return self
    def __exit__(self, type, value, traceback):
        pass


class load_test_data:
    os.open("/home/mojito/Desktop/CNN/datasets/test_data/images",os.O_RDONLY) 
    def __enter__(self):
        return self
    def __exit__(self, type, value, traceback):
        pass


with load_train_data() as f1:
   dict1 = pickle.load(f1,encoding='bytes')
I tried to fix this with the code below and I got a new error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 74, in <module>
    with open_train_data() as f1:
  File "source_code_modified.py", line 47, in open_train_data
    return open('/home/mojito/Desktop/CNN/datasets/training_data/images','rb') 
IsADirectoryError: [Errno 21] Is a directory: '/home/mojito/Desktop/CNN/datasets/training_data/images'
the code explodes in this part:

def open_train_data():
    return open('/home/mojito/Desktop/CNN/datasets/training_data/images','rb') <--- explodes here

def open_test_data():
    return open('/home/mojito/Desktop/CNN/datasets/test_data/images','rb')

with open_train_data() as f1:
   dict1 = pickle.load(f1)   <--- explodes here
What I am trying to do is to load a custom dataset of images for a CNN stored to my PC in the format (trainX, trainY), (testX, testY) = ...

Another question: Can I transform the folder with the images to a pickle (.P) file?
Reply
#2
What it looks like you are trying to do is 'open' a directory to get all the images, so you can use them elsewhere.
I would recommend the glob module for that.
import glob

def open_train_data():
    return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters
 
for f in open_train_data():
   dict1 = pickle.load(f)
   ###do whatever else you need to
Reply
#3
(Jun-12-2020, 09:01 AM)DreamingInsanity Wrote: What it looks like you are trying to do is 'open' a directory to get all the images, so you can use them elsewhere.
I would recommend the glob module for that.
import glob

def open_train_data():
    return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters
 
for f in open_train_data():
   dict1 = pickle.load(f)
   ###do whatever else you need to

I ran the code you posted. I got this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 78, in <module>
    dict1 = pickle.load(f)
TypeError: file must have 'read' and 'readline' attributes
It explodes at this point:

dict1 = pickle.load(f)
Reply
#4
Sorry, my mistake - it should work if you do this:
import glob
 
def open_train_data():
    return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters
  
for f in open_train_data():
   dict1 = pickle.load(open(f, 'rb'))
   ###do whatever else you need to
I forgot to open the file.
Reply
#5
(Jun-12-2020, 10:20 AM)DreamingInsanity Wrote: Sorry, my mistake - it should work if you do this:
import glob
 
def open_train_data():
    return glob.glob('/home/mojito/Desktop/CNN/datasets/training_data/images/*') #the '*' is a 'wildcard' and it means to match one or more characters
  
for f in open_train_data():
   dict1 = pickle.load(open(f, 'rb'))
   ###do whatever else you need to
I forgot to open the file.

I am getting this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "source_code_modified.py", line 80, in <module>
    dict1 = pickle.load(open(f, 'rb'))
_pickle.UnpicklingError: A load persistent id instruction was encountered,
but no persistent_load function was specified.
I am trying to solve it though, but I thought posting it for more help. Thank you...
Reply
#6
Any idea?
Reply
#7
(Jun-12-2020, 05:05 PM)hobbyist Wrote: Any idea?
It seems pickle.load() is for use when unpickling. If you are trying to pickle an image, you want to use pickle.dumps() which returns a pickled object.
If you want to save the pickled object to a file, you could use pickle.dump(f, open("myfile.data", "wb"))
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Recommended way to read/create PDF file? Winfried 3 2,784 Nov-26-2023, 07:51 AM
Last Post: Pedroski55
  python Read each xlsx file and write it into csv with pipe delimiter mg24 4 1,309 Nov-09-2023, 10:56 AM
Last Post: mg24
  read file txt on my pc to telegram bot api Tupa 0 1,048 Jul-06-2023, 01:52 AM
Last Post: Tupa
  parse/read from file seperated by dots giovanne 5 1,043 Jun-26-2023, 12:26 PM
Last Post: DeaD_EyE
  Formatting a date time string read from a csv file DosAtPython 5 1,161 Jun-19-2023, 02:12 PM
Last Post: DosAtPython
  How do I read and write a binary file in Python? blackears 6 6,015 Jun-06-2023, 06:37 PM
Last Post: rajeshgk
  Pyserial Readline() conversion to array bprosman 1 1,814 Apr-11-2023, 12:44 AM
Last Post: deanhystad
  Read csv file with inconsistent delimiter gracenz 2 1,140 Mar-27-2023, 08:59 PM
Last Post: deanhystad
  Greek letters with .readline() and tkinter KinkgOfKeks 7 1,628 Mar-24-2023, 05:13 PM
Last Post: deanhystad
Star Pyserial not reading serial.readline fast enough while using AccelStepper on Arduino MartyTinker 4 3,864 Mar-13-2023, 04:02 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020