Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
struct.unpack failed
#1
Hello,

I am biginning coding with python and I have 2 errors I could not solve alone :

I have imported files (like MNIST\\train-images-idx3-ubyte.gz), and I try to unpack the file with the code :

import numpy as np
import gzip
import struct

def load_images(filename):
    # Open and unzip the file of images :
    with gzip.open(filename, 'rb') as f:
        # read the header, information into a bunch of variables:
        _ignored, n_images, image_columns, image_rows = struct.unpack('>IIII', bytearray)
        # read all the pixels into a long numpy array :
        all_pixels = np.frombuffer(f.read(), dtype=np.uint8)
        # reshape the array into a matrix where each line is an image:
        images_matrix = all_pixels.reshape(n_images, image_columns * image_rows)
        # Add a bias column full of 1 as the first column in the matrix
        return np.insert(images_matrix, 0, 1, axis=1)


# 60000 images, each 785 elements (1 bias + 28x28 pixels)
X_train = load_images("\\MNIST\\train-images-idx3-ubyte.gz")
X_train = load_images("\\MNIST\\t10k-images-idx3-ubyte.gz")
but it returns me these errors :
Error:
Traceback (most recent call last): File "recognition.py", line 23, in <module> X_train = load_images("\\MNIST\\train-images-idx3-ubyte.gz") File "recognition.py", line 13, in load_images _ignored, n_images, image_columns, image_rows = struct.unpack('>IIII', bytearray) TypeError: a bytes-like object is required, not 'type'
I have tried to look at google and else, but no way to find an answer (I have just started used Python 10 days ago !).
May you help me !
Thanks

Roro noa
Reply
#2
The error message says it wants a bytes-like object that contains the packed data. You gave it a type. What you probably want to do is read some bytes from your file and unpack the bytes you just read.
Reply
#3
On the line that's erroring (line 9), you are using struct.unpack. You are passing in "bytearray". In python bytearray is a type that an object can be.
It seems you want to unpack the header of the file. In that case you would want to read the file into a byte array and pass that in instead.
This should work:
b_array = bytearray(f.read())
_ignored, n_images, image_columns, image_rows = struct.unpack('>IIII', b_array)
EDIT: I did a quick test. For me, running that code gave me an error saying it wanted a 16 byte buffer. I had to do this instead:
_ignored, n_images, image_columns, image_rows = struct.unpack('>IIII', bytearray(a.read()[:16]))

You may need that, you may not.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Too much values to unpack actualpy 3 410 Feb-11-2024, 05:38 PM
Last Post: deanhystad
  Python Struct Question, decimal 10, \n and \x0a. 3python 2 648 Aug-11-2023, 09:29 AM
Last Post: 3python
  unpack dict menator01 1 1,158 Apr-09-2022, 03:10 PM
Last Post: menator01
  ValueError: not enough values to unpack (expected 4, got 1) vlearner 2 6,279 Jan-28-2022, 06:36 PM
Last Post: deanhystad
  JS Buffer.from VS struct.pack DreamingInsanity 3 2,409 Apr-05-2021, 06:27 PM
Last Post: DreamingInsanity
  [SOLVED] [geopy] "ValueError: too many values to unpack (expected 2)" Winfried 2 2,835 Mar-30-2021, 07:01 PM
Last Post: Winfried
  Cannot unpack non-iterable NoneType object, i would like to ask for help on this. Jadiac 3 8,819 Oct-18-2020, 02:11 PM
Last Post: Jadiac
  subprogram issues: cannot unpack non-iterable function object error djwilson0495 13 5,865 Aug-20-2020, 05:53 PM
Last Post: deanhystad
  Pack integer values as single bytes in a struct bhdschmidt 3 2,277 Jun-09-2020, 09:23 PM
Last Post: bhdschmidt
  struct.decode() and '\0' deanhystad 1 3,144 Apr-09-2020, 04:13 PM
Last Post: TomToad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020