Convert String of an int array to a Numpy array of ints - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: Convert String of an int array to a Numpy array of ints (/thread-33211.html) |
Convert String of an int array to a Numpy array of ints - mdsousa - Apr-06-2021 Hi I am sending a Numpy int array over RabbitMQ and it seem to be receiving correctly. This is the output RabbitMQ's publish that is sent: >py msg_send_test.py This is the output on the receiving end from RabbitMQ's consumer:>py msg_receive_test.py This is the section of code to produce the output from the RabbitMQ consumer:message = body.decode() parts = message.split(',') print("[x] message %r" % message) print("[x] message type %s" % type(message)) print("[x] parts %s" % parts) print("[x] parts len %s" % len(parts)) imageStr = str(parts[4]) print("[x] imageStr %s" % imageStr) print("[x] imageStr type %s" % type(imageStr))What I would like to do is convert the imageStr into a Numpy array of ints. The whole array is a string, not an array of individual strings. Any suggestions would be appreciated. Thanks... When I try using Numpy's fromstring method: imageArr = np.fromstring(imageStr, dtype=float, sep=' ') print("[x] imageArr type %s" % type(imageArr)) print("[x] ", imageArr)it gives the following output:
RE: Convert String of an int array to a Numpy array of ints - mdsousa - Apr-06-2021 I'm trying this out. In the output from fromstring, I added the following imageStr = str(parts[4]) imageStr = imageStr.replace('[','') imageStr = imageStr.replace(']','') imageArr = np.fromstring(imageStr, dtype=int, sep=' ') print("[x] imageArr type %s" % type(imageArr)) print("[x] imageArr ", imageArr)And this is now the output: Looks like the '[]' had to be removed from the string, and Numpy puts them back when converting to the int array.
RE: Convert String of an int array to a Numpy array of ints - mdsousa - Apr-06-2021 I have found that this only works up to some limits. If the array becomes too large, it only handles the first three elements, I've been able to get it to work up to a 1000 element array. It does not work correctly at 10000 elements. Any idea why the conversion doesn't work? Thanks... RE: Convert String of an int array to a Numpy array of ints - mdsousa - Apr-07-2021 In sending a message (Numpy int array) using RabbitMQ, this is how it is setup and sent testImage = np.random.randint(0, 255, 10*1000) # grey scale images strImage = np.array_str(testImage) message = <string of meta data> + strImage channel.basic_publish(exchange='', routing_key='task_queue', body=message, properties=pika.BasicProperties(delivery_mode=2, # make message persistent ))This is the output of the message sent: In modifying the received message (from rabbitMQ) to get a Numpy int array I've tried:message = body.decode() parts = message.split(',') print("[x] message %r" % message) print("[x] message type %s" % type(message)) print("[x] parts %s" % parts) print("[x] parts len %s" % len(parts))This is the output for the above code: I then start working of the message part that is the Numpy array:imagePart = parts[4] print() print("[x] imagePart ", imagePart) print(repr(imagePart)) print(re.sub("\s+", ",", imagePart.strip())) print(type(re.sub("\s+", ",", imagePart.strip()))) print(np.array(re.sub("\s+", ",", imagePart.strip()))) print(np.array(re.sub("\s+", ",", imagePart.strip())).size) print(type(np.array(re.sub("\s+", ",", imagePart.strip())))) print(repr(np.array(re.sub("\s+", ",", imagePart.strip())))) print()The output for the above code block is (I'm not sure why it thinks it's of dtype 'U27'): I then try a different approach:npPart = np.asarray(imagePart, dtype=np.str) npSplit = np.char.split(npPart) npSplitPart = np.array(npSplit, dtype=object) print(npSplitPart) print(npSplitPart.dtype) print("[x] npSplitPart type ", type(npSplitPart)) print("[x] npSplitPart shape ", npSplitPart.shape) print("[x] npSplitPart size ", npSplitPart.size) print(repr(npSplitPart)) print(np.array(npSplitPart))This is the output for the above code block: What I would like is to get back to the original Numpy int array. Any idea what I am doing wrong or missing?Thanks... RE: Convert String of an int array to a Numpy array of ints - SheeppOSU - Apr-07-2021 I think it would be easier to do something like this: import numpy as np import json testImage = np.random.randint(0, 255, 10*1000) metadata = "" data = {"image": testImage, "metadata": metadata} string_data = json.dumps(data) # a string format of the data # Receiving end data = json.loads(string_data) metadata = data["metadata"] image = data["image"] RE: Convert String of an int array to a Numpy array of ints - mdsousa - Apr-08-2021 Thanks SheeppOSU, I had tried something like that (if '{}' create a list as that is what I did. I read that lists were serializable out of the box. #sending listImage = testImage.tolist() msgList = [imageID, timeStampSeconds, timeStampNano, int(testImage.size)] msgList += listImage json.dumps(msgList) #receiving message = json.loads(body) recvBody = list(message) arr = np.array(msgList[4:len(msgList)]) # metadata is len 4Thanks again... |