Python Forum

Hey everyone,

I am currently struggling with the structure of data I receive. As a result of another (not important here) function I get a list filled with numpy arrays at a large scale (about 20 thousand entries):

first_array=[array([[1, 0, 2, 4, 3]], dtype=int64),
 array([[2, 6, 8, 3, 3]], dtype=int64),
 array([[0, 0, 4, 3, 3]], dtype=int64),
 array([[4, 4, 3, 2, 3]], dtype=int64),
...]

For the further procession of this data I need this input in the reverse structure: An array of lists like the following form:

output_array=array([
       [1, 0, 2, 4, 3],
       [2, 6, 8, 3, 3],
       [0, 0, 4, 3, 3],
       [4, 4, 3, 2, 3],
....])

Is there a way of doing this in a way that keeps the computational time low?

If you change "array" to "np.array" and int64 to "int", then at least the code works:

import numpy as np

Input=[np.array([[1, 0, 2, 4, 3]], dtype=int),
       np.array([[2, 6, 8, 3, 3]], dtype=int),
       np.array([[0, 0, 4, 3, 3]], dtype=int),
       np.array([[4, 4, 3, 2, 3]], dtype=int)]

Input2Array = np.asarray(Input).reshape(4, 5)

print(Input2Array)

Output:
[[1 0 2 4 3]
 [2 6 8 3 3]
 [0 0 4 3 3]
 [4 4 3 2 3]]

Just a comment - input is a terrible name for a variable, as it overrides the standard function. You can't use input() after that...

(Jul-19-2022, 05:21 PM)paul18fr Wrote: [ -> ]If you change "array" to "np.array" and int64 to "int", then at least the code works:

import numpy as np

first_array=[np.array([[1, 0, 2, 4, 3]], dtype=int),
       np.array([[2, 6, 8, 3, 3]], dtype=int),
       np.array([[0, 0, 4, 3, 3]], dtype=int),
       np.array([[4, 4, 3, 2, 3]], dtype=int)]

Input2Array = np.asarray(Input).reshape(4, 5)

print(Input2Array)

Output:
[[1 0 2 4 3]
 [2 6 8 3 3]
 [0 0 4 3 3]
 [4 4 3 2 3]]

Is there also a possibility if it is not possible to change the numpy array to int-type?

Please pay attention to post by jeffsummers. It is important, taboo actually to use reserved words in your code.
input is used by python, and you are redefining it.

I fully agree with jefsummers and Larz60+, and that's why I've changed in first letter in upper case ... but it may lead to a misunderstanding in a future!

@Cola_Reb: it might be interesting to show how did you make your list

It is quite complicated to reproduce this in a simplified way but I will give it a try. I hope it will be clear.

First of all I have an array of the form:

first_array=np.array([0, 0, 0, 0, 0, 1, 2, 2, 2, 3, 3, 4], dtype=int64)

Then I have a function_a that takes this array and calculates a value for each entry which is summed up in a 1-dimensional array with the length of 5. It is hard to find an easy example for this calculation without posting a ton of code, so this is replaced here by a simple random_function, that returns an array of dtype=64 (not modifiable unfortunately):

def function_a(first_array):
    
    total_results = [0 for i in range(5)]

    for row_index in range(first_array.shape[0]):

        total_results += random_function([row_index])
    
    return total_results



def random_function(entry_array):
        
    result_for_entry = np.random.choice([0, 1], size=5, p=[.1, .9])
    result_for_entry = result_for_entry.astype(np.int64)
    
    return result_for_entry


function_a(first_array)

Output:
array([ 9, 11, 10, 12, 11], dtype=int64)

But I have to to this for a larger data set of this example form (here only 5 arrays are assumed, in fact there are 15.000):

testdata=np.array([array([0, 0, 0, 0, 0, 1, 2, 3, 3, 3, 4, 4], dtype=int64),
       array([0, 1, 2, 2, 2, 3, 3, 4, 4], dtype=int64),
       array([0, 0, 1, 3, 4], dtype=int64), array([0, 0], dtype=int64),
       array([0, 1, 2, 2, 4, 4], dtype=int64)], dtype=object)

For this purpose I wrote function_b which results in the mentioned result structure of a list of arrays:

def function_b(testdata):

    results = [[0 for i in range(5)] for k in range(5)]
    
    for i in range(5):
        results[i] = function_a(testdata[i])
       
        
    return results

function_b(testdata)

Output:
[array([12, 10, 11, 12, 11], dtype=int64),
 array([8, 7, 9, 8, 7], dtype=int64),
 array([5, 5, 5, 5, 5], dtype=int64),
 array([2, 2, 2, 2, 2], dtype=int64),
 array([6, 6, 6, 6, 5], dtype=int64)]

I hope I explained the structure in an understandable way. Help is much appreciated how I can get the final results as an array of lists.

Thank you!